Compare values within line of file, display line if condition met

how to read a file line by line in unix using while loop
how to read a file line by line in unix using for loop
shell script to read a file line by line and write to another file
while read line if line contains
bash read file line by line for loop
shell script to read a file line by line in unix
bash read file line by line skip first
shell script to read a file line by line and grep

Trying to sort out an issue here. I have a file that includes many lines, but I want to filter for specific line, then compare two values within that line, and if the condition is met, I want to display said line. This would be done through the entire file. I'm looking to use a BASH script for this.

Format of line:

<timestamp>  <date> : <ServerName> <Device> <In>/<Out> <Value1>/<Value2>

Example of line:

15:13:33   12/13/19 : Host1  Device1  In/Out 33/34

Using this, I want to do the following:

  • grep for "In/Out"

  • Separate 33 and 34 in variables

  • Compare 33 and 34

  • If delta between 33 and 34 is more than "X", display entire line to STDOUT.

So far, I have:

#!/bin/bash

input="logfile.log"

while IFS= read -r line
do
     echo $line
done < "$input"

I understand where the echo $line is, I could perform commands. But I'm not sure how to "echo" this and process it within the while loop.

I could do something like:

line=`grep "In/Out" logfile.log`
var1=`grep "In/Out" logfile.log | awk -F" " '{ print $7 }' | awk -F/ '{ print $1 }'`
var2=`grep "In/Out" logfile.log | awk -F" " '{ print $7 }' | awk -F/ '{ print $2 }'`

And then compare, the difference, and if met, echo the $line value, but that feels very inefficient.

Any thoughts/input would be greatly appreciated.


Read a file line by line and if condition is met continue reading till , You need to make some changes to your script (in no particular order):. Use IFS= before read to avoid removing leading and trailing spaces. Read a file line by line and if condition is met continue reading till end. 2. The shortest way to find one unique value when all other values are the same

Such (kind-of-)complicated operations on files are usually done in a single awk script.

awk -v X=10000 '
function abs(v) {return v < 0 ? -v : v}
{ 
   if ($6 == "In/Out") {
      split($7, a, "/");
      if (abs(a[1] - a[2]) > X) {
           print
      }
    }
}' 

The script is almost human readable. First check if 6th field is In/Out. If it is, split 7th field on /, compute absolute value of the numbers and compare them with delta. If they compare "more than X", print the whole line.

Tested on repl. I think you will have to tweak the script to your needs.

If - Conditionally perform command - Windows CMD, File syntax IF [NOT] EXIST filename command IF [NOT] EXIST filename IF will only parse numbers when one of (EQU, NEQ, LSS, LEQ, GTR, GEQ) is used. The == comparison operator always results in a string comparison. ELSE ( This is because CMD does a rather primitive one-line-at-a-time parsing of the command. Use expressions in conditions to check multiple values. 04/15/2019; 7 minutes to read; In this article. In this walkthrough, you'll learn to use expressions and Conditions to compare multiple values in Advanced mode. When you create a flow, you can use the Condition card in basic mode to quickly compare a single value with another value

Assuming the answer you accepted does what you want, all you need is:

awk -F'[[:space:]/]+' '/In\/Out/ && ($NF - $(NF-1)) > 3' file

EDIT: given the sample input from your comment:

$ cat file
02:22:50 11/11/19 : Host1 Device1 In/Out 208/219

$ awk -F'[[:space:]/]+' '/In\/Out/ && ($NF - $(NF-1)) > 3' file
02:22:50 11/11/19 : Host1 Device1 In/Out 208/219

so if the script doesn't work for you then the most likely cause is you're using a pre-POSIX awk (e.g. nawk or a VERY old version of mawk) that doesn't support character classes. Given that, try hardcoding the blank and tab instead of [:space:]:

$ awk -F'[ \t/]+' '/In\/Out/ && ($NF - $(NF-1)) > 3' file
02:22:50 11/11/19 : Host1 Device1 In/Out 208/219

4 Awk If Statement Examples ( if, if else, if else if, :? ), Linux Awk Tutorials - Conditional If Else Statement Examples awk user-defined variables, awk built-in variables, and awk operators. In this example, it gets changed on every 3rd line from a comma to a newline. I spend several hours a day on UNIX / Linux environment dealing with text files (data,� Otherwise, if the number is greater than 4, then assign the value of ‘False’ Here is the generic structure that you may apply in Python: df['new column name'] = df['column name'].apply(lambda x: 'value if condition is met' if x condition else 'value if condition is not met') And for our example:

Conditional Processing with If, Creating and Using Batch Files � Batch File Programming � Displaying Information in Batch Files This is used in combination with command-line variable or After running a command in a batch file, an if statement of the form when comparing strings, and it can also compare arguments and variables as� Specifies a true condition only if the previous program run by Cmd.exe returned an exit code equal to or greater than number. <command> Specifies the command that should be carried out if the preceding condition is met. <string1>==<string2> Specifies a true condition only if string1 and string2 are the same.

if, These values can be literal strings or batch variables (for example, %1 ). If the condition specified in an if clause is true, the command that follows the You must use the else clause on the same line as the command after the if. Examples. To display the message Cannot find data file if the file Product.dat� Conditional Processing with If. One of the most important capabilities of any programming language is the ability to choose from among different instructions based on conditions the program finds as it runs. For this purpose, the batch file language has the if command. The Basic If Command

Use expressions with conditions., When you create a flow, you can use the Condition card in basic mode to quickly compare a single value with another value. However, there're� df1 ['new column that will contain the comparison results'] = np.where (condition,'value if true','value if false') For our example, here is the syntax that you can add in order to compare the prices (i.e., Price1 vs. Price2) under the two DataFrames: df1 ['pricesMatch?'] = np.where (df1 ['Price1'] == df2 ['Price2'], 'True', 'False')

Comments
  • Ouch! Three subshells and two pipes per variable assignment? Also avoid using `...` for command substitution, instead use $(...).
  • Thank you for the reminder to use $(...) - got so used to the using the other its force of habit. Need to break that. By the way, yes, totally agree three subshells is awful. Unfortunately it does "work" just really sloppy!
  • That's also a good approach. Nothing significantly slower than an awk implementation there. Would be interesting to test performance difference between [[ .. =~ .. ]] with the ERE and BASH_REMATCH against a similar awk implementation...
  • @DavidC.Rankin Thank you for the comment. I've compared the execution time with the awk solution using the answer by KamilCuk generating 100,000 line input. Then the awk solution took approx. 0.2seconds while my bash script took 10seconds. I'd admit bash solution is not efficient in time when handling with long lines.
  • Thank you, tshiono. This looks to be a good option to start with. To make things a little uglier, I need to add a little more detail here. The values at the end are enclosed in parentheses and have a trailing period and possibly a space. As such it looks more like: 15:13:33 12/13/19 : Host1 Device1 In/Out (33/40). What's the easiest way to capture just the 33/40 for comparison? Thanks.
  • @DavidC.Rankin - based on tshiono's comment, it seems that awk implementation is faster than BASH_REMATCH? I'm not too worried about speed, but am interested in learning more. Is there another way you would propose this?
  • @pdxwarrior Thank you for the prompt feedback. I've updated my answer by modifying the pattern just prepending \( to the capture group. Would you please try it?
  • That's a good approach. I would have nixed the function and just incorporated the check and split as part of a rule, but there is nothing wrong with the function for reusability.
  • A function would be great, but I'll be honest.. I'm still learning scripting. While I get certain things conceptually, I am still a novice. Not quite sure how to incorporate this to call my file.
  • Just awk ' the script the script' logfile.log. You can replace all the newlines for spaces and make a cute oneliner.
  • Thank you, this was very helpful as well!
  • Thanks for the info, always looking to learn more. There's definitely more efficient ways to write code! I tried the line above and it doesn't work for me, for whatever reason. Let's say my filename is "myfile" - here's the command I used: awk -F'[[:space:]/]+' '/In\/Out/ && ($NF - $(NF-1)) > 3' myfile The line in myfile is: 02:22:50 11/11/19 : Host1 Device1 In/Out 208/219 Also tried with: 02:22:50 11/11/19 : Host1 Device1 In/Out (208/219)
  • Putting parentheses around that final field as in your "Also tried with" case would clearly not work since you'd be corrupting what is expected to just be 2 /-separated numbers. See the edit I just added to my answer.