Grep for matches inside a while loop

grep not working in while loop
grep -v loop
while loop for grep
grep command inside while loop
grep not working in for loop
bash while grep
use grep in bash script
bash while loop

I have a file that contains server names and the number of cpu cores that I extracted from Ansible. I'd like to clean up the appearance of the file by just listing the server_name and the # of cores.

while read line; do

  server_name=`grep -q "SUCCESS" $line | awk '{print $1}'`

  cores=`grep -q "ansible_processor_cores" $line | awk '{print $2}'`

  printf "$server_name has $cores cores\n"

done <ansible_file

When I use grep "SUCCESS" ansible_file | awk '{print $1}' on the cmd line, I get exactly what I am looking for. However, I need to pull both vars out of the file. This is why I am trying to read the file line-by-line.

When run as a script, grep is complaining about "No such file or directory". What am I missing here?

The format of the ansible_file is:

server_name | SUCCESS => {

         "ansible_facts": {
             "ansible_processor_cores": 8
         },
         "changed": false

    }

The output I'm getting is:

grep <server_name>: No such file or directory

grep |: No such file or directory

grep SUCCESS: No such file or directory

grep =>: No such file or directory

etc

Spawning way too many unnecessary processes here.

in awk -

awk '/SUCCESS/ { svr=$1; } /ansible_processor_cores/ { if(svr){ printf "%s has %s cores\n", svr, $2; } } /\} *$/ { svr=""; }' ansible_file

in perl -

perl -ne '/^(\S+).*SUCCESS/ and $msg="$1 has"; /"ansible_processor_cores": (\d+)/ and $msg and print "$msg $1 cores\n"; /} *$/ and $msg="";' ansible_file

In sed -

sed -En '/SUCCESS/,/^ *} *$/ {
  /SUCCESS/{ s/ .*//; h; }
  /ansible_processor_cores/{ s/.*ansible_processor_cores"*: ([0-9]+).*/\1/; H; }
  /^ *} *$/{ x; s/^(.+)(\n)([0-8]+)$/\1 has \3 cores/; p; x; d; }
}' ansible_file

pure bash -

server_name=''; cores='';
while read line; do
  if [[ "${line%% *}" =~ }$ ]]
  then server_name=''; cores='';
  fi
  [[ "$line" =~ SUCCESS ]] && server_name="${line%% *}"
  [[ "$line" =~ ansible_processor_cores ]] && cores="${line##* }"
  if [[ -n "$server_name" && -n "$cores" ]]
  then printf "$server_name has $cores cores\n"
       server_name=''; cores='';
  fi
done < ansible_file

(Also, the -q argument to grep is for quiet, meaning "don't output anything".)

Try those.

Using grep within a while loop - UNIX and Linux Forums, i cannot get any output matches (note that the match is underlined). Is it because of the "|" character in input2.txt? if so, how can i get it to grep the line? Thanks Y. grep returns the whole line from a match. Consequently what you're asking it to do is to return $line if there's a match. From your description, there always will be a match. To extract the values server_name and core from $line with minimal changes to your code:

grep returns the whole line from a match. Consequently what you're asking it to do is to return $line if there's a match. From your description, there always will be a match.

To extract the values server_name and core from $line with minimal changes to your code:

while read line; do

  if [[ $line =~ SUCCESS ]]; then
    server_name=`echo $line | awk '{print $1}'`
  fi;
  if [[ $line =~ ansible_processor_cores ]]; then
    cores=`echo $line | awk '{print $2}'`
  fi;

  printf "$server_name has $cores cores\n"

done <ansible_file



using grep in a while loop - UNIX and Linux Forums, I want a script to loop until a string is identified in a log file. when i use grep -f input1.txt input2.txt i cannot get any output matches (note that the match is� If you have a file which is a list of interesting things, in your case, a list of patches, with one token or word per line, then you may use grep -F (or fgrep) to find those items in other files. you may not need to put grep in a for-loop. here's an example: grep -Ff log.one … list of files

I don't think you should use a while loop. If each file you're parsing has a single server entry, just remove the while loop.

Otherwise, if a file can have two or more entries, I think I would grep with a multiline regex parts of the file.

$ perl -0pe 's/([0-9a-zA-Z_]*).*SUCCESS(?:(?!ansible_processor_cores)(.|\n))*ansible_processor_cores[^\d]*(\d*)(?:(?!\}\n)(.|\n))*}/Server \1 has \3 cores/g; s/.*(FAIL|ERROR)(?:(?!}\n)(\n|.))*}\n//;' data1.txt

It produces:

Server server_name has 8 cores
Server server_name has 8 cores

From data:

server_name | SUCCESS => {

 "ansible_facts": {
     "ansible_processor_cores": 8
 },
 "changed": false

}
server_name | FAIL => {

 "ansible_facts": {
     "ansible_processor_cores": 99
 },
 "changed": false

}
server_name | SUCCESS => {

 "ansible_facts": {
     "ansible_processor_cores": 8
 },
 "changed": false

}

Explained: I use two regexs, one to change matching data and one to remove unmatching data.

Matching: s/([0-9a-zA-Z_]*).*SUCCESS(?:(?!ansible_processor_cores)(.|\n))*ansible_processor_cores[^\d]*(\d*)(?:(?!\}\n)(.|\n))*}/Server \1 has \3 cores/g; Not matching: s/.*(FAIL|ERROR)(?:(?!}\n)(\n|.))*}\n//;

Matching ([0-9a-zA-Z_]*) is the server name, but is only included if it's proceeded by SUCCESS. It than searches for the first "ansible_processor_cores" hit with (?:(?!ansible_processor_cores)(.|\n))*ansible_processor_cores. It groups an N amount of numbers after the cores and proceeds to include everything until a single } is found on a line (which is closing the "ansible server node").

Not matching Takes everything matching FAIL or ERROR and finds the closing character }. If found it replaces it with nothing.

Loop until grep does not find the text in a file, #!/bin/bash while grep "sunday" file.txt > /dev/null; do sleep 1 echo "working" done. Why exactly are you piping sleep 1 to echo ? Though it� >for i in stringsfile.txt; do grep -e filename.txt; done-Returns results as if it's not obeying the '-e' flag. What I'm trying to accomplish is use a text file containing a list of strings, come which contain spaces in between and some which do not, and feed them into a loop to search for each exact match of the string in another file.

I apologize for the delay in responding. I took some time to deepen my knowledge and experience with sed. I was ultimately able to use sed to get what I needed.

Thank you to everyone that helped increase my bash knowledge. I learned something from each of you.

Using Grep In A For Loop, while read -r line; do #Get Error Logs grep "$line" /home/eximlog The -f options tells grep to read patterns from a file, one pattern per line. Run a FOR /F loop against filename.cfg.hst, use the double quote as a separator, take out occurance number 2, which is the device name, and grep that against /tmp/namelist You can use exitcodes to see if any occurance matches

1. Loops, #!/bin/bash # bin-grep.sh: Locates matching strings in a binary file. The bracket construct in a while loop is nothing more than our old friend, the test brackets� Re: Grep not functioning inside for/next or while loops. it appears that the "read" isn't parsing the ip addresses, so the grep is trying to match all of them at once. I get the same results.

[PDF] grep, awk and sed – three VERY useful command-line utilities Matt , In this example, grep would loop through every line of the file "a_file" and print out every upper and lower case as equivalent while matching the search string. using grep in a while loop Hello everybody, I have been searching it, but it seems I am unable to find the correct information, that s why I am asking you guys, hoping somebody get an idea. Here is my problem : I want a script to loop until a string is identified in a log file.

Grep inside the while Loop, I used grep inside the while loop for checking the "matched text". My code is as follows: my @list_match; my @matchArray; open IN, "Data.txt" or� While loop in bash using variable from txt file. linux,bash,rhel. As indicated in the comments, you need to provide "something" to your while loop. The while construct is written in a way that will execute with a condition; if a file is given, it will proceed until the read exhausts. #!/bin/bash file=Sheetone.txt while IFS= read -r line do echo

Comments
  • Good that you have shown your efforts in your question. Please do add samples of input and sample of expected output in your question too with code tags and let us know then.
  • -q option supresses output, are you sure you want it?
  • grep "SUCCESS" ansible_file works because ansible_file is a filename. $line is not a filename. Hence "No such file or directory."
  • @RuudHelderman It is my understanding that grep works on other input than just files. Isn't $line like STDIN?
  • @Rodney grep works on files and streams. $line is neither; it's a string. To turn a string into a stream, either use echo or use a 'here string'.
  • This should output only one line after data is gathered. If your playbook is accessing multiple servers at a time this won't work as I understood it...or is all the info on every line when it does match?
  • I tried this, but I'm getting a syntax error near unexpected token "${line[*]}". Also, the closing paren without an opening paren confuses me.
  • Yes, this is actually just an adhoc cmd, but it is accessing many servers.
  • The second example above worked, but only read the first line from the file. It did not interate through the rest of the file.
  • I forgot to mention that I moved the printf statement to be inside the while loop.
  • Yes, I am only interested in the lines that match, but not every line will match. Only the 1st and 3rd lines contain the information I am after.
  • Sorry - I responded too quickly. I've edited to something which I think will do the trick!
  • I was able to get part of this work, but not all of it. The $server_name var was correctly assigned, but not the $cores var. I have not been able to get awk to work like it did with $server_name. I don't understand why, though.
  • I have tested it all again based on your input. If you do this echo "\"ansible_processor_cores\": 8" | awk '{print $2}' you'll see that it returns 8, so perhaps your input is different?