mjhjmtu

I'm working on a script that will run a checksum process which outputs to STDOUT, which I then want to grep for lines matching OK, or FAILED and do different things with those matches (i.e. output to terminal and log). I've watched a ton of Youtube videos and read a ton about redirection, but I just can't seem to wrap my head around how exactly redirection works. What I'm trying to do is chain STDOUT to multiple greps without them gobbling up the non-matched text.

Here's a concept of what I'm trying using cat instead of md5sum with a text file of animal names on each line (DOG, CAT, PONY, RHINO, DEER, FOX):

cat test.txt 3>&1 3> results.txt

This does what I expect. What I understand here is I'm doing a cat on the file, and then opening fd3 which points to whatever is written to STDOUT(fd1). Since the grep will gobble up everything from fd1, I tee the STDOUT of cat explicitly to fd3 and then pipe the STDOUT to grep. Grep will print out the line matching DOG, and then all the text written to fd3 from cat will get pushed to a results.txt file.

Now, to chain another grep to look for other text I have to point the fd3 data back into STDOUT, tee it explicitly back to fd3 and then pipe STDOUT to a new grep.

tee /dev/fd/3 3> results.txt

The first problem here is the STDOUT from the first grep is being pushed into fd3 a second time instead of printing to the terminal. So now my results.txt is getting duplicates and I never got anything printed to the screen for the first grep. This is where my understanding of redirections is falling apart. I sort-of get what's happening but I can't figure out a simple solution.

I want to grep STDOUT, print the results to screen, and pass the original text to another grep, and maybe a third, fourth, etc without modifying the original text I'm passing to each GREP, and without each subsequent grep eating up the previous' match that should print to screen.

I could probably do this by storing a variable and calling it on multiple lines of greps, but then I have to wait for the entire first command to complete. In the case of the application I'm working on, I want to see realtime results during a checksum, not just a blank screen for an hour until the whole process is complete. Any clarification on what I'm doing wrong would be super helpful, thanks!

EDIT

I understand this exact use of cat is pointless, I just used it to demonstrate the concept. In the script I'll be applying the concept to, the first command is actually:

md5sum -c checksum.md5

Which will read a checksum file, re-hash the the source and output to STDOUT a pass/fail line. I then want to grep this stream and send the results to separate logs and/or terminal output - but cat seemed like a simpler way to demonstrate the problem as this can be applied to filtering any command and grepping the stream, such as find, md5, ls, etc.

You can better do what you ask for with process substitution:

1. 
   

   Being as close as your original command as possible:

   
   
   

   cat test.txt | tee >(grep DOG) >(grep PONY) >results.txt
   

   
   


2. 
   

   Removing the useless use of cat:

   
   
   

   <test.txt tee >(grep DOG) >(grep PONY) >results.txt
   

   
   
   

   Or:

   
   
   

   tee >(grep DOG) >(grep PONY) <test.txt >results.txt
   

   
   

isaac's solution is better but your way would look like this:

 <input tee results.txt /dev/fd/3 3>&1 |
 grep PONY >&2; 3>/dev/null

or for three

 <input tee results.txt /dev/fd/3 3>&1 |
 grep PONY >&2; 3>&1 |
 tee /dev/fd/3 3>/dev/null