mjhjmtu

My situation is that I have to search my entire cluster for files with .xyz extension and the sed/grep/awk/? off only records that contain two distinct values, "CONFNAME" and "INPUTSQL", respectively.

problems:

1. Structuring the proper pipeline that will return this information.

2. Cutting the 50 lines of junk out between these two values and formatting as csv with two columns.

example file:

1_TOD_DO_ON.x
1 !prototype|P|||$AIDQ_MP/dA_lookIP.mp
2 CONFNAME|$|||ITEST_New_Deal
...
...
50 INPUTSQL||||Select n 'ITOD|L1235'

The two pieces of data needed from these .zyz files are ITEST_New_Deal and ITOD|L1235 (comma separated).

Some of what I tried for problem 1

cd To_Correct_Search_Path

A.

find . -type f -iname "*.xyz" -exec egrep -i CONFNAME --color=auto --with-filename ; 

//only finds one value

B.

find . -type f ( -iname "*.xyz" -o -name "CONFNAME" -o -name "INPUTSQL" ) -exec sh -c 'echo "$0"' ; >

//only finds the last value if "INPUT_SQL"

C.

find . -type f -iname "*.pset" -exec egrep -i CONFNAME egrep -i INPUTSQL --color=auto --with-filename ; 

//only finds the last value

for problem 2

To remove the garbage between the to values I want to key off of these two distinct values but include those values in information returned by the search. I have some ideas here but I need problem 1 to be fixed first. Definitely open to suggestions!

Regarding your first problem:

You can combine find and grep. And you can also combine multiple grep. The first grep doesn't print anything (-q) and the last only prints the file if the other has passed.

This will find only files containing both strings:

find . -type f -iname "*.xyz" 
 -exec sh -c 'grep -qi CONFNAME "" && grep -li INPUTSQL ""' ;

The second problem is probably worth a new question when you get stuck. Probably good for awk.