mjhjmtu

I have a file input.txt which contains multiple filename in the below format.
FILENAME_DATE_LINENUMBER , the input.txt contains many such filenames.
The filename itself has precisely 5 underscore.

FILE_NAME_1.DAT_20180123_4
FILE_NAME_2.DAT_20180123_5
FILE_NAME_3.DAT_20180123_6
FILE_NAME_4.DAT_20180123_7

All files are present in sub directory as input.txt. I want to parse input.txt , iterate through each filename and print FILENAME and the specified line number ( from the FILENAME ) to output.txt

I am beginner at shell script and I understand that sed or awk will be used , and below command can do the job.

awk 'FNR==LINENUMBER print FILENAME, $0' *.txt >output.txt

But how can i iterate through the file input.txt and find the FILENAME and extract LINENUMBER from FILENAME to output.txt

The FILENAME specified in input.txt can in one of the sub directories where input.txt is located. There can be only one file with FILENAME in input.txt inside one of the sub directory ( one level ) form the input.txt location.

DIR
├── input.txt
│ ├── DIR1
│ │ ├── FILE_NAME_1.DAT
│ ├── DIR2
│ │ ├── FILE_NAME_2.DAT
│ ├── DIR3
│ │ ├── FILE_NAME_3.DAT

In output.txt it should be printed as

FILENAME
LINE( Extracted from FILENAME present in input.txt )

#!/bin/bash 

do_one() 
 # two args: $1=filename_no_dir $2=line_number 
 # Find the single filename 
 eval file=*"/$1"
 echo $1
 # $. == line number 
 perl -ne 'chomp; $.=='"$2"' and print "LINE($_)n"' $file

export -f do_one

# Generate som test data 
parallel 'mkdir DIR; seq 100 110 >DIR/FILE_NAME_.DAT' ::: 1..4

# Test input.txt 
cat <<EOF | 
FILE_NAME_1.DAT_20180123_4 
FILE_NAME_2.DAT_20180123_5 
FILE_NAME_3.DAT_20180123_6 
FILE_NAME_4.DAT_20180123_7 
EOF 
 # Remove _YYYYMMDD.* to get filename, and .*_ to get line number 
 parallel do_one '= s/_201ddddd.*// =' '= s/.*_// ='

Output:

FILE_NAME_1.DAT
LINE(103)
FILE_NAME_2.DAT
LINE(104)
FILE_NAME_3.DAT
LINE(105)
FILE_NAME_4.DAT
LINE(106)

:> awk -F_ ' print $1; print $3; ' inputfile
FILE1.DAT
4
FILE2.DAT
5
FILE3.DAT
6
FILE4.DAT
7

If I'm understanding you correctly,

while IFS=_ read -r filename unuseddate linenum
do
 printf "%sn" "$filename"
 sed -n "$linenump;q" */"$filename"
done < input.txt > output.txt

This reads one line at a time from input.txt, splitting the line into 3 parts based on the underscore. It prints the filename then fires off a sed command that (by default prints nothing) and then on the specified line number, prints the line and quits that invocation of sed. The location of the filename should be in one of the immediate subdirectories of the current directory.

All of the output is then redirected to output.txt.

Complex solution with GNU parallel + find + awk:

Let's say each input file has a content similar to the following:

cat DIR1/FILE1.DAT_20180123_4
FILE1 a 
FILE1 b 
FILE1 c 
FILE1 d 
FILE1 e 
FILE1 f 
FILE1 g

So, by the above scheme, the 2nd line in file FILE2.DAT_20180123_5 would be FILE2 b and the 7th line in file FILE4.DAT_20180123_7 - FILE4 g.
Assume input.txt file is the same as in the question.

The job:

find . -type f -regextype posix-egrep -regex ".*/($(paste -s -d'|' input.txt))" 
| parallel -j0 "awk -v n="=s/.*_//=" -v fn="/" 
 'NR==n print fn,$0; exit ' " > output.txt

The final output.txt contents:

$ cat output.txt 
FILE4.DAT_20180123_7 FILE4 g
FILE3.DAT_20180123_6 FILE3 f 
FILE1.DAT_20180123_4 FILE1 d 
FILE2.DAT_20180123_5 FILE2 e