mjhjmtu

I have multiple files (apx. 150) that look like this:

reconstructed_hap_4_Local_nt_haplo_freq_60.3 GGGCAACTGGGCCAAGGTCGCTATCATCATGGTTATGTTTTCAGGGGTCGATGCCAATACATATATCACCGGTGGCAAAGCAGCTCAAACTGCCAGAGGCCTTGTTGGCTGGTTTAATCCGGGTCCCAAACAGAACCTGCAGCTGGTCAACACCAATGGCTCGTGGCA
reconstructed_hap_6_Local_nt_haplo_freq_37.2 GGGCAACTGGGCCAAGGTCGCTATCATCATGGTTATGTTTTCAGGGGTCGATGCCGAAACATATGCCTCCGGTGGCAGTGCAGCTCGTAATACCTG-GGCCTTTCTAGCTTGTTTAGTTCGGGTCCCAAACAGAGCCTGCAGCTGGTCAACACCAATGGCTCGTGGCA
reconstructed_hap_1_Local_nt_haplo_freq_0.6 GGGCAACTGGGCCAAGGTCGCTATCATCATGGTTATGTTTTCAGGGGTCGATGCCAATACATATATCACCGGTGGCAAAGCAGCTCAAACTGCCAGAGGCCTTGTTTGGCTGTTTAATCCGGGTCCCAAACAGAACCTGCAGCTGGTCAACACCAATGGCTCGTGGCA

Each file has different number of lines.

I would like to filter from each files numbers from title line after "freq_"

In this example, I would like to filter: 60.3, 37.2, 0.6

The most preferred output should be a CSV file having each sample name

Filename1 60.3 37.2 0.6 
Filename2 56.1 26.2 52.3 42.1
Filename3 2.5 1.2

Do you have any solutions?

Try this,

cd /path/to/directory
for i in `ls`
do
 VALUE=`awk 'print $1' $i | awk -F '_' 'print $NF' | tr 'n' 't'`
 echo -e "$it$VALUE" 
done

With GNU Awk:

awk '
 BEGINFILE i=0 
 
 n=split($1,a,"_")
 freqs[i++] = a[n]
 
 ENDFILE 
 printf FILENAME
 for (j=0;j<i;j++) printf("t%s", freqs[j])
 printf "n"
 delete freqs
 
' Filename*

Ex.

$ awk 'BEGINFILEi=0; n=split($1,a,"_"); freqs[i++] = a[n] ENDFILEprintf FILENAME; for (j=0;j<i;j++) printf("t%s", freqs[j]); printf "n"; delete freqs' Filename*
Filename1 60.3 37.2 0.6
Filename2 56.1 26.2 52.3

Shell Script:

for file_number in 1..150
do
 data=$( cat file$file_number.txt | cut -f1 -d' ' | cut -f8 -d'_' | tr 'n' 't' )
 #echo $data
 file_name="file$file_number.txt"
 content="$file_name $data"
 #echo $content
 echo $content >> result.csv
done

result.csv file contains the expected result.

EDIT: The following code is better

#!/bin/bash
FILES=/path/to/directory
for file in $FILES
do
 data=$( cat $file | cut -f1 -d' ' | cut -f8 -d'_' | tr 'n' 't' )
 content="$file $data"
 echo $content >> result.csv
done

Explanation

FILES contains all the input files.
Using cut command we get the field (which contains the float number).
Using tr we replace the tabs to new lines.
result.csv file contains your expected result.

With GNU awk (extended command):

awk -F '[ _]' '
 /^[^ ]*_[^ _]* /
 a[FILENAME]=a[FILENAME] " " $(NF-1)
 
 END
 for(i in a)print i,a[i]
 
 ' Filename*

May be executed as a one liner:

$ awk -F '[ _]' '/^[^ ]*_[^ _]* /a[FILENAME]=a[FILENAME] " " $(NF-1)ENDfor(i in a)print i,a[i]' Filename*

Filename1 60.3 37.2 0.6
Filename2 56.1 26.2 52.3