mjhjmtu

I have a directory with multiple subfolders that all contain several text files which are formatted in the following way.

data01:data02

I need to extract just the data02 after the : and export this to a single file in the root directory. I got the extraction in place, but how can I run this on multiple directories and files?

You can customize the following command line with "find" and "awk"

find FOLDERLIST -type f -iname "PATTERN" 
 -exec awk -F":" 'NF>1 print $2' "" ; > /PATH/TO/RESULTFILE

where

• FOLDERLIST is a space separated list of top folders you want to recursively search, whereby the "current folder" would be a dot: find . -type f ...


• "-type f" for searching files only


• PATTERN is the common pattern of the files you are interested in, e.g. an asterix "*" will find all files, "*.csv" will find CSV files, ...


• /PATH/TO/RESULTFILE is the name of your result file in root directory


• the "awk" part splits all found files at the ":" and skips empty results

EDIT: adjusted empty results check to NF>1 as suggested by steeldriver.

You have not provided the format of the output file, so I am writing assuming that you want second field in separate line. You can use(assuming all files are in the format a:b as you said in your question):

find directory -type f 
-exec awk -F: 'print $2' "" >> /output.txt ;

It will find files in directory and in it's sub-directory, and will execute command awk -F: 'print $2' "" >> /output.txt, which will write value after :, to a file /output.txt.