mjhjmtu

I have a single source file which has multiple entries of this format:

some_id: some string

And a bunch of .txt files in different folders that have strings that are identified by id

id="some_id"

How do I parse the source file in a way so that each string in .txt file which has id matching one of source file be added a new parameter like so

id="some_id" some_param="some string"

which is taken from value of corresponding source file ID

Notes

• There can be multiple IDs in same file


• There are never duplicated IDs in any file


• They are never located on the same line


• 
  some_param is a literal


• it does't matter where to add new some_param="string" as long as it is on the same line and spaced out

Some sample

source.yml

apples: Apples are very tasty
grapes: Grapes kind of ok
trees: Gain mass from CO2
strawberry: Yummy

fruits/foo.txt

This is some plain text
Toreplace id="apples" string contains
The most vitamin-rich berry is id="strawberry"

fruits/strange/grapes.txt

And id="grapes" the

questionable/trees.txt

Or id="tress"

Given this sample, script should turn fruits/foo.txt into:

This is some plain text
Toreplace id="apples" string contains some_param="Apples are very tasty"
The most vitamin-rich berry is id="strawberry" some_param="Yummy"

Complex Grep + GNU Awk solution:

The crucial awk script content, let's say add_param_by_id.awk:

#!/bin/awk -f
BEGIN FS=":[[:space:]]+" 
NR==FNR a[$1] = $2; next 
match($0, /<id="([^"]+)"/, b) && b[1] in a 
 sub(/<id="[^"]+"/, "& some_param=42"a[b[1]]"42") 
1

The main commands:

export pat="($(cut -d':' -f1 source.yml | paste -s -d'|'))"
grep -ElZr "<id="$pat"" --include=*.txt | xargs -0 -I sh -c 
'awk -f add_param_by_id.awk source.yml "$1" > "/tmp/$$" && mv "/tmp/$$" "$1"' _ 

• 
  pat - variable containing regex alternation group with identifiers i.e. (apples|grapes|trees|strawberry)


• 
  grep -ElZr id="$pat" --include=*.txt - prints all filenames which were matched for any of the specified identifiers

Example file content after processing:

This is some plain text
Toreplace string contains id="apples" some_param="Apples are very tasty"
The most vitamin-rich berry is id="strawberry" some_param="Yummy"

With certain caveats, such as "no strange characters in filenames", and "no / characters in matching or substitution data values" this should work for you. It's not terribly efficient, in that it makes one pass through every file it can find for each line in the YAML file, but to fix that I would want more information about the relative size of the YAML file to the number of target files.

while IFS=: read -r id sp
do
 id="id="$id"" sp="some_param="$sp:1""

 find . -type f -name '*.txt' |
 while IFS= read -r file
 do
 sed -i "/<$id/s/$/ $sp/" "$file"
 done
done <source.yml