mjhjmtu

I would like to modify the content of 2 different files. How can I obtain the expected output by using a generic script in unix?

1st file:
" inside quoted string,
" and , (separator) inside quoted string

example:

"20181115","12345643","This is a "test"","","657","This is a "TEST"","","aaaa"
"20181115","12345632","This is an "example" of the file, a "sample" aaaa","123","",""TEST"","",""

expected output:

~20181115~;~12345643~;~This is a "test"~;~~;~657~;~This is a "TEST"~;~~;~aaaa~
~20181115~;~12345632~;~This is an "example" of the file, a "sample" aaaa~;~123~;~~;~"TEST"~;~~;~~

2nd file:
| (separator) inside quoted string and multiple " inside the string

example:

"098789"|"Hello world!"| 12,7|"Cities I want to visit Rome| London"|15.11.2018|"Yes"
"032425"|"Travel in ""New York"", USA"| 113,3||15.11.2018|"Yes"

expected output:

~098789~;~Hello world!~; 12,7;~Cities I want to visit Rome| London~;15.11.2018;~Yes~
~032425~;~Travel in /"New York/", USA~; 113,3;;15.11.2018;~Yes~

Try simple sed substitutions for your first problem:

sed 's/","/~;~/g; s/^"|"$/~/g' file

and a more involved awk script for your second:

awk -F" '$1=$1; for (i=2; i<=NF; i+=2) gsub (" 1' OFS="~" file 

It first replaces all | within double quotes with an unusual placeholder not commonly found in text files, then does the desired replacements, then reverses the placeholder substitution.

Please be aware that both are taylored to your problems and thus not generally applicable to other, even similar, problems without adaption.

Output if applied to sample in your question (Ubuntu, mawk 1.3.3 Nov 1996):

~098789~;~Hello world!~; 12,7;~Cities I want to visit Rome| London~;15.11.2018;~Yes~
~032425~;~Travel in /"New York/", USA~; 113,3;;15.11.2018;~Yes~