mjhjmtu

How to append multiple lines to a file, if these lines doesn't exist in that file?

For example, to add multiple global aliases to /etc/bash.bashrc I use an heredocument:

cat <<-"BASHRC" >> /etc/bash.bashrc
 alias rss="/etc/init.d/php*-fpm restart && systemctl restart nginx.service"
 alias brc="nano /etc/bash.bashrc"
BASHRC

I was criticized that this operation doesn't include a way to check if the lines are already there, and if mistakenly rexecute the heredocument, I could cause redundancy, as well as conflict.

Update

Please ensure not to miss the Bounty message.

Simple shell script to add lines from the file newdata to datafile. It should be straightforward to change newdata to a here-doc. This is really not very effective since it calls grep for every (new) input line:

target=datafile
while IFS= read -r line ; do
 if ! grep -Fqxe "$line" "$target" ; then
 printf "%sn" "$line" >> "$target"
 fi
done < newdata 

For each line, we use grep to see if it already exists in the target file, -F for fixed-string match (no regexes), -x for full line match, and -q to suppress output of matched lines. grep returns a falsy error code if it doesn't find a matching line, so append to target file if the negated result is truthy.

More effectively, in awk. This relies on awk being able handle arbitrary lines as keys to an array.

$ awk 'FNR == NR lines[$0] = 1; next ! ($0 in lines) print' datafile newdata 

The first part FNR == NR lines[$0] = 1; next loads all lines of the first input file as keys into the (associative) array lines. The second part ! ($0 in lines) print runs on following input lines, and prints the line if it's not in the array, i.e. the "new" lines.

The resulting output contains the new lines, only, so it needs to be appended to the original file, e.g. with sponge:

$ awk 'FNR == NR lines[$0] = 1; next ! ($0 in lines) print' datafile newdata | sponge -a datafile

Or we could have awk append the lines to the final line, it just requires passing the file name to awk:

$ target=datafile 
$ awk -vtarget="$target" 'FNR == NR lines[$0] = 1; next 
 ! ($0 in lines) print >> target' "$target" newdata

To use a here-doc with awk, we'll need to add - (stdin) as an explicit source file, in addition to setting the redirection, so awk ... "$target" - <<EOF

This solution is probably a little different than what you had in mind but I'd check whether an alias is actually defined and only add it to bashrc if it isn't. Assuming you are using a relatively current version of Bash...

# instead of 'alias x=y' form put them in an associative array
declare -A aliases
aliases['a']='apple'
aliases['ba']='banana'

for key in "$!aliases[@]"; do
 if ! alias "$key" > /dev/null 2>&1; then
 printf "alias %s='%s'n" "$key" "$aliases[$key]" >> /etc/bash.bashrc
 fi
done

unset aliases key

Source this . ./scriptname rather than run it as a normal script.

Note: If you have a bunch of the candidate aliases and are wary of manually converting them to associative array form then you c edit the file with vim and run this commmand: :%s/valias *([^=]+)=['"]?([^'"]+)['"]?$/aliases['1']='2'/.

OR put your candidate aliases (and nothing else) in a file (e.g. /tmp/aliases.txt) and replace the aliases[..]='..' lines in the script with this:

while IFS= read -r a; do 
 eval "$a"; 
done < <(sed -E "s/alias *([^=]+)=['"]?([^'"]+)['"]?$/aliases['1']='2'/" /tmp/aliases.txt)

Of course, there are other ways besides using AAs but they're clean and easy to work with...and I'm all in at this point. :)

A generic solution for exact matches (not optimized for performance); the file input contains the lines to be checked for:

awk 'FNR==NR lines[NR]=$0; next; ;
 for(i=1;i<=length(lines);i++) if ($0==lines[i]) matches[i]=1; print; ;
 END for(i=1;i<=length(lines);i++)
 if (matches[i]==0) print lines[i]; ' input file

Testing:

:> cat input
alias rss="/etc/init.d/php*-fpm restart && systemctl restart nginx.service"
alias brc="nano /etc/bash.bashrc"

:> cat file
a
b
c
alias rss="/etc/init.d/php*-fpm restart && systemctl restart nginx.service"
d

Output of the awk command:

a
b
c
alias rss="/etc/init.d/php*-fpm restart && systemctl restart nginx.service"
d
alias brc="nano /etc/bash.bashrc"

Does the order of appended lines matter?

If no, you can try such a solution:

comm -1 -3 <(sort /etc/bash.bashrc) <(sort aliases) >> /etc/bash.bashrc

assuming you have previously saved lines to add in the file aliases.

comm is GNU coreutil (you can see the man page for it), which allows comparing two sorted files line by line.

Hmmmm. How about this...

Put the aliases to add in file new. Then...

cp bashrc bashrc.tmp && comm -23 <(sort -u new) <(sort -u bashrc.tmp) >> bashrc && rm -f bashrc.tmp

We unique sort both the new aliases and the bashrc contents (put in a temp file to avoid race condition when we append to bashrc) and run them through comm. comm does line-by-line comparisons of sorted files and shows unique and common lines. We suppress columns 2 and 3 (-23) so the result is simply those lines that are unique to new and we append those to bashrc. That's it.

(This is a totally different approach than my other answer, focusing on your quest for more of a "one-liner".)