mjhjmtu

How to I continuously tail -f a log, find all files (sed), and display (cat) the found files

example data in audit logs.

tail -f /var/log/httpd/modsec_audit.log | sed 's/[^/]*/./;s/].*$//g'

output

./apache/20180508/20180508-1428/20180508-142802-WvH6QgoeANwAAMwsFZ4AAAAF
./apache/20180508/20180508-1428/20180508-142803-WvH6QgoeANwAAMwtFfcAAAAG
./apache/20180508/20180508-1428/20180508-142803-WvH6QwoeANwAAMwuFlUAAAAH
./apache/20180508/20180508-1513/20180508-151357-WvIFBQoeANwAAMwnE@4AAAAA
./apache/20180508/20180508-1513/20180508-151357-WvIFBQoeANwAAMwoFD8AAAAB
./apache/20180508/20180508-1516/20180508-151608-WvIFiAoeANwAAMz1FSwAAAAA
./apache/20180508/20180508-1516/20180508-151609-WvIFiQoeANwAAMz2FYIAAAAB
./apache/20180508/20180508-1516/20180508-151611-WvIFiwoeANwAAMz3FeEAAAAC
./apache/20180508/20180508-1516/20180508-151611-WvIFiwoeANwAAMz4Fj4AAAAD
./apache/20180508/20180508-2112/20180508-211205-WvJY9QoeANwAAM1MFCoAAAAA

works with echo

echo "./apache/20180508/20180508-1428/20180508-142802-WvH6QgoeANwAAMwsFZ4AAAAF" | sed 's/[^/]*/./;s/].*$//g' | awk 'print $0' | xargs cat

works with cat

cat /var/log/httpd/modsec_audit.log | sed 's/[^/]*/./;s/].*$//g' | awk 'print $0' | xargs cat

does not work with tail...

tail -f /var/log/httpd/modsec_audit.log | sed 's/[^/]*/./;s/].*$//g' | awk 'print $0' | xargs cat

I assume the tailing does not work because the script never terminates and sed is still caching the results until termination of the script.

Is there a way to make this work, continuously?

Buffering is getting in the way.

Use a while read line from the shell, which should read line-by-line and avoid most problems with buffering:

tail -f /var/log/httpd/modsec_audit.log | while read line; do
 echo "$line" | sed 's/[^/]*/./;s/].*$//g' | awk 'print $0' | xargs cat
done

Though you can probably do better than that, by using shell (bash) to match the filename from the log line too:

tail -f /var/log/httpd/modsec_audit.log | while read line; do
 line=$line/*([^/])/.
 line=$line%]*
 [[ -n "$line" ]] && cat "$line"
done

try with this:

tail -f /var/log/httpd/modsec_audit.log | stdbuf -oL sed 's/[^/]*/./;s/].*$//g' | stdbuf -oL awk 'print $0' | while IFS='' read -r file; do cat $file ; done