mjhjmtu

The following script will run every 1 hour. The script creates a new file each time it has run.

The first echo will push header and the for loop will push the results.

But what I additionally want is:

• every day the header should go only once into the file and it should keep appending the for loop results to the same file.


• it should create only one CSV file per day.


• it should delete any old files with name result_ something.

The code I have

echo "collectiontime,hostname,diskusage,directory" > /home/result_$CURR_TIME_EPOCH.csv

for i in /data01 /opt /opt/splunk /opt/splunk/var/lib/splunk/kvstore /opt/splunk/var/run/searchpeers /opt/splunk/var/run
do
 T=`sudo du -sh $i 2> /dev/null`
if [ $? -eq 0 ]; then 
 T=`echo $T | awk 'print $1","$2'`
 echo `date +%s`,$HOSTNAME,$T 
fi

done >> /home/result_$CURR_TIME_EPOCH.csv

Just test if the file exists, and if not: echo the header:

if [ ! -f "/home/result_$CURR_TIME_EPOCH.csv" ]; then 
 echo "collectiontime,hostname,diskusage,directory" > "/home/result_$CURR_TIME_EPOCH.csv"
fi

As Ljm Dullaart suggests, only output the header if the output file does not exist.

This is a suggestion for general improvement of your script (assuming GNU awk or mawk):

if [ ! -f "/home/result_$CURR_TIME_EPOCH.csv" ]; then 
 echo 'collectiontime,hostname,diskusage,directory' >"/home/result_$CURR_TIME_EPOCH.csv"
fi

for dir in /data01 /opt /opt/splunk /opt/splunk/var/lib/splunk/kvstore /opt/splunk/var/run/searchpeers /opt/splunk/var/run
do
 sudo du -sh "$dir" |
 awk -v OFS=',' ' print strftime("%s"), $1, $2 '
done >>"/home/result_$CURR_TIME_EPOCH.csv" 2>/dev/null

If you are using bash, you may benefit in terms of readability, from using an array for the directories (and a separate variable for the output file):

dirs=( /data01
 /opt
 /opt/splunk
 /opt/splunk/var/lib/splunk/kvstore
 /opt/splunk/var/run/searchpeers
 /opt/splunk/var/run )

outfile="/home/result_$CURR_TIME_EPOCH.csv"

if [ ! -f "$outfile" ]; then
 echo 'collectiontime,hostname,diskusage,directory' >"$outfile"
fi

for dir in "$dirs[@]"; do
 sudo du -sh "$dir" |
 awk -v OFS=',' ' print strftime("%s"), $1, $2 '
done >>"$outfile" 2>/dev/null

To delete old files, you could use, for example,

find /home -maxdepth 1 -type f -name 'result_*.cvs' -ctime +1 -delete

This would find and delete all regular files that has names matching the given pattern and that has a ctime timestamp of more than one day ago. Subdirectories of /home would not be searched thanks to -maxdepth 1.