mjhjmtu

I have a script which is started as part of a daemon, and runs an infinite loop to load data into a database. Depending on the hour, the data is loaded into one of 24 tables, and all data from the table that holds data for the next hour ahead is removed in order that there is never more than a days worth of data stored in the database. The data is loaded in using an application called dataflow which I believe is written in Java. My script is loosing on average 15m of memory each iteration of the loop - which continues running down until empty (normally takes a couple of days).

In order to more easily test where the leak was coming from, I put the commands that were running in the loop into a separate script, and called that script from the loop. The following is the content of that script:

#!/bin/bash

MYTABLECLEAR=$1
LOG_FILE=$2
DR_HOME=$3
CP_PLUGIN=$4
HOME_DIR=$5
PROPERTIES_HC=$6

#Run SQL to clear down table table 1 hour ahead of now (23 hours behind) 
(sql -uactian echo <<-EOSQL 
MODIFY reverb$MYTABLECLEAR TO TRUNCATED;g
EOSQL
) 1> /dev/null 2>> $LOG_FILE

#Run Dataflow
$DR_HOME/bin/dr -cp $CP_PLUGIN -Xmx64g --engine parallelism=1 --runjson $HOME_DIR/workflows/dl_Reverb.dr --overridefile $PROPERTIES_HC 1> /dev/null 2>> $LOG_FILE

If I comment out the Run SQL and the Run Dataflow sections of this code whilst my service is running, my memory loss flattens out i.e. I don't get anything back, but it doesn't get worse. If I then un-comment either of these, my leak resumes.

If I stop the service, I regain all memory immediately.

I have tried monitoring the PIDs of the processes I am running, by adding a echo "PID: " $! >> $LOG_FILE after each process, but the log file just shows blank.

Any ideas why my script is holding onto memory and how can I make it not?