mjhjmtu

Why does this shell script gradually consume more memory in bash? I’m not using any local variables. Although the value of cpid is changing at every iteration because ls exits immediately, it should never grow too large.

The shell script in question just monitors a subprocess and checks if it’s up. This script is named run-always

set -euf

# monitor a child process
# see if it's up

cpid=
while : ; do
 # if cpid isn't set, spawn child
 if [ -z "$cpid" ]; then
 "$@" &
 cpid="$!"
 fi

 # if child isn't active unset cpid
 if ps -o pid= -p "$cpid"; then
 :
 else
 cpid=
 fi

 sleep 1
done

If I use it with a process that exits immediately (like ls) then we get the following memory usage when I execute the script with dash run-always ls

4476kb and it never grows.

This is the command I’m using to get the memory usage once per second on Linux … there’s probably a way to get the memory and peak memory usage directly from ps, but this does the job.

 while : ; do cat /proc/"$(ps a | grep run-always | grep -v grep | awk 'print $1')"/status | grep -i 'vmsize|vmpeak' ; sleep 1; done

When I execute the above script with dash, the memory usage never grows, it hovers at a constant 4476kb on my machine.

However, when I use bash (bash run-always ls), every ten seconds or so it allocates another few kilobytes of memory and never frees it.

Why is bash doing this?

Example output:

VmPeak: 16676 kB
VmSize: 16676 kB
VmPeak: 16676 kB
VmSize: 16676 kB
VmPeak: 16676 kB
VmSize: 16676 kB
VmPeak: 16676 kB
VmSize: 16676 kB
VmPeak: 16676 kB
VmSize: 16676 kB
VmPeak: 16676 kB
VmSize: 16676 kB
VmPeak: 16676 kB
VmSize: 16676 kB
VmPeak: 16676 kB
VmSize: 16676 kB
VmPeak: 16676 kB
VmSize: 16676 kB
VmPeak: 16676 kB
VmSize: 16676 kB
VmPeak: 16676 kB
VmSize: 16676 kB
VmPeak: 16680 kB
VmSize: 16680 kB
VmPeak: 16680 kB
VmSize: 16680 kB
VmPeak: 16680 kB
VmSize: 16680 kB
VmPeak: 16680 kB
VmSize: 16680 kB
VmPeak: 16680 kB
VmSize: 16680 kB
VmPeak: 16680 kB
VmSize: 16680 kB
VmPeak: 16680 kB
VmSize: 16680 kB
VmPeak: 16680 kB
VmSize: 16680 kB
VmPeak: 16680 kB
VmSize: 16680 kB
VmPeak: 16684 kB
VmSize: 16684 kB
VmPeak: 16684 kB
VmSize: 16684 kB
VmPeak: 16684 kB
VmSize: 16684 kB
VmPeak: 16684 kB
VmSize: 16684 kB
VmPeak: 16684 kB
VmSize: 16684 kB
VmPeak: 16684 kB
VmSize: 16684 kB
VmPeak: 16684 kB
VmSize: 16684 kB
VmPeak: 16684 kB
VmSize: 16684 kB
VmPeak: 16684 kB
VmSize: 16684 kB
VmPeak: 16684 kB
VmSize: 16684 kB
VmPeak: 16684 kB
VmSize: 16684 kB
VmPeak: 16684 kB
VmSize: 16684 kB
VmPeak: 16684 kB
VmSize: 16684 kB
VmPeak: 16684 kB
VmSize: 16684 kB
VmPeak: 16688 kB
VmSize: 16688 kB
VmPeak: 16688 kB
VmSize: 16688 kB
VmPeak: 16688 kB
VmSize: 16688 kB
VmPeak: 16688 kB
VmSize: 16688 kB
VmPeak: 16688 kB
VmSize: 16688 kB
VmPeak: 16688 kB
VmSize: 16688 kB
VmPeak: 16688 kB
VmSize: 16688 kB
VmPeak: 16688 kB
VmSize: 16688 kB
VmPeak: 16688 kB
VmSize: 16688 kB
VmPeak: 16688 kB
VmSize: 16688 kB
VmPeak: 16692 kB
VmSize: 16692 kB

Bash saves each backgrounded process in a table of active jobs. Because you're spawning new jobs without explicitly checking the old ones, the table potentially increases without bound. The problem goes away if you disown the process after backgrounding it or check the exit status using jobs before launching the next background process.

For example, this version of your script does not increase in memory usage:

set -euf

# monitor a child process
# see if it's up

cpid=
while : ; do
 # if cpid isn't set, spawn child
 if [ -z "$cpid" ]; then
 "$@" &
 disown $! # Don't waste memory on this red-headed child
 cpid="$!"
 fi

 # if child isn't active unset cpid
 if ps -o pid= -p "$cpid"; then
 :
 else
 cpid=
 fi

 sleep 1
done

As you already discovered that this problem does not appear with dash, it seems to be obvious that there is a high probability that this is caused by a bug.

It does also grow if you are using bash-3.x but very slowly - still faster than with other shells.

I recommend you to make a bug-report against your version of bash.

BTW: I did some tests and there are only two shells where there is no growth at all - regardless how long you wait: mksh and the original Bourne Shell that was not yet converted to use malloc() instead of sbrk().

All other shells grow very slowly.