mjhjmtu

On Debian Stretch (running as root) this the current behaviour:

# Create base directory
mkdir base
touch base/example

# Create merge, upper and work directories for 2 layers
mkdir layer1 layer1.upper layer1.work
mkdir layer2 layer2.upper layer2.work

# Mount layer1 as the merged directory using layer1.upper as the true upper layer,
# with base as a lower layer and layer1.work as the necessary work directory
mount -t overlay overlay -o lowerdir=$(pwd)/base,upperdir=$(pwd)/layer1.upper,workdir=$(pwd)/layer1.work layer1
ls layer1 # should show example as expected
ls layer1.upper # shows no file (this is expected behaviour, it should only show files written on layer1)
rm layer1/example
ls layer1 # should show no files
ls layer1.upper # should show a special character device called "example", this is the "whiteout" file

# unmount, and remount with layer2 being the new upper layer and using layer1.upper directory as the top level lower layer.
umount layer1
mount -t overlay overlay -o lowerdir=$(pwd)/base:$(pwd)/layer1.upper,upperdir=$(pwd)/layer2.upper,workdir=$(pwd)/layer2.work layer2
ls layer2 # now shows example again as if it was never deleted

Is this a bug? Or is this a limitation/expected behaviour?

If expected, any suggestions on a quick and easy workaround?

FWIW it works as desired under auFS, so one workaround is to install aufs-dkms and continue to use auFS... I may do that regardless, but I would really like clarity on whether this is a bug or expected behaviour.

[update] I was doing it wrong, please see (the now corrected) answer!

I've decided that this is a bug. As such, I've lodged a bug report with Debian:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=896646

Oops! Turns out I was doing it wrong! As noted in response to the Debian bug:

overlayfs is behaving as documented. The documentation
(filesystems/overlayfs.txt) says: "The specified lower directories will
be stacked beginning from the rightmost one and going left. In the
above example lower1 will be the top, lower2 the middle and lower3 the
bottom layer."

In your example this means that "layer1.upper" is the lowest layer, and
its whiteout is overridden by the file in "base" which is on top of it.
I think you just need to swap the order of these directories in the
mount options.

I had read that doc, but missed the "right to left" bit!

I can confirm, that when done correctly (i.e. swapped the order so it is right to left) it works as expected.