mjhjmtu

Is there a way to limit how much data Linux will cache in the write cache for a given block device prior to beginning to flush that data out to disk?

My server has a large amount of RAM (64GB or so). I find that whenever I write large files to spinning media (i.e. slow media) or to slow USB flash drives, Linux will cache gigabytes of writable data in RAM. For many operations where only a small amount of data is written, this is fine and Linux will ultimately flush the cache to disk.

However, when I'm doing big operations on large files, the write caching and subsequent flushing makes the user interface seem very erratic. Take the simple example of using pv to pipe a large file from a RAM disk to stable storage on a USB flash drive which has slow write speeds. It will pull in well over 1GB of data at an insane rate, and then slow down to normal disk write speed, but then once the write is finished it will take well over a minute to flush the cache.

Other applications will appear to do a huge amount of writing almost instantly, then completely stall for a minute, then appear to do another bunch of writing instantly, then stall. Since the spinning disks are quite slow, sometimes the lag in flushing the cache is enough to throw a kernel OOPS with a process being hung for more than 120sec. Doing things like using ffmpeg to remux a file, transferring a VMware disk image, and similar "bulk" write operations on large, sequential files tend to cause this effect.

I would like to be able to tell Linux "on device /dev/sdX, only cache a maximum of, say, 64MB of data before beginning to flush the cache". What this should mean is that, in the above pv example, only a maximum 64MB of data should remain to be written after the UI indicates the write has completed. Typing the sync command should only have to flush out that 64MB. (Right now, doing a pv followed by a sync could hang sync for over two minutes, simply because Linux cached gigabytes of data and is now flushing it to a device with only single digits or tens of megabytes per second write speeds.)

I don't necessarily want to limit write-cache system wide, because e.g. my boot drive which is on an SSD definitely benefits from the cache, and typical operations involving very small writes also benefit well from it. It's only the case where large files are being written more or less sequentially where it's visible.

EDIT: My preferred distro is Arch Linux, but I'm pretty good at adapting instructions for other distros to work with my setup.