mjhjmtu

For example, when I archive a few gigs of files (using tar), Linux uses quite a lot of disk caching (and some swap) but never cleans it up when the operation has completed. As a result, because there's no free memory Linux will try to swap out something from memory which in its turn creates an additional load on CPU.

Of course, I can clean up caches by running echo 1 > /proc/sys/vm/drop_caches but isn't that stupid that I have to do that?

Even worse with swap, there's no command to clean up unused swap, I have to disable/enable it completely which I don't think is a safe thing to do at all.

UPD:

I've run a few tests and found out a few things:

1. The swapped out memory pages during the archive command not related to archived files, it seems it's just a usual swapping out process caused by decreased free memory (because disk caching ate it all) according to swappiness




2. Running swapoff -a is actually safe, meaning swapped pages will move back to memory

My current solution is to limit archive command memory usage via cgroups (I run docker container with -m flag). If you don't use docker, there's a project https://github.com/Feh/nocache that might help.

The remaining question is when will Linux clean up disk caching and will it at all? If not, is it a good practice to manually clean up disk cache (echo 1 > /proc/sys/vm/drop_caches)?

(Nitpick: the CPU time used simply by swapping is normally insignificant. When the system is slow to respond during swapping, the normal problem is the disk time).

Even worse with swap, there's no command to clean up unused swap

Swap space is reclaimed on demand. But when a swap page is read back in, it is not specifically erased, and it is still kept track of - this means, if the page is to be swapped out again without it having been changed in between, the page does not need to be written again.

http://linux-tutorial.info/modules.php?name=MContent&pageid=314

The remaining question is when will Linux clean up disk caching and will it at all? If not, is it a good practice to manually clean up disk cache (echo 1 > /proc/sys/vm/drop_caches)?

It will clean up disk cache on demand only. I.e. inactive disk cache pages will be evicted when memory is needed. AIUI, vm.swappiness allows setting some bias between reclaiming inactive file cache, and reclaiming inactive "anonymous" (swap-backed) program memory, with the default being biased against swapping. (Make or force tmpfs to swap before the file cache)

There is no general recommendation to run drop_caches; it is more of a debugging tool. The official documentation of it is

This file is not a means to control the growth of the various kernel caches
(inodes, dentries, pagecache, etc...) These objects are automatically
reclaimed by the kernel when memory is needed elsewhere on the system.

Use of this file can cause performance problems. Since it discards cached
objects, it may cost a significant amount of I/O and CPU to recreate the
dropped objects, especially if they were under heavy use. Because of this,
use outside of a testing or debugging environment is not recommended.

This is the expected behavior. You can adjust swap ussage using the vm.swappines sysctl and tune it according to your needs.