Creating a ram disk on Linux

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
68
down vote

favorite
24












I have a machine with 62GB of RAM, and a trunk that's only 7GB, so I thought I would create a RAM disk and compile there. I am not a Linux expert. I found instructions on the internet to create the RAM disk:



mkfs -q /dev/ram1 8192


but I changed the 8192 to 16777216 in an attempt to allocate 16GB of ram disk.



I got the following error:



mkfs.ext2: Filesystem larger than apparent device size.
Proceed anyway? (y,n)


At which point I got spooked and bailed.



sudo dmidecode --type 17 | grep Size


shows



8x8192MB + 2048MB = 67584 MB


but du on /dev gives 804K.



Is that the problem? Can I overcome that /dev size?










share|improve this question



















  • 14




    Did you try tmpfs? It is a filesystem in RAM, no need for ext2. mount -o size=16G -t tmpfs none /mnt/tmpfs
    – t-8ch
    Feb 27 '13 at 18:17











  • That worked! Thanks! But so far, not much speed-up: I think the tools I'm using to build are still using the regular disk. I'll put more stuff on the ram disk.
    – Frank
    Feb 27 '13 at 18:24






  • 3




    Putting the tools themselves on the ramdisk shouldn't make much difference as the kernel will cache them anyways in ram.
    – t-8ch
    Feb 27 '13 at 18:28






  • 1




    @goldilocks It's anecdotal evidence, but when compiling our Java projects with Maven, there is a significant speedup when using a ramdisk. I would guess though that this is more because of seek time than read time.
    – SpellingD
    Feb 27 '13 at 18:54






  • 1




    /dev/shm, actually /run/shm, can be used; it's almost always there.
    – Camille Goudeseune
    Feb 17 '14 at 22:06















up vote
68
down vote

favorite
24












I have a machine with 62GB of RAM, and a trunk that's only 7GB, so I thought I would create a RAM disk and compile there. I am not a Linux expert. I found instructions on the internet to create the RAM disk:



mkfs -q /dev/ram1 8192


but I changed the 8192 to 16777216 in an attempt to allocate 16GB of ram disk.



I got the following error:



mkfs.ext2: Filesystem larger than apparent device size.
Proceed anyway? (y,n)


At which point I got spooked and bailed.



sudo dmidecode --type 17 | grep Size


shows



8x8192MB + 2048MB = 67584 MB


but du on /dev gives 804K.



Is that the problem? Can I overcome that /dev size?










share|improve this question



















  • 14




    Did you try tmpfs? It is a filesystem in RAM, no need for ext2. mount -o size=16G -t tmpfs none /mnt/tmpfs
    – t-8ch
    Feb 27 '13 at 18:17











  • That worked! Thanks! But so far, not much speed-up: I think the tools I'm using to build are still using the regular disk. I'll put more stuff on the ram disk.
    – Frank
    Feb 27 '13 at 18:24






  • 3




    Putting the tools themselves on the ramdisk shouldn't make much difference as the kernel will cache them anyways in ram.
    – t-8ch
    Feb 27 '13 at 18:28






  • 1




    @goldilocks It's anecdotal evidence, but when compiling our Java projects with Maven, there is a significant speedup when using a ramdisk. I would guess though that this is more because of seek time than read time.
    – SpellingD
    Feb 27 '13 at 18:54






  • 1




    /dev/shm, actually /run/shm, can be used; it's almost always there.
    – Camille Goudeseune
    Feb 17 '14 at 22:06













up vote
68
down vote

favorite
24









up vote
68
down vote

favorite
24






24





I have a machine with 62GB of RAM, and a trunk that's only 7GB, so I thought I would create a RAM disk and compile there. I am not a Linux expert. I found instructions on the internet to create the RAM disk:



mkfs -q /dev/ram1 8192


but I changed the 8192 to 16777216 in an attempt to allocate 16GB of ram disk.



I got the following error:



mkfs.ext2: Filesystem larger than apparent device size.
Proceed anyway? (y,n)


At which point I got spooked and bailed.



sudo dmidecode --type 17 | grep Size


shows



8x8192MB + 2048MB = 67584 MB


but du on /dev gives 804K.



Is that the problem? Can I overcome that /dev size?










share|improve this question















I have a machine with 62GB of RAM, and a trunk that's only 7GB, so I thought I would create a RAM disk and compile there. I am not a Linux expert. I found instructions on the internet to create the RAM disk:



mkfs -q /dev/ram1 8192


but I changed the 8192 to 16777216 in an attempt to allocate 16GB of ram disk.



I got the following error:



mkfs.ext2: Filesystem larger than apparent device size.
Proceed anyway? (y,n)


At which point I got spooked and bailed.



sudo dmidecode --type 17 | grep Size


shows



8x8192MB + 2048MB = 67584 MB


but du on /dev gives 804K.



Is that the problem? Can I overcome that /dev size?







linux ramdisk






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Sep 22 at 15:02









Vlastimil

6,8261149123




6,8261149123










asked Feb 27 '13 at 18:16









Frank

341133




341133







  • 14




    Did you try tmpfs? It is a filesystem in RAM, no need for ext2. mount -o size=16G -t tmpfs none /mnt/tmpfs
    – t-8ch
    Feb 27 '13 at 18:17











  • That worked! Thanks! But so far, not much speed-up: I think the tools I'm using to build are still using the regular disk. I'll put more stuff on the ram disk.
    – Frank
    Feb 27 '13 at 18:24






  • 3




    Putting the tools themselves on the ramdisk shouldn't make much difference as the kernel will cache them anyways in ram.
    – t-8ch
    Feb 27 '13 at 18:28






  • 1




    @goldilocks It's anecdotal evidence, but when compiling our Java projects with Maven, there is a significant speedup when using a ramdisk. I would guess though that this is more because of seek time than read time.
    – SpellingD
    Feb 27 '13 at 18:54






  • 1




    /dev/shm, actually /run/shm, can be used; it's almost always there.
    – Camille Goudeseune
    Feb 17 '14 at 22:06













  • 14




    Did you try tmpfs? It is a filesystem in RAM, no need for ext2. mount -o size=16G -t tmpfs none /mnt/tmpfs
    – t-8ch
    Feb 27 '13 at 18:17











  • That worked! Thanks! But so far, not much speed-up: I think the tools I'm using to build are still using the regular disk. I'll put more stuff on the ram disk.
    – Frank
    Feb 27 '13 at 18:24






  • 3




    Putting the tools themselves on the ramdisk shouldn't make much difference as the kernel will cache them anyways in ram.
    – t-8ch
    Feb 27 '13 at 18:28






  • 1




    @goldilocks It's anecdotal evidence, but when compiling our Java projects with Maven, there is a significant speedup when using a ramdisk. I would guess though that this is more because of seek time than read time.
    – SpellingD
    Feb 27 '13 at 18:54






  • 1




    /dev/shm, actually /run/shm, can be used; it's almost always there.
    – Camille Goudeseune
    Feb 17 '14 at 22:06








14




14




Did you try tmpfs? It is a filesystem in RAM, no need for ext2. mount -o size=16G -t tmpfs none /mnt/tmpfs
– t-8ch
Feb 27 '13 at 18:17





Did you try tmpfs? It is a filesystem in RAM, no need for ext2. mount -o size=16G -t tmpfs none /mnt/tmpfs
– t-8ch
Feb 27 '13 at 18:17













That worked! Thanks! But so far, not much speed-up: I think the tools I'm using to build are still using the regular disk. I'll put more stuff on the ram disk.
– Frank
Feb 27 '13 at 18:24




That worked! Thanks! But so far, not much speed-up: I think the tools I'm using to build are still using the regular disk. I'll put more stuff on the ram disk.
– Frank
Feb 27 '13 at 18:24




3




3




Putting the tools themselves on the ramdisk shouldn't make much difference as the kernel will cache them anyways in ram.
– t-8ch
Feb 27 '13 at 18:28




Putting the tools themselves on the ramdisk shouldn't make much difference as the kernel will cache them anyways in ram.
– t-8ch
Feb 27 '13 at 18:28




1




1




@goldilocks It's anecdotal evidence, but when compiling our Java projects with Maven, there is a significant speedup when using a ramdisk. I would guess though that this is more because of seek time than read time.
– SpellingD
Feb 27 '13 at 18:54




@goldilocks It's anecdotal evidence, but when compiling our Java projects with Maven, there is a significant speedup when using a ramdisk. I would guess though that this is more because of seek time than read time.
– SpellingD
Feb 27 '13 at 18:54




1




1




/dev/shm, actually /run/shm, can be used; it's almost always there.
– Camille Goudeseune
Feb 17 '14 at 22:06





/dev/shm, actually /run/shm, can be used; it's almost always there.
– Camille Goudeseune
Feb 17 '14 at 22:06











6 Answers
6






active

oldest

votes

















up vote
72
down vote













The best way to create a ram disk on linux is tmpfs. It's a filesystem living in ram, so there is no need for ext2. You can create a tmpfs of 16Gb size with:



mount -o size=16G -t tmpfs none /mnt/tmpfs





share|improve this answer


















  • 2




    on my system, with nothing at all in /mnt, it says: ls: cannot access /mnt/tmpfs: No such file or directory mount: mount point /mnt/tmpfs does not exist. Is that something to worry about? If I simply mkdir /mnt/tmpfs, does that defeat the purpose (by creating tmpfs on the regular disk - please no flames, I'm a beginner here).
    – Frank
    Feb 27 '13 at 19:23







  • 9




    You need a mountpoint (directory) as target, so after you created this directory (you can use any directory, existing contents are shadowed) you can mount it with the command from the answer.
    – t-8ch
    Feb 27 '13 at 19:26










  • tmpfs may use swap, which you probably do not want in a pure RAM disk.
    – palswim
    Aug 17 '17 at 17:55










  • Why none there? Should not it be tmpfs?
    – Roman Susi
    Feb 6 at 10:46










  • @RomanSusi tmpfs is the filetype (passed after -t). "none" is the backing device ("disk") which does not exist for tmpfs
    – t-8ch
    Apr 29 at 21:24


















up vote
18
down vote













Linux is very efficient in using RAM. There is little surprise that you see little if any speedup with tmpfs. The largest pieces to read into memory (and thus able to slow the process down) are the tools (compiler, assembler, linker), and in a longish make they will be loaded into memory at startup and never leave it. What is left is reading in source (the writing out of the results won't slow you down, unless severely memory constrained). Again, comon header files will stay around, only the user's source will require reading. And that is unlikely to be more than a few megabytes. Creating a large RAMdisk (or even much use of tmpfs) can very well slow things down (by making the build memory constrained, the files on RAMdisk or on tmpfs can not be used directly from there).






share|improve this answer




















  • What! How can they not be used directly from there?
    – Kazark
    Feb 28 '13 at 16:08










  • They are in RAM, but not in a format that is directly usable.
    – vonbrand
    Feb 28 '13 at 16:09






  • 1




    Really! How so? (Pardon my slowness.)
    – Kazark
    Feb 28 '13 at 16:34






  • 7




    @Kazark, to handle executables in memory special data structures are used. As RAMdisks and tmpfs aren't in common use to store executables (RAMdisks are a remnant from the good old days of excruciatingly slow floppy disks and such, tmpfs is for stricty temporary data), nobody has considered it important enough to add the required ugly hacks.
    – vonbrand
    Feb 28 '13 at 16:39






  • 7




    I have tried running my rails code from a tmpfs (RAM) filesystem, and I did not see any difference at all. I was really hoping for a noticeable difference but I was disappointed by how awesome linux is.
    – Khaja Minhajuddin
    Mar 4 '13 at 19:25

















up vote
6
down vote













The problem is that the maximum size of a ramdisk, more specifically of size of memory that can be accessed via the ramdisk driver is configured at compiletime, can be overwritten at boottime, but remains fixed once the kernel is loaded into memory. The default value is probably measured in Megabytes. If I recall correctly the memory for a ramdisk is reserved right when the driver is loaded, all ramdisks are the same size and there is are some 16 ramdisks by default. So not even you want a ramdisk size of 16G :-)



As stated in the other answer, tmpfs is what you want to use. Further, you won't win a lot by having your entire OS in a ramdisk/tmpfs. Just copy your builddir to a tmpfs and do your compiling then. You may have to ensure that all temporary results are written to a location thats in the tmpfs as well.






share|improve this answer




















  • They don't actually use any memory until you write things to them. The boot time limit is just the limit. Even after filling one you can free the memory back up with blockdev --flushbufs.
    – psusi
    Feb 28 '13 at 13:57










  • @psusi: can you give us more information on that? I can only find statements mentioning that once claimed by the ramdisk memory is never reclaimed, e.g. in Documentation/blockdev/ramdisk.txt in the kernel sources. And on my answer: that file also says the ramdisk grows as memory is consumed so it is not all allocated at once.
    – Bananguin
    Mar 1 '13 at 14:46










  • What sort of information? You run the command and it frees the ram, assuming you don't still have it mounted anyhow.
    – psusi
    Mar 1 '13 at 15:13










  • How do you know that the command does what you say it does? Its man page doesn't confirm that and the documentation in the kernel source tree can be understood to contradict your information.
    – Bananguin
    Mar 1 '13 at 20:40






  • 6




    I read the source code, and verified it by trying it.
    – psusi
    Mar 1 '13 at 20:50

















up vote
2
down vote













To make a large ram disk after boot, with no messing around with kernel parameters, this seems to work.
Use tmpfs, make a file, mount it via loop, and mount that via a filesystem:



mount -t tmpfs -o size=200M tmpfs temp/
cd temp/
dd if=/dev/zero of=disk.img bs=1M count=199
losetup /dev/loop0 disk.img
mkfs.ext4 /dev/loop0
mount /dev/loop0 temp2/


Probably a bit of performance penalty going through multiple different layers... but at least it works.






share|improve this answer



























    up vote
    1
    down vote













    OP the amount of RAM is expressed in MB. So all you need to enter there is 16384. And then voila you'd be in business.






    share|improve this answer




















    • Nope. "If fs-size does not have a suffix, it is interpreted as power-of-two kilobytes." - man mkfs.ext2
      – sourcejedi
      Sep 22 at 14:49


















    up vote
    0
    down vote













    You can mount a ramfs filesystem, copy your project into it and work from there. This guarantees your input files are loaded in to RAM, and they will not be re-read from the much slower disk drive. However as you discovered, this is generally not a useful strategy. You already get the exact same benefit.




    Ramfs is a very simple filesystem that exports Linux's disk caching
    mechanisms
    (the page cache and dentry cache) as a dynamically resizable
    RAM-based filesystem.




    -- https://github.com/torvalds/linux/blob/v4.18/Documentation/filesystems/ramfs-rootfs-initramfs.txt



    You can already trust your input files are cached in RAM, the first time they are read. Your output files are also cached, so that you do not wait for them to be written to disk.



    There is no artificial limit on how much you can cache, how long it stays cached, etc. Caches only start to be dropped once you have filled RAM. Which cache is dropped first is chosen by terrifyingly elaborated algorithms. The first approximation, is we describe it as Least Recently Used. See What page replacement algorithms are used in Linux kernel for OS file cache?



    Note your text editor will explicitly fsync() saved files to disk.



    If you run tests of a program that involves fsync(), running these in a filesystem like ramfs may speed them up. Another strategy is to try and disable fsync() with eatmydata / nosync.so.



    Some other operating systems might have specific limitations, that can be bypassed using a ramdisk. At one end, the lack of any file caching is why ramdisks were popular on DOS.



    tmpfs



    tmpfs works the same as ramfs, except that it can use swap space if you have one. I.e. if you need RAM for something else, the Least Recently Used algorithms may select data blocks from tmpfs and swap them to disk.



    Most people stick with tmpfs, because it also lets you limit the total size, and shows the space used correctly e.g. in the df command. I'm not sure why this difference exists. The size limit in tmpfs protects you from accidentally filling your entire RAM and basically killing your system. It defaults to half of your RAM.



    Other reasons why writes can slow down



    The above is a simplification tailored to your case. The writes to files in your case should not need to wait for the disk. However there are some cases of writes that do. See the excellent blog post Why buffered writes are sometimes stalled. The most surprising case is a recent change to Linux called "stable page writes".






    share|improve this answer






















      Your Answer







      StackExchange.ready(function()
      var channelOptions =
      tags: "".split(" "),
      id: "106"
      ;
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function()
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled)
      StackExchange.using("snippets", function()
      createEditor();
      );

      else
      createEditor();

      );

      function createEditor()
      StackExchange.prepareEditor(
      heartbeatType: 'answer',
      convertImagesToLinks: false,
      noModals: false,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      );



      );













       

      draft saved


      draft discarded


















      StackExchange.ready(
      function ()
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f66329%2fcreating-a-ram-disk-on-linux%23new-answer', 'question_page');

      );

      Post as a guest






























      6 Answers
      6






      active

      oldest

      votes








      6 Answers
      6






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes








      up vote
      72
      down vote













      The best way to create a ram disk on linux is tmpfs. It's a filesystem living in ram, so there is no need for ext2. You can create a tmpfs of 16Gb size with:



      mount -o size=16G -t tmpfs none /mnt/tmpfs





      share|improve this answer


















      • 2




        on my system, with nothing at all in /mnt, it says: ls: cannot access /mnt/tmpfs: No such file or directory mount: mount point /mnt/tmpfs does not exist. Is that something to worry about? If I simply mkdir /mnt/tmpfs, does that defeat the purpose (by creating tmpfs on the regular disk - please no flames, I'm a beginner here).
        – Frank
        Feb 27 '13 at 19:23







      • 9




        You need a mountpoint (directory) as target, so after you created this directory (you can use any directory, existing contents are shadowed) you can mount it with the command from the answer.
        – t-8ch
        Feb 27 '13 at 19:26










      • tmpfs may use swap, which you probably do not want in a pure RAM disk.
        – palswim
        Aug 17 '17 at 17:55










      • Why none there? Should not it be tmpfs?
        – Roman Susi
        Feb 6 at 10:46










      • @RomanSusi tmpfs is the filetype (passed after -t). "none" is the backing device ("disk") which does not exist for tmpfs
        – t-8ch
        Apr 29 at 21:24















      up vote
      72
      down vote













      The best way to create a ram disk on linux is tmpfs. It's a filesystem living in ram, so there is no need for ext2. You can create a tmpfs of 16Gb size with:



      mount -o size=16G -t tmpfs none /mnt/tmpfs





      share|improve this answer


















      • 2




        on my system, with nothing at all in /mnt, it says: ls: cannot access /mnt/tmpfs: No such file or directory mount: mount point /mnt/tmpfs does not exist. Is that something to worry about? If I simply mkdir /mnt/tmpfs, does that defeat the purpose (by creating tmpfs on the regular disk - please no flames, I'm a beginner here).
        – Frank
        Feb 27 '13 at 19:23







      • 9




        You need a mountpoint (directory) as target, so after you created this directory (you can use any directory, existing contents are shadowed) you can mount it with the command from the answer.
        – t-8ch
        Feb 27 '13 at 19:26










      • tmpfs may use swap, which you probably do not want in a pure RAM disk.
        – palswim
        Aug 17 '17 at 17:55










      • Why none there? Should not it be tmpfs?
        – Roman Susi
        Feb 6 at 10:46










      • @RomanSusi tmpfs is the filetype (passed after -t). "none" is the backing device ("disk") which does not exist for tmpfs
        – t-8ch
        Apr 29 at 21:24













      up vote
      72
      down vote










      up vote
      72
      down vote









      The best way to create a ram disk on linux is tmpfs. It's a filesystem living in ram, so there is no need for ext2. You can create a tmpfs of 16Gb size with:



      mount -o size=16G -t tmpfs none /mnt/tmpfs





      share|improve this answer














      The best way to create a ram disk on linux is tmpfs. It's a filesystem living in ram, so there is no need for ext2. You can create a tmpfs of 16Gb size with:



      mount -o size=16G -t tmpfs none /mnt/tmpfs






      share|improve this answer














      share|improve this answer



      share|improve this answer








      edited Feb 27 '13 at 19:16









      Ulrich Dangel

      20k25671




      20k25671










      answered Feb 27 '13 at 18:38









      t-8ch

      1,35166




      1,35166







      • 2




        on my system, with nothing at all in /mnt, it says: ls: cannot access /mnt/tmpfs: No such file or directory mount: mount point /mnt/tmpfs does not exist. Is that something to worry about? If I simply mkdir /mnt/tmpfs, does that defeat the purpose (by creating tmpfs on the regular disk - please no flames, I'm a beginner here).
        – Frank
        Feb 27 '13 at 19:23







      • 9




        You need a mountpoint (directory) as target, so after you created this directory (you can use any directory, existing contents are shadowed) you can mount it with the command from the answer.
        – t-8ch
        Feb 27 '13 at 19:26










      • tmpfs may use swap, which you probably do not want in a pure RAM disk.
        – palswim
        Aug 17 '17 at 17:55










      • Why none there? Should not it be tmpfs?
        – Roman Susi
        Feb 6 at 10:46










      • @RomanSusi tmpfs is the filetype (passed after -t). "none" is the backing device ("disk") which does not exist for tmpfs
        – t-8ch
        Apr 29 at 21:24













      • 2




        on my system, with nothing at all in /mnt, it says: ls: cannot access /mnt/tmpfs: No such file or directory mount: mount point /mnt/tmpfs does not exist. Is that something to worry about? If I simply mkdir /mnt/tmpfs, does that defeat the purpose (by creating tmpfs on the regular disk - please no flames, I'm a beginner here).
        – Frank
        Feb 27 '13 at 19:23







      • 9




        You need a mountpoint (directory) as target, so after you created this directory (you can use any directory, existing contents are shadowed) you can mount it with the command from the answer.
        – t-8ch
        Feb 27 '13 at 19:26










      • tmpfs may use swap, which you probably do not want in a pure RAM disk.
        – palswim
        Aug 17 '17 at 17:55










      • Why none there? Should not it be tmpfs?
        – Roman Susi
        Feb 6 at 10:46










      • @RomanSusi tmpfs is the filetype (passed after -t). "none" is the backing device ("disk") which does not exist for tmpfs
        – t-8ch
        Apr 29 at 21:24








      2




      2




      on my system, with nothing at all in /mnt, it says: ls: cannot access /mnt/tmpfs: No such file or directory mount: mount point /mnt/tmpfs does not exist. Is that something to worry about? If I simply mkdir /mnt/tmpfs, does that defeat the purpose (by creating tmpfs on the regular disk - please no flames, I'm a beginner here).
      – Frank
      Feb 27 '13 at 19:23





      on my system, with nothing at all in /mnt, it says: ls: cannot access /mnt/tmpfs: No such file or directory mount: mount point /mnt/tmpfs does not exist. Is that something to worry about? If I simply mkdir /mnt/tmpfs, does that defeat the purpose (by creating tmpfs on the regular disk - please no flames, I'm a beginner here).
      – Frank
      Feb 27 '13 at 19:23





      9




      9




      You need a mountpoint (directory) as target, so after you created this directory (you can use any directory, existing contents are shadowed) you can mount it with the command from the answer.
      – t-8ch
      Feb 27 '13 at 19:26




      You need a mountpoint (directory) as target, so after you created this directory (you can use any directory, existing contents are shadowed) you can mount it with the command from the answer.
      – t-8ch
      Feb 27 '13 at 19:26












      tmpfs may use swap, which you probably do not want in a pure RAM disk.
      – palswim
      Aug 17 '17 at 17:55




      tmpfs may use swap, which you probably do not want in a pure RAM disk.
      – palswim
      Aug 17 '17 at 17:55












      Why none there? Should not it be tmpfs?
      – Roman Susi
      Feb 6 at 10:46




      Why none there? Should not it be tmpfs?
      – Roman Susi
      Feb 6 at 10:46












      @RomanSusi tmpfs is the filetype (passed after -t). "none" is the backing device ("disk") which does not exist for tmpfs
      – t-8ch
      Apr 29 at 21:24





      @RomanSusi tmpfs is the filetype (passed after -t). "none" is the backing device ("disk") which does not exist for tmpfs
      – t-8ch
      Apr 29 at 21:24













      up vote
      18
      down vote













      Linux is very efficient in using RAM. There is little surprise that you see little if any speedup with tmpfs. The largest pieces to read into memory (and thus able to slow the process down) are the tools (compiler, assembler, linker), and in a longish make they will be loaded into memory at startup and never leave it. What is left is reading in source (the writing out of the results won't slow you down, unless severely memory constrained). Again, comon header files will stay around, only the user's source will require reading. And that is unlikely to be more than a few megabytes. Creating a large RAMdisk (or even much use of tmpfs) can very well slow things down (by making the build memory constrained, the files on RAMdisk or on tmpfs can not be used directly from there).






      share|improve this answer




















      • What! How can they not be used directly from there?
        – Kazark
        Feb 28 '13 at 16:08










      • They are in RAM, but not in a format that is directly usable.
        – vonbrand
        Feb 28 '13 at 16:09






      • 1




        Really! How so? (Pardon my slowness.)
        – Kazark
        Feb 28 '13 at 16:34






      • 7




        @Kazark, to handle executables in memory special data structures are used. As RAMdisks and tmpfs aren't in common use to store executables (RAMdisks are a remnant from the good old days of excruciatingly slow floppy disks and such, tmpfs is for stricty temporary data), nobody has considered it important enough to add the required ugly hacks.
        – vonbrand
        Feb 28 '13 at 16:39






      • 7




        I have tried running my rails code from a tmpfs (RAM) filesystem, and I did not see any difference at all. I was really hoping for a noticeable difference but I was disappointed by how awesome linux is.
        – Khaja Minhajuddin
        Mar 4 '13 at 19:25














      up vote
      18
      down vote













      Linux is very efficient in using RAM. There is little surprise that you see little if any speedup with tmpfs. The largest pieces to read into memory (and thus able to slow the process down) are the tools (compiler, assembler, linker), and in a longish make they will be loaded into memory at startup and never leave it. What is left is reading in source (the writing out of the results won't slow you down, unless severely memory constrained). Again, comon header files will stay around, only the user's source will require reading. And that is unlikely to be more than a few megabytes. Creating a large RAMdisk (or even much use of tmpfs) can very well slow things down (by making the build memory constrained, the files on RAMdisk or on tmpfs can not be used directly from there).






      share|improve this answer




















      • What! How can they not be used directly from there?
        – Kazark
        Feb 28 '13 at 16:08










      • They are in RAM, but not in a format that is directly usable.
        – vonbrand
        Feb 28 '13 at 16:09






      • 1




        Really! How so? (Pardon my slowness.)
        – Kazark
        Feb 28 '13 at 16:34






      • 7




        @Kazark, to handle executables in memory special data structures are used. As RAMdisks and tmpfs aren't in common use to store executables (RAMdisks are a remnant from the good old days of excruciatingly slow floppy disks and such, tmpfs is for stricty temporary data), nobody has considered it important enough to add the required ugly hacks.
        – vonbrand
        Feb 28 '13 at 16:39






      • 7




        I have tried running my rails code from a tmpfs (RAM) filesystem, and I did not see any difference at all. I was really hoping for a noticeable difference but I was disappointed by how awesome linux is.
        – Khaja Minhajuddin
        Mar 4 '13 at 19:25












      up vote
      18
      down vote










      up vote
      18
      down vote









      Linux is very efficient in using RAM. There is little surprise that you see little if any speedup with tmpfs. The largest pieces to read into memory (and thus able to slow the process down) are the tools (compiler, assembler, linker), and in a longish make they will be loaded into memory at startup and never leave it. What is left is reading in source (the writing out of the results won't slow you down, unless severely memory constrained). Again, comon header files will stay around, only the user's source will require reading. And that is unlikely to be more than a few megabytes. Creating a large RAMdisk (or even much use of tmpfs) can very well slow things down (by making the build memory constrained, the files on RAMdisk or on tmpfs can not be used directly from there).






      share|improve this answer












      Linux is very efficient in using RAM. There is little surprise that you see little if any speedup with tmpfs. The largest pieces to read into memory (and thus able to slow the process down) are the tools (compiler, assembler, linker), and in a longish make they will be loaded into memory at startup and never leave it. What is left is reading in source (the writing out of the results won't slow you down, unless severely memory constrained). Again, comon header files will stay around, only the user's source will require reading. And that is unlikely to be more than a few megabytes. Creating a large RAMdisk (or even much use of tmpfs) can very well slow things down (by making the build memory constrained, the files on RAMdisk or on tmpfs can not be used directly from there).







      share|improve this answer












      share|improve this answer



      share|improve this answer










      answered Feb 28 '13 at 13:45









      vonbrand

      14k22444




      14k22444











      • What! How can they not be used directly from there?
        – Kazark
        Feb 28 '13 at 16:08










      • They are in RAM, but not in a format that is directly usable.
        – vonbrand
        Feb 28 '13 at 16:09






      • 1




        Really! How so? (Pardon my slowness.)
        – Kazark
        Feb 28 '13 at 16:34






      • 7




        @Kazark, to handle executables in memory special data structures are used. As RAMdisks and tmpfs aren't in common use to store executables (RAMdisks are a remnant from the good old days of excruciatingly slow floppy disks and such, tmpfs is for stricty temporary data), nobody has considered it important enough to add the required ugly hacks.
        – vonbrand
        Feb 28 '13 at 16:39






      • 7




        I have tried running my rails code from a tmpfs (RAM) filesystem, and I did not see any difference at all. I was really hoping for a noticeable difference but I was disappointed by how awesome linux is.
        – Khaja Minhajuddin
        Mar 4 '13 at 19:25
















      • What! How can they not be used directly from there?
        – Kazark
        Feb 28 '13 at 16:08










      • They are in RAM, but not in a format that is directly usable.
        – vonbrand
        Feb 28 '13 at 16:09






      • 1




        Really! How so? (Pardon my slowness.)
        – Kazark
        Feb 28 '13 at 16:34






      • 7




        @Kazark, to handle executables in memory special data structures are used. As RAMdisks and tmpfs aren't in common use to store executables (RAMdisks are a remnant from the good old days of excruciatingly slow floppy disks and such, tmpfs is for stricty temporary data), nobody has considered it important enough to add the required ugly hacks.
        – vonbrand
        Feb 28 '13 at 16:39






      • 7




        I have tried running my rails code from a tmpfs (RAM) filesystem, and I did not see any difference at all. I was really hoping for a noticeable difference but I was disappointed by how awesome linux is.
        – Khaja Minhajuddin
        Mar 4 '13 at 19:25















      What! How can they not be used directly from there?
      – Kazark
      Feb 28 '13 at 16:08




      What! How can they not be used directly from there?
      – Kazark
      Feb 28 '13 at 16:08












      They are in RAM, but not in a format that is directly usable.
      – vonbrand
      Feb 28 '13 at 16:09




      They are in RAM, but not in a format that is directly usable.
      – vonbrand
      Feb 28 '13 at 16:09




      1




      1




      Really! How so? (Pardon my slowness.)
      – Kazark
      Feb 28 '13 at 16:34




      Really! How so? (Pardon my slowness.)
      – Kazark
      Feb 28 '13 at 16:34




      7




      7




      @Kazark, to handle executables in memory special data structures are used. As RAMdisks and tmpfs aren't in common use to store executables (RAMdisks are a remnant from the good old days of excruciatingly slow floppy disks and such, tmpfs is for stricty temporary data), nobody has considered it important enough to add the required ugly hacks.
      – vonbrand
      Feb 28 '13 at 16:39




      @Kazark, to handle executables in memory special data structures are used. As RAMdisks and tmpfs aren't in common use to store executables (RAMdisks are a remnant from the good old days of excruciatingly slow floppy disks and such, tmpfs is for stricty temporary data), nobody has considered it important enough to add the required ugly hacks.
      – vonbrand
      Feb 28 '13 at 16:39




      7




      7




      I have tried running my rails code from a tmpfs (RAM) filesystem, and I did not see any difference at all. I was really hoping for a noticeable difference but I was disappointed by how awesome linux is.
      – Khaja Minhajuddin
      Mar 4 '13 at 19:25




      I have tried running my rails code from a tmpfs (RAM) filesystem, and I did not see any difference at all. I was really hoping for a noticeable difference but I was disappointed by how awesome linux is.
      – Khaja Minhajuddin
      Mar 4 '13 at 19:25










      up vote
      6
      down vote













      The problem is that the maximum size of a ramdisk, more specifically of size of memory that can be accessed via the ramdisk driver is configured at compiletime, can be overwritten at boottime, but remains fixed once the kernel is loaded into memory. The default value is probably measured in Megabytes. If I recall correctly the memory for a ramdisk is reserved right when the driver is loaded, all ramdisks are the same size and there is are some 16 ramdisks by default. So not even you want a ramdisk size of 16G :-)



      As stated in the other answer, tmpfs is what you want to use. Further, you won't win a lot by having your entire OS in a ramdisk/tmpfs. Just copy your builddir to a tmpfs and do your compiling then. You may have to ensure that all temporary results are written to a location thats in the tmpfs as well.






      share|improve this answer




















      • They don't actually use any memory until you write things to them. The boot time limit is just the limit. Even after filling one you can free the memory back up with blockdev --flushbufs.
        – psusi
        Feb 28 '13 at 13:57










      • @psusi: can you give us more information on that? I can only find statements mentioning that once claimed by the ramdisk memory is never reclaimed, e.g. in Documentation/blockdev/ramdisk.txt in the kernel sources. And on my answer: that file also says the ramdisk grows as memory is consumed so it is not all allocated at once.
        – Bananguin
        Mar 1 '13 at 14:46










      • What sort of information? You run the command and it frees the ram, assuming you don't still have it mounted anyhow.
        – psusi
        Mar 1 '13 at 15:13










      • How do you know that the command does what you say it does? Its man page doesn't confirm that and the documentation in the kernel source tree can be understood to contradict your information.
        – Bananguin
        Mar 1 '13 at 20:40






      • 6




        I read the source code, and verified it by trying it.
        – psusi
        Mar 1 '13 at 20:50














      up vote
      6
      down vote













      The problem is that the maximum size of a ramdisk, more specifically of size of memory that can be accessed via the ramdisk driver is configured at compiletime, can be overwritten at boottime, but remains fixed once the kernel is loaded into memory. The default value is probably measured in Megabytes. If I recall correctly the memory for a ramdisk is reserved right when the driver is loaded, all ramdisks are the same size and there is are some 16 ramdisks by default. So not even you want a ramdisk size of 16G :-)



      As stated in the other answer, tmpfs is what you want to use. Further, you won't win a lot by having your entire OS in a ramdisk/tmpfs. Just copy your builddir to a tmpfs and do your compiling then. You may have to ensure that all temporary results are written to a location thats in the tmpfs as well.






      share|improve this answer




















      • They don't actually use any memory until you write things to them. The boot time limit is just the limit. Even after filling one you can free the memory back up with blockdev --flushbufs.
        – psusi
        Feb 28 '13 at 13:57










      • @psusi: can you give us more information on that? I can only find statements mentioning that once claimed by the ramdisk memory is never reclaimed, e.g. in Documentation/blockdev/ramdisk.txt in the kernel sources. And on my answer: that file also says the ramdisk grows as memory is consumed so it is not all allocated at once.
        – Bananguin
        Mar 1 '13 at 14:46










      • What sort of information? You run the command and it frees the ram, assuming you don't still have it mounted anyhow.
        – psusi
        Mar 1 '13 at 15:13










      • How do you know that the command does what you say it does? Its man page doesn't confirm that and the documentation in the kernel source tree can be understood to contradict your information.
        – Bananguin
        Mar 1 '13 at 20:40






      • 6




        I read the source code, and verified it by trying it.
        – psusi
        Mar 1 '13 at 20:50












      up vote
      6
      down vote










      up vote
      6
      down vote









      The problem is that the maximum size of a ramdisk, more specifically of size of memory that can be accessed via the ramdisk driver is configured at compiletime, can be overwritten at boottime, but remains fixed once the kernel is loaded into memory. The default value is probably measured in Megabytes. If I recall correctly the memory for a ramdisk is reserved right when the driver is loaded, all ramdisks are the same size and there is are some 16 ramdisks by default. So not even you want a ramdisk size of 16G :-)



      As stated in the other answer, tmpfs is what you want to use. Further, you won't win a lot by having your entire OS in a ramdisk/tmpfs. Just copy your builddir to a tmpfs and do your compiling then. You may have to ensure that all temporary results are written to a location thats in the tmpfs as well.






      share|improve this answer












      The problem is that the maximum size of a ramdisk, more specifically of size of memory that can be accessed via the ramdisk driver is configured at compiletime, can be overwritten at boottime, but remains fixed once the kernel is loaded into memory. The default value is probably measured in Megabytes. If I recall correctly the memory for a ramdisk is reserved right when the driver is loaded, all ramdisks are the same size and there is are some 16 ramdisks by default. So not even you want a ramdisk size of 16G :-)



      As stated in the other answer, tmpfs is what you want to use. Further, you won't win a lot by having your entire OS in a ramdisk/tmpfs. Just copy your builddir to a tmpfs and do your compiling then. You may have to ensure that all temporary results are written to a location thats in the tmpfs as well.







      share|improve this answer












      share|improve this answer



      share|improve this answer










      answered Feb 27 '13 at 21:04









      Bananguin

      5,1801338




      5,1801338











      • They don't actually use any memory until you write things to them. The boot time limit is just the limit. Even after filling one you can free the memory back up with blockdev --flushbufs.
        – psusi
        Feb 28 '13 at 13:57










      • @psusi: can you give us more information on that? I can only find statements mentioning that once claimed by the ramdisk memory is never reclaimed, e.g. in Documentation/blockdev/ramdisk.txt in the kernel sources. And on my answer: that file also says the ramdisk grows as memory is consumed so it is not all allocated at once.
        – Bananguin
        Mar 1 '13 at 14:46










      • What sort of information? You run the command and it frees the ram, assuming you don't still have it mounted anyhow.
        – psusi
        Mar 1 '13 at 15:13










      • How do you know that the command does what you say it does? Its man page doesn't confirm that and the documentation in the kernel source tree can be understood to contradict your information.
        – Bananguin
        Mar 1 '13 at 20:40






      • 6




        I read the source code, and verified it by trying it.
        – psusi
        Mar 1 '13 at 20:50
















      • They don't actually use any memory until you write things to them. The boot time limit is just the limit. Even after filling one you can free the memory back up with blockdev --flushbufs.
        – psusi
        Feb 28 '13 at 13:57










      • @psusi: can you give us more information on that? I can only find statements mentioning that once claimed by the ramdisk memory is never reclaimed, e.g. in Documentation/blockdev/ramdisk.txt in the kernel sources. And on my answer: that file also says the ramdisk grows as memory is consumed so it is not all allocated at once.
        – Bananguin
        Mar 1 '13 at 14:46










      • What sort of information? You run the command and it frees the ram, assuming you don't still have it mounted anyhow.
        – psusi
        Mar 1 '13 at 15:13










      • How do you know that the command does what you say it does? Its man page doesn't confirm that and the documentation in the kernel source tree can be understood to contradict your information.
        – Bananguin
        Mar 1 '13 at 20:40






      • 6




        I read the source code, and verified it by trying it.
        – psusi
        Mar 1 '13 at 20:50















      They don't actually use any memory until you write things to them. The boot time limit is just the limit. Even after filling one you can free the memory back up with blockdev --flushbufs.
      – psusi
      Feb 28 '13 at 13:57




      They don't actually use any memory until you write things to them. The boot time limit is just the limit. Even after filling one you can free the memory back up with blockdev --flushbufs.
      – psusi
      Feb 28 '13 at 13:57












      @psusi: can you give us more information on that? I can only find statements mentioning that once claimed by the ramdisk memory is never reclaimed, e.g. in Documentation/blockdev/ramdisk.txt in the kernel sources. And on my answer: that file also says the ramdisk grows as memory is consumed so it is not all allocated at once.
      – Bananguin
      Mar 1 '13 at 14:46




      @psusi: can you give us more information on that? I can only find statements mentioning that once claimed by the ramdisk memory is never reclaimed, e.g. in Documentation/blockdev/ramdisk.txt in the kernel sources. And on my answer: that file also says the ramdisk grows as memory is consumed so it is not all allocated at once.
      – Bananguin
      Mar 1 '13 at 14:46












      What sort of information? You run the command and it frees the ram, assuming you don't still have it mounted anyhow.
      – psusi
      Mar 1 '13 at 15:13




      What sort of information? You run the command and it frees the ram, assuming you don't still have it mounted anyhow.
      – psusi
      Mar 1 '13 at 15:13












      How do you know that the command does what you say it does? Its man page doesn't confirm that and the documentation in the kernel source tree can be understood to contradict your information.
      – Bananguin
      Mar 1 '13 at 20:40




      How do you know that the command does what you say it does? Its man page doesn't confirm that and the documentation in the kernel source tree can be understood to contradict your information.
      – Bananguin
      Mar 1 '13 at 20:40




      6




      6




      I read the source code, and verified it by trying it.
      – psusi
      Mar 1 '13 at 20:50




      I read the source code, and verified it by trying it.
      – psusi
      Mar 1 '13 at 20:50










      up vote
      2
      down vote













      To make a large ram disk after boot, with no messing around with kernel parameters, this seems to work.
      Use tmpfs, make a file, mount it via loop, and mount that via a filesystem:



      mount -t tmpfs -o size=200M tmpfs temp/
      cd temp/
      dd if=/dev/zero of=disk.img bs=1M count=199
      losetup /dev/loop0 disk.img
      mkfs.ext4 /dev/loop0
      mount /dev/loop0 temp2/


      Probably a bit of performance penalty going through multiple different layers... but at least it works.






      share|improve this answer
























        up vote
        2
        down vote













        To make a large ram disk after boot, with no messing around with kernel parameters, this seems to work.
        Use tmpfs, make a file, mount it via loop, and mount that via a filesystem:



        mount -t tmpfs -o size=200M tmpfs temp/
        cd temp/
        dd if=/dev/zero of=disk.img bs=1M count=199
        losetup /dev/loop0 disk.img
        mkfs.ext4 /dev/loop0
        mount /dev/loop0 temp2/


        Probably a bit of performance penalty going through multiple different layers... but at least it works.






        share|improve this answer






















          up vote
          2
          down vote










          up vote
          2
          down vote









          To make a large ram disk after boot, with no messing around with kernel parameters, this seems to work.
          Use tmpfs, make a file, mount it via loop, and mount that via a filesystem:



          mount -t tmpfs -o size=200M tmpfs temp/
          cd temp/
          dd if=/dev/zero of=disk.img bs=1M count=199
          losetup /dev/loop0 disk.img
          mkfs.ext4 /dev/loop0
          mount /dev/loop0 temp2/


          Probably a bit of performance penalty going through multiple different layers... but at least it works.






          share|improve this answer












          To make a large ram disk after boot, with no messing around with kernel parameters, this seems to work.
          Use tmpfs, make a file, mount it via loop, and mount that via a filesystem:



          mount -t tmpfs -o size=200M tmpfs temp/
          cd temp/
          dd if=/dev/zero of=disk.img bs=1M count=199
          losetup /dev/loop0 disk.img
          mkfs.ext4 /dev/loop0
          mount /dev/loop0 temp2/


          Probably a bit of performance penalty going through multiple different layers... but at least it works.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered May 23 '13 at 14:55









          user39725

          211




          211




















              up vote
              1
              down vote













              OP the amount of RAM is expressed in MB. So all you need to enter there is 16384. And then voila you'd be in business.






              share|improve this answer




















              • Nope. "If fs-size does not have a suffix, it is interpreted as power-of-two kilobytes." - man mkfs.ext2
                – sourcejedi
                Sep 22 at 14:49















              up vote
              1
              down vote













              OP the amount of RAM is expressed in MB. So all you need to enter there is 16384. And then voila you'd be in business.






              share|improve this answer




















              • Nope. "If fs-size does not have a suffix, it is interpreted as power-of-two kilobytes." - man mkfs.ext2
                – sourcejedi
                Sep 22 at 14:49













              up vote
              1
              down vote










              up vote
              1
              down vote









              OP the amount of RAM is expressed in MB. So all you need to enter there is 16384. And then voila you'd be in business.






              share|improve this answer












              OP the amount of RAM is expressed in MB. So all you need to enter there is 16384. And then voila you'd be in business.







              share|improve this answer












              share|improve this answer



              share|improve this answer










              answered Aug 20 '15 at 12:11









              CyberWolf64

              111




              111











              • Nope. "If fs-size does not have a suffix, it is interpreted as power-of-two kilobytes." - man mkfs.ext2
                – sourcejedi
                Sep 22 at 14:49

















              • Nope. "If fs-size does not have a suffix, it is interpreted as power-of-two kilobytes." - man mkfs.ext2
                – sourcejedi
                Sep 22 at 14:49
















              Nope. "If fs-size does not have a suffix, it is interpreted as power-of-two kilobytes." - man mkfs.ext2
              – sourcejedi
              Sep 22 at 14:49





              Nope. "If fs-size does not have a suffix, it is interpreted as power-of-two kilobytes." - man mkfs.ext2
              – sourcejedi
              Sep 22 at 14:49











              up vote
              0
              down vote













              You can mount a ramfs filesystem, copy your project into it and work from there. This guarantees your input files are loaded in to RAM, and they will not be re-read from the much slower disk drive. However as you discovered, this is generally not a useful strategy. You already get the exact same benefit.




              Ramfs is a very simple filesystem that exports Linux's disk caching
              mechanisms
              (the page cache and dentry cache) as a dynamically resizable
              RAM-based filesystem.




              -- https://github.com/torvalds/linux/blob/v4.18/Documentation/filesystems/ramfs-rootfs-initramfs.txt



              You can already trust your input files are cached in RAM, the first time they are read. Your output files are also cached, so that you do not wait for them to be written to disk.



              There is no artificial limit on how much you can cache, how long it stays cached, etc. Caches only start to be dropped once you have filled RAM. Which cache is dropped first is chosen by terrifyingly elaborated algorithms. The first approximation, is we describe it as Least Recently Used. See What page replacement algorithms are used in Linux kernel for OS file cache?



              Note your text editor will explicitly fsync() saved files to disk.



              If you run tests of a program that involves fsync(), running these in a filesystem like ramfs may speed them up. Another strategy is to try and disable fsync() with eatmydata / nosync.so.



              Some other operating systems might have specific limitations, that can be bypassed using a ramdisk. At one end, the lack of any file caching is why ramdisks were popular on DOS.



              tmpfs



              tmpfs works the same as ramfs, except that it can use swap space if you have one. I.e. if you need RAM for something else, the Least Recently Used algorithms may select data blocks from tmpfs and swap them to disk.



              Most people stick with tmpfs, because it also lets you limit the total size, and shows the space used correctly e.g. in the df command. I'm not sure why this difference exists. The size limit in tmpfs protects you from accidentally filling your entire RAM and basically killing your system. It defaults to half of your RAM.



              Other reasons why writes can slow down



              The above is a simplification tailored to your case. The writes to files in your case should not need to wait for the disk. However there are some cases of writes that do. See the excellent blog post Why buffered writes are sometimes stalled. The most surprising case is a recent change to Linux called "stable page writes".






              share|improve this answer


























                up vote
                0
                down vote













                You can mount a ramfs filesystem, copy your project into it and work from there. This guarantees your input files are loaded in to RAM, and they will not be re-read from the much slower disk drive. However as you discovered, this is generally not a useful strategy. You already get the exact same benefit.




                Ramfs is a very simple filesystem that exports Linux's disk caching
                mechanisms
                (the page cache and dentry cache) as a dynamically resizable
                RAM-based filesystem.




                -- https://github.com/torvalds/linux/blob/v4.18/Documentation/filesystems/ramfs-rootfs-initramfs.txt



                You can already trust your input files are cached in RAM, the first time they are read. Your output files are also cached, so that you do not wait for them to be written to disk.



                There is no artificial limit on how much you can cache, how long it stays cached, etc. Caches only start to be dropped once you have filled RAM. Which cache is dropped first is chosen by terrifyingly elaborated algorithms. The first approximation, is we describe it as Least Recently Used. See What page replacement algorithms are used in Linux kernel for OS file cache?



                Note your text editor will explicitly fsync() saved files to disk.



                If you run tests of a program that involves fsync(), running these in a filesystem like ramfs may speed them up. Another strategy is to try and disable fsync() with eatmydata / nosync.so.



                Some other operating systems might have specific limitations, that can be bypassed using a ramdisk. At one end, the lack of any file caching is why ramdisks were popular on DOS.



                tmpfs



                tmpfs works the same as ramfs, except that it can use swap space if you have one. I.e. if you need RAM for something else, the Least Recently Used algorithms may select data blocks from tmpfs and swap them to disk.



                Most people stick with tmpfs, because it also lets you limit the total size, and shows the space used correctly e.g. in the df command. I'm not sure why this difference exists. The size limit in tmpfs protects you from accidentally filling your entire RAM and basically killing your system. It defaults to half of your RAM.



                Other reasons why writes can slow down



                The above is a simplification tailored to your case. The writes to files in your case should not need to wait for the disk. However there are some cases of writes that do. See the excellent blog post Why buffered writes are sometimes stalled. The most surprising case is a recent change to Linux called "stable page writes".






                share|improve this answer
























                  up vote
                  0
                  down vote










                  up vote
                  0
                  down vote









                  You can mount a ramfs filesystem, copy your project into it and work from there. This guarantees your input files are loaded in to RAM, and they will not be re-read from the much slower disk drive. However as you discovered, this is generally not a useful strategy. You already get the exact same benefit.




                  Ramfs is a very simple filesystem that exports Linux's disk caching
                  mechanisms
                  (the page cache and dentry cache) as a dynamically resizable
                  RAM-based filesystem.




                  -- https://github.com/torvalds/linux/blob/v4.18/Documentation/filesystems/ramfs-rootfs-initramfs.txt



                  You can already trust your input files are cached in RAM, the first time they are read. Your output files are also cached, so that you do not wait for them to be written to disk.



                  There is no artificial limit on how much you can cache, how long it stays cached, etc. Caches only start to be dropped once you have filled RAM. Which cache is dropped first is chosen by terrifyingly elaborated algorithms. The first approximation, is we describe it as Least Recently Used. See What page replacement algorithms are used in Linux kernel for OS file cache?



                  Note your text editor will explicitly fsync() saved files to disk.



                  If you run tests of a program that involves fsync(), running these in a filesystem like ramfs may speed them up. Another strategy is to try and disable fsync() with eatmydata / nosync.so.



                  Some other operating systems might have specific limitations, that can be bypassed using a ramdisk. At one end, the lack of any file caching is why ramdisks were popular on DOS.



                  tmpfs



                  tmpfs works the same as ramfs, except that it can use swap space if you have one. I.e. if you need RAM for something else, the Least Recently Used algorithms may select data blocks from tmpfs and swap them to disk.



                  Most people stick with tmpfs, because it also lets you limit the total size, and shows the space used correctly e.g. in the df command. I'm not sure why this difference exists. The size limit in tmpfs protects you from accidentally filling your entire RAM and basically killing your system. It defaults to half of your RAM.



                  Other reasons why writes can slow down



                  The above is a simplification tailored to your case. The writes to files in your case should not need to wait for the disk. However there are some cases of writes that do. See the excellent blog post Why buffered writes are sometimes stalled. The most surprising case is a recent change to Linux called "stable page writes".






                  share|improve this answer














                  You can mount a ramfs filesystem, copy your project into it and work from there. This guarantees your input files are loaded in to RAM, and they will not be re-read from the much slower disk drive. However as you discovered, this is generally not a useful strategy. You already get the exact same benefit.




                  Ramfs is a very simple filesystem that exports Linux's disk caching
                  mechanisms
                  (the page cache and dentry cache) as a dynamically resizable
                  RAM-based filesystem.




                  -- https://github.com/torvalds/linux/blob/v4.18/Documentation/filesystems/ramfs-rootfs-initramfs.txt



                  You can already trust your input files are cached in RAM, the first time they are read. Your output files are also cached, so that you do not wait for them to be written to disk.



                  There is no artificial limit on how much you can cache, how long it stays cached, etc. Caches only start to be dropped once you have filled RAM. Which cache is dropped first is chosen by terrifyingly elaborated algorithms. The first approximation, is we describe it as Least Recently Used. See What page replacement algorithms are used in Linux kernel for OS file cache?



                  Note your text editor will explicitly fsync() saved files to disk.



                  If you run tests of a program that involves fsync(), running these in a filesystem like ramfs may speed them up. Another strategy is to try and disable fsync() with eatmydata / nosync.so.



                  Some other operating systems might have specific limitations, that can be bypassed using a ramdisk. At one end, the lack of any file caching is why ramdisks were popular on DOS.



                  tmpfs



                  tmpfs works the same as ramfs, except that it can use swap space if you have one. I.e. if you need RAM for something else, the Least Recently Used algorithms may select data blocks from tmpfs and swap them to disk.



                  Most people stick with tmpfs, because it also lets you limit the total size, and shows the space used correctly e.g. in the df command. I'm not sure why this difference exists. The size limit in tmpfs protects you from accidentally filling your entire RAM and basically killing your system. It defaults to half of your RAM.



                  Other reasons why writes can slow down



                  The above is a simplification tailored to your case. The writes to files in your case should not need to wait for the disk. However there are some cases of writes that do. See the excellent blog post Why buffered writes are sometimes stalled. The most surprising case is a recent change to Linux called "stable page writes".







                  share|improve this answer














                  share|improve this answer



                  share|improve this answer








                  edited Sep 22 at 14:30

























                  answered Sep 22 at 14:18









                  sourcejedi

                  20.4k42888




                  20.4k42888



























                       

                      draft saved


                      draft discarded















































                       


                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function ()
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f66329%2fcreating-a-ram-disk-on-linux%23new-answer', 'question_page');

                      );

                      Post as a guest













































































                      Popular posts from this blog

                      How to check contact read email or not when send email to Individual?

                      Displaying single band from multi-band raster using QGIS

                      How many registers does an x86_64 CPU actually have?