Explain ZFS system output

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
4
down vote

favorite












I have output of zdb -DDD geek1 command:



[root@zfs-test centos]# zdb -DDD geek1
DDT-sha256-zap-duplicate: 8193 entries, size 290 on disk, 141 in core

bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 8K 1G 1G 1023M 24K 3G 3G 3.00G
16K 1 128K 128K 128K 16K 2G 2G 2.00G


DDT histogram (aggregated over all DDTs):

bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 8K 1G 1G 1023M 24K 3G 3G 3.00G
16K 1 128K 128K 128K 16K 2G 2G 2.00G
Total 8.00K 1.00G 1.00G 1023M 40K 5G 5G 5.00G

dedup = 5.00, compress = 1.00, copies = 1.00, dedup * compress / copies
= 5.00


I read about ZFS on internet and also here.
I created 2 files. First is random file from /dev/urandom and second one from /dev/zero. Each file have 1GB. I copied 3x urandom file and 2x zero file to my ZFS server. Minimum block size of ZFS is 128k.



I have following questions:



  1. What does refcnt means?

  2. What does DSIZE mean in allocated column and how can I count it?

  3. There was 8193 entries (unique blocks). What does numbers 290 and 141 means? According this article, I should be able to count how much data table takes in my RAM (8193x141B=1,1MB). If it is ok, what does 8193x290B=2,26MB means? Is that table also saved on disk?

  4. What compres and copies in last line means?

Please, include some example if it is possible.







share|improve this question














migrated from stackoverflow.com Nov 19 '17 at 21:22


This question came from our site for professional and enthusiast programmers.


















    up vote
    4
    down vote

    favorite












    I have output of zdb -DDD geek1 command:



    [root@zfs-test centos]# zdb -DDD geek1
    DDT-sha256-zap-duplicate: 8193 entries, size 290 on disk, 141 in core

    bucket allocated referenced
    ______ ______________________________ ______________________________
    refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
    ------ ------ ----- ----- ----- ------ ----- ----- -----
    2 8K 1G 1G 1023M 24K 3G 3G 3.00G
    16K 1 128K 128K 128K 16K 2G 2G 2.00G


    DDT histogram (aggregated over all DDTs):

    bucket allocated referenced
    ______ ______________________________ ______________________________
    refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
    ------ ------ ----- ----- ----- ------ ----- ----- -----
    2 8K 1G 1G 1023M 24K 3G 3G 3.00G
    16K 1 128K 128K 128K 16K 2G 2G 2.00G
    Total 8.00K 1.00G 1.00G 1023M 40K 5G 5G 5.00G

    dedup = 5.00, compress = 1.00, copies = 1.00, dedup * compress / copies
    = 5.00


    I read about ZFS on internet and also here.
    I created 2 files. First is random file from /dev/urandom and second one from /dev/zero. Each file have 1GB. I copied 3x urandom file and 2x zero file to my ZFS server. Minimum block size of ZFS is 128k.



    I have following questions:



    1. What does refcnt means?

    2. What does DSIZE mean in allocated column and how can I count it?

    3. There was 8193 entries (unique blocks). What does numbers 290 and 141 means? According this article, I should be able to count how much data table takes in my RAM (8193x141B=1,1MB). If it is ok, what does 8193x290B=2,26MB means? Is that table also saved on disk?

    4. What compres and copies in last line means?

    Please, include some example if it is possible.







    share|improve this question














    migrated from stackoverflow.com Nov 19 '17 at 21:22


    This question came from our site for professional and enthusiast programmers.
















      up vote
      4
      down vote

      favorite









      up vote
      4
      down vote

      favorite











      I have output of zdb -DDD geek1 command:



      [root@zfs-test centos]# zdb -DDD geek1
      DDT-sha256-zap-duplicate: 8193 entries, size 290 on disk, 141 in core

      bucket allocated referenced
      ______ ______________________________ ______________________________
      refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
      ------ ------ ----- ----- ----- ------ ----- ----- -----
      2 8K 1G 1G 1023M 24K 3G 3G 3.00G
      16K 1 128K 128K 128K 16K 2G 2G 2.00G


      DDT histogram (aggregated over all DDTs):

      bucket allocated referenced
      ______ ______________________________ ______________________________
      refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
      ------ ------ ----- ----- ----- ------ ----- ----- -----
      2 8K 1G 1G 1023M 24K 3G 3G 3.00G
      16K 1 128K 128K 128K 16K 2G 2G 2.00G
      Total 8.00K 1.00G 1.00G 1023M 40K 5G 5G 5.00G

      dedup = 5.00, compress = 1.00, copies = 1.00, dedup * compress / copies
      = 5.00


      I read about ZFS on internet and also here.
      I created 2 files. First is random file from /dev/urandom and second one from /dev/zero. Each file have 1GB. I copied 3x urandom file and 2x zero file to my ZFS server. Minimum block size of ZFS is 128k.



      I have following questions:



      1. What does refcnt means?

      2. What does DSIZE mean in allocated column and how can I count it?

      3. There was 8193 entries (unique blocks). What does numbers 290 and 141 means? According this article, I should be able to count how much data table takes in my RAM (8193x141B=1,1MB). If it is ok, what does 8193x290B=2,26MB means? Is that table also saved on disk?

      4. What compres and copies in last line means?

      Please, include some example if it is possible.







      share|improve this question














      I have output of zdb -DDD geek1 command:



      [root@zfs-test centos]# zdb -DDD geek1
      DDT-sha256-zap-duplicate: 8193 entries, size 290 on disk, 141 in core

      bucket allocated referenced
      ______ ______________________________ ______________________________
      refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
      ------ ------ ----- ----- ----- ------ ----- ----- -----
      2 8K 1G 1G 1023M 24K 3G 3G 3.00G
      16K 1 128K 128K 128K 16K 2G 2G 2.00G


      DDT histogram (aggregated over all DDTs):

      bucket allocated referenced
      ______ ______________________________ ______________________________
      refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
      ------ ------ ----- ----- ----- ------ ----- ----- -----
      2 8K 1G 1G 1023M 24K 3G 3G 3.00G
      16K 1 128K 128K 128K 16K 2G 2G 2.00G
      Total 8.00K 1.00G 1.00G 1023M 40K 5G 5G 5.00G

      dedup = 5.00, compress = 1.00, copies = 1.00, dedup * compress / copies
      = 5.00


      I read about ZFS on internet and also here.
      I created 2 files. First is random file from /dev/urandom and second one from /dev/zero. Each file have 1GB. I copied 3x urandom file and 2x zero file to my ZFS server. Minimum block size of ZFS is 128k.



      I have following questions:



      1. What does refcnt means?

      2. What does DSIZE mean in allocated column and how can I count it?

      3. There was 8193 entries (unique blocks). What does numbers 290 and 141 means? According this article, I should be able to count how much data table takes in my RAM (8193x141B=1,1MB). If it is ok, what does 8193x290B=2,26MB means? Is that table also saved on disk?

      4. What compres and copies in last line means?

      Please, include some example if it is possible.









      share|improve this question













      share|improve this question




      share|improve this question








      edited Mar 3 at 3:10









      Jeff Schaller

      32.1k849109




      32.1k849109










      asked Nov 18 '17 at 19:59









      dorinand

      313213




      313213




      migrated from stackoverflow.com Nov 19 '17 at 21:22


      This question came from our site for professional and enthusiast programmers.






      migrated from stackoverflow.com Nov 19 '17 at 21:22


      This question came from our site for professional and enthusiast programmers.






















          1 Answer
          1






          active

          oldest

          votes

















          up vote
          4
          down vote



          accepted










          1: refcnt means reference count, i.e. the number of times blocks are referenced by objects (It is a power of two progression so 1 is 1 but 2 actually means 2 or 31; 4 means 4,5,6, or 7 and so on)



          2: DSIZE= On disk size, LSIZE = logical (in memory), PSIZE = physical. On disk and physical sizes are equal or very close together unless you set zfs to have more than one copy of data blocks in which case on disk size will be larger.



          3: 290 bytes is the size a deduplication table entry uses on disk and 141 bytes is the amount of RAM an entry uses. The DDT is stored and synced on disk and indeed currently uses 2.26MB in your case.



          4: Compress means the zpool compression ratio (it is unrelated to deduplication). As compression is likely not enabled, the ratio is 1., i.e. no compression at all. Copies means the ratio of ditto blocks (kind of the opposite of deduplication) stored. There are no ditto blocks so the ratio is 1. too.



          Example of output with compression and ditto blocks enabled:



          bucket allocated referenced
          ______ ______________________________ ______________________________
          refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
          ------ ------ ----- ----- ----- ------ ----- ----- -----
          2 2 256K 49K 98K 6 768K 147K 294K
          Total 2 256K 49K 98K 6 768K 147K 294K

          dedup = 3.00, compress = 5.22, copies = 2.00, dedup * compress / copies = 7.84


          1 In your case, the count is clearly exactly equal to 3 (3 GB referenced stored in 1 GB of disk).






          share|improve this answer






















          • At first, thank you @jlliagre for great explanation. I have one more question. What is difference between LSIZE, PSIZE and DSIZE? I understand it like this: e.g block size is 4k, file is 6k, so if I uplode it, the PSIZE is 6k but LSIZE is 8k (2 blocks=2x4k=8k). So how DSIZE differ from PSIZE? Should't it be the same? I sow output where each LSIZE, PSIZE and DSIZE are differ(e.g article I mentioned above). Thank you again.
            – dorinand
            Nov 20 '17 at 17:13






          • 1




            The block size (zfs recordsize) is 128k by default but files smaller than 128k will use less. LSIZE is the cumulated size of distinct data blocks for this reference count range. PSIZE is the size that would be required to store all that data and DSIZE is the size actually used to store it.
            – jlliagre
            Nov 20 '17 at 17:47










          Your Answer







          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "106"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          convertImagesToLinks: false,
          noModals: false,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );













           

          draft saved


          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f405664%2fexplain-zfs-system-output%23new-answer', 'question_page');

          );

          Post as a guest






























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          up vote
          4
          down vote



          accepted










          1: refcnt means reference count, i.e. the number of times blocks are referenced by objects (It is a power of two progression so 1 is 1 but 2 actually means 2 or 31; 4 means 4,5,6, or 7 and so on)



          2: DSIZE= On disk size, LSIZE = logical (in memory), PSIZE = physical. On disk and physical sizes are equal or very close together unless you set zfs to have more than one copy of data blocks in which case on disk size will be larger.



          3: 290 bytes is the size a deduplication table entry uses on disk and 141 bytes is the amount of RAM an entry uses. The DDT is stored and synced on disk and indeed currently uses 2.26MB in your case.



          4: Compress means the zpool compression ratio (it is unrelated to deduplication). As compression is likely not enabled, the ratio is 1., i.e. no compression at all. Copies means the ratio of ditto blocks (kind of the opposite of deduplication) stored. There are no ditto blocks so the ratio is 1. too.



          Example of output with compression and ditto blocks enabled:



          bucket allocated referenced
          ______ ______________________________ ______________________________
          refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
          ------ ------ ----- ----- ----- ------ ----- ----- -----
          2 2 256K 49K 98K 6 768K 147K 294K
          Total 2 256K 49K 98K 6 768K 147K 294K

          dedup = 3.00, compress = 5.22, copies = 2.00, dedup * compress / copies = 7.84


          1 In your case, the count is clearly exactly equal to 3 (3 GB referenced stored in 1 GB of disk).






          share|improve this answer






















          • At first, thank you @jlliagre for great explanation. I have one more question. What is difference between LSIZE, PSIZE and DSIZE? I understand it like this: e.g block size is 4k, file is 6k, so if I uplode it, the PSIZE is 6k but LSIZE is 8k (2 blocks=2x4k=8k). So how DSIZE differ from PSIZE? Should't it be the same? I sow output where each LSIZE, PSIZE and DSIZE are differ(e.g article I mentioned above). Thank you again.
            – dorinand
            Nov 20 '17 at 17:13






          • 1




            The block size (zfs recordsize) is 128k by default but files smaller than 128k will use less. LSIZE is the cumulated size of distinct data blocks for this reference count range. PSIZE is the size that would be required to store all that data and DSIZE is the size actually used to store it.
            – jlliagre
            Nov 20 '17 at 17:47














          up vote
          4
          down vote



          accepted










          1: refcnt means reference count, i.e. the number of times blocks are referenced by objects (It is a power of two progression so 1 is 1 but 2 actually means 2 or 31; 4 means 4,5,6, or 7 and so on)



          2: DSIZE= On disk size, LSIZE = logical (in memory), PSIZE = physical. On disk and physical sizes are equal or very close together unless you set zfs to have more than one copy of data blocks in which case on disk size will be larger.



          3: 290 bytes is the size a deduplication table entry uses on disk and 141 bytes is the amount of RAM an entry uses. The DDT is stored and synced on disk and indeed currently uses 2.26MB in your case.



          4: Compress means the zpool compression ratio (it is unrelated to deduplication). As compression is likely not enabled, the ratio is 1., i.e. no compression at all. Copies means the ratio of ditto blocks (kind of the opposite of deduplication) stored. There are no ditto blocks so the ratio is 1. too.



          Example of output with compression and ditto blocks enabled:



          bucket allocated referenced
          ______ ______________________________ ______________________________
          refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
          ------ ------ ----- ----- ----- ------ ----- ----- -----
          2 2 256K 49K 98K 6 768K 147K 294K
          Total 2 256K 49K 98K 6 768K 147K 294K

          dedup = 3.00, compress = 5.22, copies = 2.00, dedup * compress / copies = 7.84


          1 In your case, the count is clearly exactly equal to 3 (3 GB referenced stored in 1 GB of disk).






          share|improve this answer






















          • At first, thank you @jlliagre for great explanation. I have one more question. What is difference between LSIZE, PSIZE and DSIZE? I understand it like this: e.g block size is 4k, file is 6k, so if I uplode it, the PSIZE is 6k but LSIZE is 8k (2 blocks=2x4k=8k). So how DSIZE differ from PSIZE? Should't it be the same? I sow output where each LSIZE, PSIZE and DSIZE are differ(e.g article I mentioned above). Thank you again.
            – dorinand
            Nov 20 '17 at 17:13






          • 1




            The block size (zfs recordsize) is 128k by default but files smaller than 128k will use less. LSIZE is the cumulated size of distinct data blocks for this reference count range. PSIZE is the size that would be required to store all that data and DSIZE is the size actually used to store it.
            – jlliagre
            Nov 20 '17 at 17:47












          up vote
          4
          down vote



          accepted







          up vote
          4
          down vote



          accepted






          1: refcnt means reference count, i.e. the number of times blocks are referenced by objects (It is a power of two progression so 1 is 1 but 2 actually means 2 or 31; 4 means 4,5,6, or 7 and so on)



          2: DSIZE= On disk size, LSIZE = logical (in memory), PSIZE = physical. On disk and physical sizes are equal or very close together unless you set zfs to have more than one copy of data blocks in which case on disk size will be larger.



          3: 290 bytes is the size a deduplication table entry uses on disk and 141 bytes is the amount of RAM an entry uses. The DDT is stored and synced on disk and indeed currently uses 2.26MB in your case.



          4: Compress means the zpool compression ratio (it is unrelated to deduplication). As compression is likely not enabled, the ratio is 1., i.e. no compression at all. Copies means the ratio of ditto blocks (kind of the opposite of deduplication) stored. There are no ditto blocks so the ratio is 1. too.



          Example of output with compression and ditto blocks enabled:



          bucket allocated referenced
          ______ ______________________________ ______________________________
          refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
          ------ ------ ----- ----- ----- ------ ----- ----- -----
          2 2 256K 49K 98K 6 768K 147K 294K
          Total 2 256K 49K 98K 6 768K 147K 294K

          dedup = 3.00, compress = 5.22, copies = 2.00, dedup * compress / copies = 7.84


          1 In your case, the count is clearly exactly equal to 3 (3 GB referenced stored in 1 GB of disk).






          share|improve this answer














          1: refcnt means reference count, i.e. the number of times blocks are referenced by objects (It is a power of two progression so 1 is 1 but 2 actually means 2 or 31; 4 means 4,5,6, or 7 and so on)



          2: DSIZE= On disk size, LSIZE = logical (in memory), PSIZE = physical. On disk and physical sizes are equal or very close together unless you set zfs to have more than one copy of data blocks in which case on disk size will be larger.



          3: 290 bytes is the size a deduplication table entry uses on disk and 141 bytes is the amount of RAM an entry uses. The DDT is stored and synced on disk and indeed currently uses 2.26MB in your case.



          4: Compress means the zpool compression ratio (it is unrelated to deduplication). As compression is likely not enabled, the ratio is 1., i.e. no compression at all. Copies means the ratio of ditto blocks (kind of the opposite of deduplication) stored. There are no ditto blocks so the ratio is 1. too.



          Example of output with compression and ditto blocks enabled:



          bucket allocated referenced
          ______ ______________________________ ______________________________
          refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
          ------ ------ ----- ----- ----- ------ ----- ----- -----
          2 2 256K 49K 98K 6 768K 147K 294K
          Total 2 256K 49K 98K 6 768K 147K 294K

          dedup = 3.00, compress = 5.22, copies = 2.00, dedup * compress / copies = 7.84


          1 In your case, the count is clearly exactly equal to 3 (3 GB referenced stored in 1 GB of disk).







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Nov 20 '17 at 13:28

























          answered Nov 20 '17 at 3:31









          jlliagre

          45k578124




          45k578124











          • At first, thank you @jlliagre for great explanation. I have one more question. What is difference between LSIZE, PSIZE and DSIZE? I understand it like this: e.g block size is 4k, file is 6k, so if I uplode it, the PSIZE is 6k but LSIZE is 8k (2 blocks=2x4k=8k). So how DSIZE differ from PSIZE? Should't it be the same? I sow output where each LSIZE, PSIZE and DSIZE are differ(e.g article I mentioned above). Thank you again.
            – dorinand
            Nov 20 '17 at 17:13






          • 1




            The block size (zfs recordsize) is 128k by default but files smaller than 128k will use less. LSIZE is the cumulated size of distinct data blocks for this reference count range. PSIZE is the size that would be required to store all that data and DSIZE is the size actually used to store it.
            – jlliagre
            Nov 20 '17 at 17:47
















          • At first, thank you @jlliagre for great explanation. I have one more question. What is difference between LSIZE, PSIZE and DSIZE? I understand it like this: e.g block size is 4k, file is 6k, so if I uplode it, the PSIZE is 6k but LSIZE is 8k (2 blocks=2x4k=8k). So how DSIZE differ from PSIZE? Should't it be the same? I sow output where each LSIZE, PSIZE and DSIZE are differ(e.g article I mentioned above). Thank you again.
            – dorinand
            Nov 20 '17 at 17:13






          • 1




            The block size (zfs recordsize) is 128k by default but files smaller than 128k will use less. LSIZE is the cumulated size of distinct data blocks for this reference count range. PSIZE is the size that would be required to store all that data and DSIZE is the size actually used to store it.
            – jlliagre
            Nov 20 '17 at 17:47















          At first, thank you @jlliagre for great explanation. I have one more question. What is difference between LSIZE, PSIZE and DSIZE? I understand it like this: e.g block size is 4k, file is 6k, so if I uplode it, the PSIZE is 6k but LSIZE is 8k (2 blocks=2x4k=8k). So how DSIZE differ from PSIZE? Should't it be the same? I sow output where each LSIZE, PSIZE and DSIZE are differ(e.g article I mentioned above). Thank you again.
          – dorinand
          Nov 20 '17 at 17:13




          At first, thank you @jlliagre for great explanation. I have one more question. What is difference between LSIZE, PSIZE and DSIZE? I understand it like this: e.g block size is 4k, file is 6k, so if I uplode it, the PSIZE is 6k but LSIZE is 8k (2 blocks=2x4k=8k). So how DSIZE differ from PSIZE? Should't it be the same? I sow output where each LSIZE, PSIZE and DSIZE are differ(e.g article I mentioned above). Thank you again.
          – dorinand
          Nov 20 '17 at 17:13




          1




          1




          The block size (zfs recordsize) is 128k by default but files smaller than 128k will use less. LSIZE is the cumulated size of distinct data blocks for this reference count range. PSIZE is the size that would be required to store all that data and DSIZE is the size actually used to store it.
          – jlliagre
          Nov 20 '17 at 17:47




          The block size (zfs recordsize) is 128k by default but files smaller than 128k will use less. LSIZE is the cumulated size of distinct data blocks for this reference count range. PSIZE is the size that would be required to store all that data and DSIZE is the size actually used to store it.
          – jlliagre
          Nov 20 '17 at 17:47

















           

          draft saved


          draft discarded















































           


          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f405664%2fexplain-zfs-system-output%23new-answer', 'question_page');

          );

          Post as a guest













































































          Popular posts from this blog

          How to check contact read email or not when send email to Individual?

          Displaying single band from multi-band raster using QGIS

          How many registers does an x86_64 CPU actually have?