Explain ZFS system output
Clash Royale CLAN TAG#URR8PPP
up vote
4
down vote
favorite
I have output of zdb -DDD geek1
command:
[root@zfs-test centos]# zdb -DDD geek1
DDT-sha256-zap-duplicate: 8193 entries, size 290 on disk, 141 in core
bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 8K 1G 1G 1023M 24K 3G 3G 3.00G
16K 1 128K 128K 128K 16K 2G 2G 2.00G
DDT histogram (aggregated over all DDTs):
bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 8K 1G 1G 1023M 24K 3G 3G 3.00G
16K 1 128K 128K 128K 16K 2G 2G 2.00G
Total 8.00K 1.00G 1.00G 1023M 40K 5G 5G 5.00G
dedup = 5.00, compress = 1.00, copies = 1.00, dedup * compress / copies
= 5.00
I read about ZFS on internet and also here.
I created 2 files. First is random file from /dev/urandom and second one from /dev/zero. Each file have 1GB. I copied 3x urandom file and 2x zero file to my ZFS server. Minimum block size of ZFS is 128k.
I have following questions:
- What does
refcnt
means? - What does
DSIZE
mean in allocated column and how can I count it? - There was 8193 entries (unique blocks). What does numbers 290 and 141 means? According this article, I should be able to count how much data table takes in my RAM (8193x141B=1,1MB). If it is ok, what does 8193x290B=2,26MB means? Is that table also saved on disk?
- What
compres
andcopies
in last line means?
Please, include some example if it is possible.
linux zfs
migrated from stackoverflow.com Nov 19 '17 at 21:22
This question came from our site for professional and enthusiast programmers.
add a comment |Â
up vote
4
down vote
favorite
I have output of zdb -DDD geek1
command:
[root@zfs-test centos]# zdb -DDD geek1
DDT-sha256-zap-duplicate: 8193 entries, size 290 on disk, 141 in core
bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 8K 1G 1G 1023M 24K 3G 3G 3.00G
16K 1 128K 128K 128K 16K 2G 2G 2.00G
DDT histogram (aggregated over all DDTs):
bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 8K 1G 1G 1023M 24K 3G 3G 3.00G
16K 1 128K 128K 128K 16K 2G 2G 2.00G
Total 8.00K 1.00G 1.00G 1023M 40K 5G 5G 5.00G
dedup = 5.00, compress = 1.00, copies = 1.00, dedup * compress / copies
= 5.00
I read about ZFS on internet and also here.
I created 2 files. First is random file from /dev/urandom and second one from /dev/zero. Each file have 1GB. I copied 3x urandom file and 2x zero file to my ZFS server. Minimum block size of ZFS is 128k.
I have following questions:
- What does
refcnt
means? - What does
DSIZE
mean in allocated column and how can I count it? - There was 8193 entries (unique blocks). What does numbers 290 and 141 means? According this article, I should be able to count how much data table takes in my RAM (8193x141B=1,1MB). If it is ok, what does 8193x290B=2,26MB means? Is that table also saved on disk?
- What
compres
andcopies
in last line means?
Please, include some example if it is possible.
linux zfs
migrated from stackoverflow.com Nov 19 '17 at 21:22
This question came from our site for professional and enthusiast programmers.
add a comment |Â
up vote
4
down vote
favorite
up vote
4
down vote
favorite
I have output of zdb -DDD geek1
command:
[root@zfs-test centos]# zdb -DDD geek1
DDT-sha256-zap-duplicate: 8193 entries, size 290 on disk, 141 in core
bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 8K 1G 1G 1023M 24K 3G 3G 3.00G
16K 1 128K 128K 128K 16K 2G 2G 2.00G
DDT histogram (aggregated over all DDTs):
bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 8K 1G 1G 1023M 24K 3G 3G 3.00G
16K 1 128K 128K 128K 16K 2G 2G 2.00G
Total 8.00K 1.00G 1.00G 1023M 40K 5G 5G 5.00G
dedup = 5.00, compress = 1.00, copies = 1.00, dedup * compress / copies
= 5.00
I read about ZFS on internet and also here.
I created 2 files. First is random file from /dev/urandom and second one from /dev/zero. Each file have 1GB. I copied 3x urandom file and 2x zero file to my ZFS server. Minimum block size of ZFS is 128k.
I have following questions:
- What does
refcnt
means? - What does
DSIZE
mean in allocated column and how can I count it? - There was 8193 entries (unique blocks). What does numbers 290 and 141 means? According this article, I should be able to count how much data table takes in my RAM (8193x141B=1,1MB). If it is ok, what does 8193x290B=2,26MB means? Is that table also saved on disk?
- What
compres
andcopies
in last line means?
Please, include some example if it is possible.
linux zfs
I have output of zdb -DDD geek1
command:
[root@zfs-test centos]# zdb -DDD geek1
DDT-sha256-zap-duplicate: 8193 entries, size 290 on disk, 141 in core
bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 8K 1G 1G 1023M 24K 3G 3G 3.00G
16K 1 128K 128K 128K 16K 2G 2G 2.00G
DDT histogram (aggregated over all DDTs):
bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 8K 1G 1G 1023M 24K 3G 3G 3.00G
16K 1 128K 128K 128K 16K 2G 2G 2.00G
Total 8.00K 1.00G 1.00G 1023M 40K 5G 5G 5.00G
dedup = 5.00, compress = 1.00, copies = 1.00, dedup * compress / copies
= 5.00
I read about ZFS on internet and also here.
I created 2 files. First is random file from /dev/urandom and second one from /dev/zero. Each file have 1GB. I copied 3x urandom file and 2x zero file to my ZFS server. Minimum block size of ZFS is 128k.
I have following questions:
- What does
refcnt
means? - What does
DSIZE
mean in allocated column and how can I count it? - There was 8193 entries (unique blocks). What does numbers 290 and 141 means? According this article, I should be able to count how much data table takes in my RAM (8193x141B=1,1MB). If it is ok, what does 8193x290B=2,26MB means? Is that table also saved on disk?
- What
compres
andcopies
in last line means?
Please, include some example if it is possible.
linux zfs
edited Mar 3 at 3:10
Jeff Schaller
32.1k849109
32.1k849109
asked Nov 18 '17 at 19:59
dorinand
313213
313213
migrated from stackoverflow.com Nov 19 '17 at 21:22
This question came from our site for professional and enthusiast programmers.
migrated from stackoverflow.com Nov 19 '17 at 21:22
This question came from our site for professional and enthusiast programmers.
add a comment |Â
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
4
down vote
accepted
1: refcnt
means reference count, i.e. the number of times blocks are referenced by objects (It is a power of two progression so 1
is 1
but 2
actually means 2
or 3
1; 4
means 4
,5
,6
, or 7
and so on)
2: DSIZE
= On disk size, LSIZE
= logical (in memory), PSIZE
= physical. On disk and physical sizes are equal or very close together unless you set zfs to have more than one copy of data blocks in which case on disk size will be larger.
3: 290 bytes is the size a deduplication table entry uses on disk and 141 bytes is the amount of RAM an entry uses. The DDT
is stored and synced on disk and indeed currently uses 2.26MB in your case.
4: Compress
means the zpool compression ratio (it is unrelated to deduplication). As compression is likely not enabled, the ratio is 1., i.e. no compression at all. Copies means the ratio of ditto blocks (kind of the opposite of deduplication) stored. There are no ditto blocks so the ratio is 1. too.
Example of output with compression and ditto blocks enabled:
bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 2 256K 49K 98K 6 768K 147K 294K
Total 2 256K 49K 98K 6 768K 147K 294K
dedup = 3.00, compress = 5.22, copies = 2.00, dedup * compress / copies = 7.84
1 In your case, the count is clearly exactly equal to 3
(3 GB referenced stored in 1 GB of disk).
At first, thank you @jlliagre for great explanation. I have one more question. What is difference between LSIZE, PSIZE and DSIZE? I understand it like this: e.g block size is 4k, file is 6k, so if I uplode it, the PSIZE is 6k but LSIZE is 8k (2 blocks=2x4k=8k). So how DSIZE differ from PSIZE? Should't it be the same? I sow output where each LSIZE, PSIZE and DSIZE are differ(e.g article I mentioned above). Thank you again.
â dorinand
Nov 20 '17 at 17:13
1
The block size (zfs recordsize) is 128k by default but files smaller than 128k will use less. LSIZE is the cumulated size of distinct data blocks for this reference count range. PSIZE is the size that would be required to store all that data and DSIZE is the size actually used to store it.
â jlliagre
Nov 20 '17 at 17:47
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
4
down vote
accepted
1: refcnt
means reference count, i.e. the number of times blocks are referenced by objects (It is a power of two progression so 1
is 1
but 2
actually means 2
or 3
1; 4
means 4
,5
,6
, or 7
and so on)
2: DSIZE
= On disk size, LSIZE
= logical (in memory), PSIZE
= physical. On disk and physical sizes are equal or very close together unless you set zfs to have more than one copy of data blocks in which case on disk size will be larger.
3: 290 bytes is the size a deduplication table entry uses on disk and 141 bytes is the amount of RAM an entry uses. The DDT
is stored and synced on disk and indeed currently uses 2.26MB in your case.
4: Compress
means the zpool compression ratio (it is unrelated to deduplication). As compression is likely not enabled, the ratio is 1., i.e. no compression at all. Copies means the ratio of ditto blocks (kind of the opposite of deduplication) stored. There are no ditto blocks so the ratio is 1. too.
Example of output with compression and ditto blocks enabled:
bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 2 256K 49K 98K 6 768K 147K 294K
Total 2 256K 49K 98K 6 768K 147K 294K
dedup = 3.00, compress = 5.22, copies = 2.00, dedup * compress / copies = 7.84
1 In your case, the count is clearly exactly equal to 3
(3 GB referenced stored in 1 GB of disk).
At first, thank you @jlliagre for great explanation. I have one more question. What is difference between LSIZE, PSIZE and DSIZE? I understand it like this: e.g block size is 4k, file is 6k, so if I uplode it, the PSIZE is 6k but LSIZE is 8k (2 blocks=2x4k=8k). So how DSIZE differ from PSIZE? Should't it be the same? I sow output where each LSIZE, PSIZE and DSIZE are differ(e.g article I mentioned above). Thank you again.
â dorinand
Nov 20 '17 at 17:13
1
The block size (zfs recordsize) is 128k by default but files smaller than 128k will use less. LSIZE is the cumulated size of distinct data blocks for this reference count range. PSIZE is the size that would be required to store all that data and DSIZE is the size actually used to store it.
â jlliagre
Nov 20 '17 at 17:47
add a comment |Â
up vote
4
down vote
accepted
1: refcnt
means reference count, i.e. the number of times blocks are referenced by objects (It is a power of two progression so 1
is 1
but 2
actually means 2
or 3
1; 4
means 4
,5
,6
, or 7
and so on)
2: DSIZE
= On disk size, LSIZE
= logical (in memory), PSIZE
= physical. On disk and physical sizes are equal or very close together unless you set zfs to have more than one copy of data blocks in which case on disk size will be larger.
3: 290 bytes is the size a deduplication table entry uses on disk and 141 bytes is the amount of RAM an entry uses. The DDT
is stored and synced on disk and indeed currently uses 2.26MB in your case.
4: Compress
means the zpool compression ratio (it is unrelated to deduplication). As compression is likely not enabled, the ratio is 1., i.e. no compression at all. Copies means the ratio of ditto blocks (kind of the opposite of deduplication) stored. There are no ditto blocks so the ratio is 1. too.
Example of output with compression and ditto blocks enabled:
bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 2 256K 49K 98K 6 768K 147K 294K
Total 2 256K 49K 98K 6 768K 147K 294K
dedup = 3.00, compress = 5.22, copies = 2.00, dedup * compress / copies = 7.84
1 In your case, the count is clearly exactly equal to 3
(3 GB referenced stored in 1 GB of disk).
At first, thank you @jlliagre for great explanation. I have one more question. What is difference between LSIZE, PSIZE and DSIZE? I understand it like this: e.g block size is 4k, file is 6k, so if I uplode it, the PSIZE is 6k but LSIZE is 8k (2 blocks=2x4k=8k). So how DSIZE differ from PSIZE? Should't it be the same? I sow output where each LSIZE, PSIZE and DSIZE are differ(e.g article I mentioned above). Thank you again.
â dorinand
Nov 20 '17 at 17:13
1
The block size (zfs recordsize) is 128k by default but files smaller than 128k will use less. LSIZE is the cumulated size of distinct data blocks for this reference count range. PSIZE is the size that would be required to store all that data and DSIZE is the size actually used to store it.
â jlliagre
Nov 20 '17 at 17:47
add a comment |Â
up vote
4
down vote
accepted
up vote
4
down vote
accepted
1: refcnt
means reference count, i.e. the number of times blocks are referenced by objects (It is a power of two progression so 1
is 1
but 2
actually means 2
or 3
1; 4
means 4
,5
,6
, or 7
and so on)
2: DSIZE
= On disk size, LSIZE
= logical (in memory), PSIZE
= physical. On disk and physical sizes are equal or very close together unless you set zfs to have more than one copy of data blocks in which case on disk size will be larger.
3: 290 bytes is the size a deduplication table entry uses on disk and 141 bytes is the amount of RAM an entry uses. The DDT
is stored and synced on disk and indeed currently uses 2.26MB in your case.
4: Compress
means the zpool compression ratio (it is unrelated to deduplication). As compression is likely not enabled, the ratio is 1., i.e. no compression at all. Copies means the ratio of ditto blocks (kind of the opposite of deduplication) stored. There are no ditto blocks so the ratio is 1. too.
Example of output with compression and ditto blocks enabled:
bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 2 256K 49K 98K 6 768K 147K 294K
Total 2 256K 49K 98K 6 768K 147K 294K
dedup = 3.00, compress = 5.22, copies = 2.00, dedup * compress / copies = 7.84
1 In your case, the count is clearly exactly equal to 3
(3 GB referenced stored in 1 GB of disk).
1: refcnt
means reference count, i.e. the number of times blocks are referenced by objects (It is a power of two progression so 1
is 1
but 2
actually means 2
or 3
1; 4
means 4
,5
,6
, or 7
and so on)
2: DSIZE
= On disk size, LSIZE
= logical (in memory), PSIZE
= physical. On disk and physical sizes are equal or very close together unless you set zfs to have more than one copy of data blocks in which case on disk size will be larger.
3: 290 bytes is the size a deduplication table entry uses on disk and 141 bytes is the amount of RAM an entry uses. The DDT
is stored and synced on disk and indeed currently uses 2.26MB in your case.
4: Compress
means the zpool compression ratio (it is unrelated to deduplication). As compression is likely not enabled, the ratio is 1., i.e. no compression at all. Copies means the ratio of ditto blocks (kind of the opposite of deduplication) stored. There are no ditto blocks so the ratio is 1. too.
Example of output with compression and ditto blocks enabled:
bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 2 256K 49K 98K 6 768K 147K 294K
Total 2 256K 49K 98K 6 768K 147K 294K
dedup = 3.00, compress = 5.22, copies = 2.00, dedup * compress / copies = 7.84
1 In your case, the count is clearly exactly equal to 3
(3 GB referenced stored in 1 GB of disk).
edited Nov 20 '17 at 13:28
answered Nov 20 '17 at 3:31
jlliagre
45k578124
45k578124
At first, thank you @jlliagre for great explanation. I have one more question. What is difference between LSIZE, PSIZE and DSIZE? I understand it like this: e.g block size is 4k, file is 6k, so if I uplode it, the PSIZE is 6k but LSIZE is 8k (2 blocks=2x4k=8k). So how DSIZE differ from PSIZE? Should't it be the same? I sow output where each LSIZE, PSIZE and DSIZE are differ(e.g article I mentioned above). Thank you again.
â dorinand
Nov 20 '17 at 17:13
1
The block size (zfs recordsize) is 128k by default but files smaller than 128k will use less. LSIZE is the cumulated size of distinct data blocks for this reference count range. PSIZE is the size that would be required to store all that data and DSIZE is the size actually used to store it.
â jlliagre
Nov 20 '17 at 17:47
add a comment |Â
At first, thank you @jlliagre for great explanation. I have one more question. What is difference between LSIZE, PSIZE and DSIZE? I understand it like this: e.g block size is 4k, file is 6k, so if I uplode it, the PSIZE is 6k but LSIZE is 8k (2 blocks=2x4k=8k). So how DSIZE differ from PSIZE? Should't it be the same? I sow output where each LSIZE, PSIZE and DSIZE are differ(e.g article I mentioned above). Thank you again.
â dorinand
Nov 20 '17 at 17:13
1
The block size (zfs recordsize) is 128k by default but files smaller than 128k will use less. LSIZE is the cumulated size of distinct data blocks for this reference count range. PSIZE is the size that would be required to store all that data and DSIZE is the size actually used to store it.
â jlliagre
Nov 20 '17 at 17:47
At first, thank you @jlliagre for great explanation. I have one more question. What is difference between LSIZE, PSIZE and DSIZE? I understand it like this: e.g block size is 4k, file is 6k, so if I uplode it, the PSIZE is 6k but LSIZE is 8k (2 blocks=2x4k=8k). So how DSIZE differ from PSIZE? Should't it be the same? I sow output where each LSIZE, PSIZE and DSIZE are differ(e.g article I mentioned above). Thank you again.
â dorinand
Nov 20 '17 at 17:13
At first, thank you @jlliagre for great explanation. I have one more question. What is difference between LSIZE, PSIZE and DSIZE? I understand it like this: e.g block size is 4k, file is 6k, so if I uplode it, the PSIZE is 6k but LSIZE is 8k (2 blocks=2x4k=8k). So how DSIZE differ from PSIZE? Should't it be the same? I sow output where each LSIZE, PSIZE and DSIZE are differ(e.g article I mentioned above). Thank you again.
â dorinand
Nov 20 '17 at 17:13
1
1
The block size (zfs recordsize) is 128k by default but files smaller than 128k will use less. LSIZE is the cumulated size of distinct data blocks for this reference count range. PSIZE is the size that would be required to store all that data and DSIZE is the size actually used to store it.
â jlliagre
Nov 20 '17 at 17:47
The block size (zfs recordsize) is 128k by default but files smaller than 128k will use less. LSIZE is the cumulated size of distinct data blocks for this reference count range. PSIZE is the size that would be required to store all that data and DSIZE is the size actually used to store it.
â jlliagre
Nov 20 '17 at 17:47
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f405664%2fexplain-zfs-system-output%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password