Partitioning and formatting a 4k-emulated (512e) HDD
Clash Royale CLAN TAG#URR8PPP
I just plugged into USB A 3.0 / C 3.1 my new external HDD to Debian Buster system.
The disk was sold as LaCie 2.5" Porsche Design P'9227 2TB USB-C.
Here is the output of fdisk -l /dev/sdc
:
Disk /dev/sdc: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: P9227 Slim
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
I just read some articles about 4k-emulated drives (512e), this one should be the case.
I am confused as to how to format it with NTFS.
I tried to use my brain, and here is what I came with:
Start sector of the partition should probably start on 4096 sector (?)
So I created a partition with
gdisk
like this:Device Start End Sectors Size Type
/dev/sdc1 4096 3907029134 3907025039 1.8T Microsoft basic dataSector size should probably be forced with the
--sector-size
option like I did (?) issuing:mkfs.ntfs --no-indexing --verbose --with-uuid --label EXTERNAL_2TB --quick --sector-size 4096 /dev/sdc1
EDIT1:
Windows 10 fully updated did not recognize the partition and asked me to format, I used my favorite tool for that, and back to Linux here is the output of fdisk -l /dev/sdc
:
Device Start End Sectors Size Type
/dev/sdc1 2048 3907028991 3907026944 1,8T Microsoft basic data
So why it must start at sector 2048, I don't understand.
EDIT2:
I don't understand what I am doing wrong in terms of compatibility with Windows. Every time I re-partition it / re-format it and boot Windows and plug the drive in, it just offers me to Format it itself.
I am quite positive I tried everything from inside gdisk
+ mkfs.ntfs
.
I would like to know why I am unable to do the same as Windows does from my Linux CLI.
I will answer all questions tomorrow morning as well as comments.
I am now running:
pv --progress --timer --eta --rate --average-rate --bytes -s 1953314876k < /dev/zero > /media/vlastimil/LACIE_2TB/zero
with an expected speed of 123 MiB/s.
partition hard-disk ntfs mkfs gdisk
add a comment |
I just plugged into USB A 3.0 / C 3.1 my new external HDD to Debian Buster system.
The disk was sold as LaCie 2.5" Porsche Design P'9227 2TB USB-C.
Here is the output of fdisk -l /dev/sdc
:
Disk /dev/sdc: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: P9227 Slim
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
I just read some articles about 4k-emulated drives (512e), this one should be the case.
I am confused as to how to format it with NTFS.
I tried to use my brain, and here is what I came with:
Start sector of the partition should probably start on 4096 sector (?)
So I created a partition with
gdisk
like this:Device Start End Sectors Size Type
/dev/sdc1 4096 3907029134 3907025039 1.8T Microsoft basic dataSector size should probably be forced with the
--sector-size
option like I did (?) issuing:mkfs.ntfs --no-indexing --verbose --with-uuid --label EXTERNAL_2TB --quick --sector-size 4096 /dev/sdc1
EDIT1:
Windows 10 fully updated did not recognize the partition and asked me to format, I used my favorite tool for that, and back to Linux here is the output of fdisk -l /dev/sdc
:
Device Start End Sectors Size Type
/dev/sdc1 2048 3907028991 3907026944 1,8T Microsoft basic data
So why it must start at sector 2048, I don't understand.
EDIT2:
I don't understand what I am doing wrong in terms of compatibility with Windows. Every time I re-partition it / re-format it and boot Windows and plug the drive in, it just offers me to Format it itself.
I am quite positive I tried everything from inside gdisk
+ mkfs.ntfs
.
I would like to know why I am unable to do the same as Windows does from my Linux CLI.
I will answer all questions tomorrow morning as well as comments.
I am now running:
pv --progress --timer --eta --rate --average-rate --bytes -s 1953314876k < /dev/zero > /media/vlastimil/LACIE_2TB/zero
with an expected speed of 123 MiB/s.
partition hard-disk ntfs mkfs gdisk
1
The only weird thing here is the bogus "optimal io size". Nothing wrong with start sector 2048 or 4096 (MiB alignment). As to why Windows doesn't like it, who knows?
– frostschutz
Jan 27 at 19:37
2
There is no need for the partition to start on sector 4096. The only requirement for optimal alignment is that the starting sector number is divisible by eight. The physical sectors are aligned to logical sectors N*8. 4096/512 = 8.
– Johan Myréen
Jan 27 at 20:34
I fail to understand why the disk not working in Windows is a concern here, do not understand why you are finding it strange it starting at sector 2048 (it is pretty normal), and I think you are mixing up logical and physical sectors. However, you true problem is that you external adapter sees the disk in a different way than pluggin it in your Windows machine. There is another question that talks about that, but there are people here that are able to explain it better.
– Rui F Ribeiro
Jan 27 at 20:34
add a comment |
I just plugged into USB A 3.0 / C 3.1 my new external HDD to Debian Buster system.
The disk was sold as LaCie 2.5" Porsche Design P'9227 2TB USB-C.
Here is the output of fdisk -l /dev/sdc
:
Disk /dev/sdc: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: P9227 Slim
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
I just read some articles about 4k-emulated drives (512e), this one should be the case.
I am confused as to how to format it with NTFS.
I tried to use my brain, and here is what I came with:
Start sector of the partition should probably start on 4096 sector (?)
So I created a partition with
gdisk
like this:Device Start End Sectors Size Type
/dev/sdc1 4096 3907029134 3907025039 1.8T Microsoft basic dataSector size should probably be forced with the
--sector-size
option like I did (?) issuing:mkfs.ntfs --no-indexing --verbose --with-uuid --label EXTERNAL_2TB --quick --sector-size 4096 /dev/sdc1
EDIT1:
Windows 10 fully updated did not recognize the partition and asked me to format, I used my favorite tool for that, and back to Linux here is the output of fdisk -l /dev/sdc
:
Device Start End Sectors Size Type
/dev/sdc1 2048 3907028991 3907026944 1,8T Microsoft basic data
So why it must start at sector 2048, I don't understand.
EDIT2:
I don't understand what I am doing wrong in terms of compatibility with Windows. Every time I re-partition it / re-format it and boot Windows and plug the drive in, it just offers me to Format it itself.
I am quite positive I tried everything from inside gdisk
+ mkfs.ntfs
.
I would like to know why I am unable to do the same as Windows does from my Linux CLI.
I will answer all questions tomorrow morning as well as comments.
I am now running:
pv --progress --timer --eta --rate --average-rate --bytes -s 1953314876k < /dev/zero > /media/vlastimil/LACIE_2TB/zero
with an expected speed of 123 MiB/s.
partition hard-disk ntfs mkfs gdisk
I just plugged into USB A 3.0 / C 3.1 my new external HDD to Debian Buster system.
The disk was sold as LaCie 2.5" Porsche Design P'9227 2TB USB-C.
Here is the output of fdisk -l /dev/sdc
:
Disk /dev/sdc: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: P9227 Slim
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
I just read some articles about 4k-emulated drives (512e), this one should be the case.
I am confused as to how to format it with NTFS.
I tried to use my brain, and here is what I came with:
Start sector of the partition should probably start on 4096 sector (?)
So I created a partition with
gdisk
like this:Device Start End Sectors Size Type
/dev/sdc1 4096 3907029134 3907025039 1.8T Microsoft basic dataSector size should probably be forced with the
--sector-size
option like I did (?) issuing:mkfs.ntfs --no-indexing --verbose --with-uuid --label EXTERNAL_2TB --quick --sector-size 4096 /dev/sdc1
EDIT1:
Windows 10 fully updated did not recognize the partition and asked me to format, I used my favorite tool for that, and back to Linux here is the output of fdisk -l /dev/sdc
:
Device Start End Sectors Size Type
/dev/sdc1 2048 3907028991 3907026944 1,8T Microsoft basic data
So why it must start at sector 2048, I don't understand.
EDIT2:
I don't understand what I am doing wrong in terms of compatibility with Windows. Every time I re-partition it / re-format it and boot Windows and plug the drive in, it just offers me to Format it itself.
I am quite positive I tried everything from inside gdisk
+ mkfs.ntfs
.
I would like to know why I am unable to do the same as Windows does from my Linux CLI.
I will answer all questions tomorrow morning as well as comments.
I am now running:
pv --progress --timer --eta --rate --average-rate --bytes -s 1953314876k < /dev/zero > /media/vlastimil/LACIE_2TB/zero
with an expected speed of 123 MiB/s.
partition hard-disk ntfs mkfs gdisk
partition hard-disk ntfs mkfs gdisk
edited Jan 27 at 20:34
Vlastimil
asked Jan 27 at 18:36
VlastimilVlastimil
8,0721464138
8,0721464138
1
The only weird thing here is the bogus "optimal io size". Nothing wrong with start sector 2048 or 4096 (MiB alignment). As to why Windows doesn't like it, who knows?
– frostschutz
Jan 27 at 19:37
2
There is no need for the partition to start on sector 4096. The only requirement for optimal alignment is that the starting sector number is divisible by eight. The physical sectors are aligned to logical sectors N*8. 4096/512 = 8.
– Johan Myréen
Jan 27 at 20:34
I fail to understand why the disk not working in Windows is a concern here, do not understand why you are finding it strange it starting at sector 2048 (it is pretty normal), and I think you are mixing up logical and physical sectors. However, you true problem is that you external adapter sees the disk in a different way than pluggin it in your Windows machine. There is another question that talks about that, but there are people here that are able to explain it better.
– Rui F Ribeiro
Jan 27 at 20:34
add a comment |
1
The only weird thing here is the bogus "optimal io size". Nothing wrong with start sector 2048 or 4096 (MiB alignment). As to why Windows doesn't like it, who knows?
– frostschutz
Jan 27 at 19:37
2
There is no need for the partition to start on sector 4096. The only requirement for optimal alignment is that the starting sector number is divisible by eight. The physical sectors are aligned to logical sectors N*8. 4096/512 = 8.
– Johan Myréen
Jan 27 at 20:34
I fail to understand why the disk not working in Windows is a concern here, do not understand why you are finding it strange it starting at sector 2048 (it is pretty normal), and I think you are mixing up logical and physical sectors. However, you true problem is that you external adapter sees the disk in a different way than pluggin it in your Windows machine. There is another question that talks about that, but there are people here that are able to explain it better.
– Rui F Ribeiro
Jan 27 at 20:34
1
1
The only weird thing here is the bogus "optimal io size". Nothing wrong with start sector 2048 or 4096 (MiB alignment). As to why Windows doesn't like it, who knows?
– frostschutz
Jan 27 at 19:37
The only weird thing here is the bogus "optimal io size". Nothing wrong with start sector 2048 or 4096 (MiB alignment). As to why Windows doesn't like it, who knows?
– frostschutz
Jan 27 at 19:37
2
2
There is no need for the partition to start on sector 4096. The only requirement for optimal alignment is that the starting sector number is divisible by eight. The physical sectors are aligned to logical sectors N*8. 4096/512 = 8.
– Johan Myréen
Jan 27 at 20:34
There is no need for the partition to start on sector 4096. The only requirement for optimal alignment is that the starting sector number is divisible by eight. The physical sectors are aligned to logical sectors N*8. 4096/512 = 8.
– Johan Myréen
Jan 27 at 20:34
I fail to understand why the disk not working in Windows is a concern here, do not understand why you are finding it strange it starting at sector 2048 (it is pretty normal), and I think you are mixing up logical and physical sectors. However, you true problem is that you external adapter sees the disk in a different way than pluggin it in your Windows machine. There is another question that talks about that, but there are people here that are able to explain it better.
– Rui F Ribeiro
Jan 27 at 20:34
I fail to understand why the disk not working in Windows is a concern here, do not understand why you are finding it strange it starting at sector 2048 (it is pretty normal), and I think you are mixing up logical and physical sectors. However, you true problem is that you external adapter sees the disk in a different way than pluggin it in your Windows machine. There is another question that talks about that, but there are people here that are able to explain it better.
– Rui F Ribeiro
Jan 27 at 20:34
add a comment |
1 Answer
1
active
oldest
votes
A physical sector size of 4096 means that the data on the drive is laid out in units of 4096 bytes, i.e. disk comprised of sequential "compartments" of 4096 bytes, that have to be written atomically. For compatibility reasons, most disks with 4096 byte sectors present themselves as having traditional 512 byte "logical sectors", which means the addressing unit is a 512 byte block.
The practical implication of this emulation of a 512 sector drive with an underlying disk with 4096 byte sectors is a potential performance issue. When writing a single 512 byte sector to a 512e disk, the drive must read the whole 4096 byte sector containing the 512-byte sector, modify the sector in RAM (on the disk controller) by replacing the 512-byte sector with the new contents, and finally write the whole 4096 sector back to the disk. Things get worse if you are reading or writing a couple of consecutive 512 sectors that happen to cross a 4096 sector boundary.
File systems usually lay out their data structures well, i.e. they are aligned to multiples of at least 4096 bytes, so the bigger sector size normally does not present a problem. This all breaks down, however, if the partition containing the file system itself is not aligned properly. In the case of a 512e disk, the partitions should be aligned so that the first 512-byte logical sector number is a multiple of eight.
Good thinking. This is a known issue, specially when aligning VMs because all the improvement gains add up and benefit the infra-structure as a whole. +1
– Rui F Ribeiro
Jan 28 at 17:49
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f497063%2fpartitioning-and-formatting-a-4k-emulated-512e-hdd%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
A physical sector size of 4096 means that the data on the drive is laid out in units of 4096 bytes, i.e. disk comprised of sequential "compartments" of 4096 bytes, that have to be written atomically. For compatibility reasons, most disks with 4096 byte sectors present themselves as having traditional 512 byte "logical sectors", which means the addressing unit is a 512 byte block.
The practical implication of this emulation of a 512 sector drive with an underlying disk with 4096 byte sectors is a potential performance issue. When writing a single 512 byte sector to a 512e disk, the drive must read the whole 4096 byte sector containing the 512-byte sector, modify the sector in RAM (on the disk controller) by replacing the 512-byte sector with the new contents, and finally write the whole 4096 sector back to the disk. Things get worse if you are reading or writing a couple of consecutive 512 sectors that happen to cross a 4096 sector boundary.
File systems usually lay out their data structures well, i.e. they are aligned to multiples of at least 4096 bytes, so the bigger sector size normally does not present a problem. This all breaks down, however, if the partition containing the file system itself is not aligned properly. In the case of a 512e disk, the partitions should be aligned so that the first 512-byte logical sector number is a multiple of eight.
Good thinking. This is a known issue, specially when aligning VMs because all the improvement gains add up and benefit the infra-structure as a whole. +1
– Rui F Ribeiro
Jan 28 at 17:49
add a comment |
A physical sector size of 4096 means that the data on the drive is laid out in units of 4096 bytes, i.e. disk comprised of sequential "compartments" of 4096 bytes, that have to be written atomically. For compatibility reasons, most disks with 4096 byte sectors present themselves as having traditional 512 byte "logical sectors", which means the addressing unit is a 512 byte block.
The practical implication of this emulation of a 512 sector drive with an underlying disk with 4096 byte sectors is a potential performance issue. When writing a single 512 byte sector to a 512e disk, the drive must read the whole 4096 byte sector containing the 512-byte sector, modify the sector in RAM (on the disk controller) by replacing the 512-byte sector with the new contents, and finally write the whole 4096 sector back to the disk. Things get worse if you are reading or writing a couple of consecutive 512 sectors that happen to cross a 4096 sector boundary.
File systems usually lay out their data structures well, i.e. they are aligned to multiples of at least 4096 bytes, so the bigger sector size normally does not present a problem. This all breaks down, however, if the partition containing the file system itself is not aligned properly. In the case of a 512e disk, the partitions should be aligned so that the first 512-byte logical sector number is a multiple of eight.
Good thinking. This is a known issue, specially when aligning VMs because all the improvement gains add up and benefit the infra-structure as a whole. +1
– Rui F Ribeiro
Jan 28 at 17:49
add a comment |
A physical sector size of 4096 means that the data on the drive is laid out in units of 4096 bytes, i.e. disk comprised of sequential "compartments" of 4096 bytes, that have to be written atomically. For compatibility reasons, most disks with 4096 byte sectors present themselves as having traditional 512 byte "logical sectors", which means the addressing unit is a 512 byte block.
The practical implication of this emulation of a 512 sector drive with an underlying disk with 4096 byte sectors is a potential performance issue. When writing a single 512 byte sector to a 512e disk, the drive must read the whole 4096 byte sector containing the 512-byte sector, modify the sector in RAM (on the disk controller) by replacing the 512-byte sector with the new contents, and finally write the whole 4096 sector back to the disk. Things get worse if you are reading or writing a couple of consecutive 512 sectors that happen to cross a 4096 sector boundary.
File systems usually lay out their data structures well, i.e. they are aligned to multiples of at least 4096 bytes, so the bigger sector size normally does not present a problem. This all breaks down, however, if the partition containing the file system itself is not aligned properly. In the case of a 512e disk, the partitions should be aligned so that the first 512-byte logical sector number is a multiple of eight.
A physical sector size of 4096 means that the data on the drive is laid out in units of 4096 bytes, i.e. disk comprised of sequential "compartments" of 4096 bytes, that have to be written atomically. For compatibility reasons, most disks with 4096 byte sectors present themselves as having traditional 512 byte "logical sectors", which means the addressing unit is a 512 byte block.
The practical implication of this emulation of a 512 sector drive with an underlying disk with 4096 byte sectors is a potential performance issue. When writing a single 512 byte sector to a 512e disk, the drive must read the whole 4096 byte sector containing the 512-byte sector, modify the sector in RAM (on the disk controller) by replacing the 512-byte sector with the new contents, and finally write the whole 4096 sector back to the disk. Things get worse if you are reading or writing a couple of consecutive 512 sectors that happen to cross a 4096 sector boundary.
File systems usually lay out their data structures well, i.e. they are aligned to multiples of at least 4096 bytes, so the bigger sector size normally does not present a problem. This all breaks down, however, if the partition containing the file system itself is not aligned properly. In the case of a 512e disk, the partitions should be aligned so that the first 512-byte logical sector number is a multiple of eight.
answered Jan 28 at 7:15
Johan MyréenJohan Myréen
7,68911524
7,68911524
Good thinking. This is a known issue, specially when aligning VMs because all the improvement gains add up and benefit the infra-structure as a whole. +1
– Rui F Ribeiro
Jan 28 at 17:49
add a comment |
Good thinking. This is a known issue, specially when aligning VMs because all the improvement gains add up and benefit the infra-structure as a whole. +1
– Rui F Ribeiro
Jan 28 at 17:49
Good thinking. This is a known issue, specially when aligning VMs because all the improvement gains add up and benefit the infra-structure as a whole. +1
– Rui F Ribeiro
Jan 28 at 17:49
Good thinking. This is a known issue, specially when aligning VMs because all the improvement gains add up and benefit the infra-structure as a whole. +1
– Rui F Ribeiro
Jan 28 at 17:49
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f497063%2fpartitioning-and-formatting-a-4k-emulated-512e-hdd%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
The only weird thing here is the bogus "optimal io size". Nothing wrong with start sector 2048 or 4096 (MiB alignment). As to why Windows doesn't like it, who knows?
– frostschutz
Jan 27 at 19:37
2
There is no need for the partition to start on sector 4096. The only requirement for optimal alignment is that the starting sector number is divisible by eight. The physical sectors are aligned to logical sectors N*8. 4096/512 = 8.
– Johan Myréen
Jan 27 at 20:34
I fail to understand why the disk not working in Windows is a concern here, do not understand why you are finding it strange it starting at sector 2048 (it is pretty normal), and I think you are mixing up logical and physical sectors. However, you true problem is that you external adapter sees the disk in a different way than pluggin it in your Windows machine. There is another question that talks about that, but there are people here that are able to explain it better.
– Rui F Ribeiro
Jan 27 at 20:34