Partitioning and formatting a 4k-emulated (512e) HDD

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP












3















I just plugged into USB A 3.0 / C 3.1 my new external HDD to Debian Buster system.



The disk was sold as LaCie 2.5" Porsche Design P'9227 2TB USB-C.



Here is the output of fdisk -l /dev/sdc:



Disk /dev/sdc: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: P9227 Slim
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes


I just read some articles about 4k-emulated drives (512e), this one should be the case.



I am confused as to how to format it with NTFS.



I tried to use my brain, and here is what I came with:




  • Start sector of the partition should probably start on 4096 sector (?)



    So I created a partition with gdisk like this:



    Device Start End Sectors Size Type
    /dev/sdc1 4096 3907029134 3907025039 1.8T Microsoft basic data



  • Sector size should probably be forced with the --sector-size option like I did (?) issuing:



    mkfs.ntfs --no-indexing --verbose --with-uuid --label EXTERNAL_2TB --quick --sector-size 4096 /dev/sdc1



EDIT1:



Windows 10 fully updated did not recognize the partition and asked me to format, I used my favorite tool for that, and back to Linux here is the output of fdisk -l /dev/sdc:



Device Start End Sectors Size Type
/dev/sdc1 2048 3907028991 3907026944 1,8T Microsoft basic data


So why it must start at sector 2048, I don't understand.




EDIT2:



I don't understand what I am doing wrong in terms of compatibility with Windows. Every time I re-partition it / re-format it and boot Windows and plug the drive in, it just offers me to Format it itself.



I am quite positive I tried everything from inside gdisk + mkfs.ntfs.



I would like to know why I am unable to do the same as Windows does from my Linux CLI.




I will answer all questions tomorrow morning as well as comments.



I am now running:



pv --progress --timer --eta --rate --average-rate --bytes -s 1953314876k < /dev/zero > /media/vlastimil/LACIE_2TB/zero


with an expected speed of 123 MiB/s.










share|improve this question



















  • 1





    The only weird thing here is the bogus "optimal io size". Nothing wrong with start sector 2048 or 4096 (MiB alignment). As to why Windows doesn't like it, who knows?

    – frostschutz
    Jan 27 at 19:37







  • 2





    There is no need for the partition to start on sector 4096. The only requirement for optimal alignment is that the starting sector number is divisible by eight. The physical sectors are aligned to logical sectors N*8. 4096/512 = 8.

    – Johan Myréen
    Jan 27 at 20:34











  • I fail to understand why the disk not working in Windows is a concern here, do not understand why you are finding it strange it starting at sector 2048 (it is pretty normal), and I think you are mixing up logical and physical sectors. However, you true problem is that you external adapter sees the disk in a different way than pluggin it in your Windows machine. There is another question that talks about that, but there are people here that are able to explain it better.

    – Rui F Ribeiro
    Jan 27 at 20:34
















3















I just plugged into USB A 3.0 / C 3.1 my new external HDD to Debian Buster system.



The disk was sold as LaCie 2.5" Porsche Design P'9227 2TB USB-C.



Here is the output of fdisk -l /dev/sdc:



Disk /dev/sdc: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: P9227 Slim
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes


I just read some articles about 4k-emulated drives (512e), this one should be the case.



I am confused as to how to format it with NTFS.



I tried to use my brain, and here is what I came with:




  • Start sector of the partition should probably start on 4096 sector (?)



    So I created a partition with gdisk like this:



    Device Start End Sectors Size Type
    /dev/sdc1 4096 3907029134 3907025039 1.8T Microsoft basic data



  • Sector size should probably be forced with the --sector-size option like I did (?) issuing:



    mkfs.ntfs --no-indexing --verbose --with-uuid --label EXTERNAL_2TB --quick --sector-size 4096 /dev/sdc1



EDIT1:



Windows 10 fully updated did not recognize the partition and asked me to format, I used my favorite tool for that, and back to Linux here is the output of fdisk -l /dev/sdc:



Device Start End Sectors Size Type
/dev/sdc1 2048 3907028991 3907026944 1,8T Microsoft basic data


So why it must start at sector 2048, I don't understand.




EDIT2:



I don't understand what I am doing wrong in terms of compatibility with Windows. Every time I re-partition it / re-format it and boot Windows and plug the drive in, it just offers me to Format it itself.



I am quite positive I tried everything from inside gdisk + mkfs.ntfs.



I would like to know why I am unable to do the same as Windows does from my Linux CLI.




I will answer all questions tomorrow morning as well as comments.



I am now running:



pv --progress --timer --eta --rate --average-rate --bytes -s 1953314876k < /dev/zero > /media/vlastimil/LACIE_2TB/zero


with an expected speed of 123 MiB/s.










share|improve this question



















  • 1





    The only weird thing here is the bogus "optimal io size". Nothing wrong with start sector 2048 or 4096 (MiB alignment). As to why Windows doesn't like it, who knows?

    – frostschutz
    Jan 27 at 19:37







  • 2





    There is no need for the partition to start on sector 4096. The only requirement for optimal alignment is that the starting sector number is divisible by eight. The physical sectors are aligned to logical sectors N*8. 4096/512 = 8.

    – Johan Myréen
    Jan 27 at 20:34











  • I fail to understand why the disk not working in Windows is a concern here, do not understand why you are finding it strange it starting at sector 2048 (it is pretty normal), and I think you are mixing up logical and physical sectors. However, you true problem is that you external adapter sees the disk in a different way than pluggin it in your Windows machine. There is another question that talks about that, but there are people here that are able to explain it better.

    – Rui F Ribeiro
    Jan 27 at 20:34














3












3








3


1






I just plugged into USB A 3.0 / C 3.1 my new external HDD to Debian Buster system.



The disk was sold as LaCie 2.5" Porsche Design P'9227 2TB USB-C.



Here is the output of fdisk -l /dev/sdc:



Disk /dev/sdc: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: P9227 Slim
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes


I just read some articles about 4k-emulated drives (512e), this one should be the case.



I am confused as to how to format it with NTFS.



I tried to use my brain, and here is what I came with:




  • Start sector of the partition should probably start on 4096 sector (?)



    So I created a partition with gdisk like this:



    Device Start End Sectors Size Type
    /dev/sdc1 4096 3907029134 3907025039 1.8T Microsoft basic data



  • Sector size should probably be forced with the --sector-size option like I did (?) issuing:



    mkfs.ntfs --no-indexing --verbose --with-uuid --label EXTERNAL_2TB --quick --sector-size 4096 /dev/sdc1



EDIT1:



Windows 10 fully updated did not recognize the partition and asked me to format, I used my favorite tool for that, and back to Linux here is the output of fdisk -l /dev/sdc:



Device Start End Sectors Size Type
/dev/sdc1 2048 3907028991 3907026944 1,8T Microsoft basic data


So why it must start at sector 2048, I don't understand.




EDIT2:



I don't understand what I am doing wrong in terms of compatibility with Windows. Every time I re-partition it / re-format it and boot Windows and plug the drive in, it just offers me to Format it itself.



I am quite positive I tried everything from inside gdisk + mkfs.ntfs.



I would like to know why I am unable to do the same as Windows does from my Linux CLI.




I will answer all questions tomorrow morning as well as comments.



I am now running:



pv --progress --timer --eta --rate --average-rate --bytes -s 1953314876k < /dev/zero > /media/vlastimil/LACIE_2TB/zero


with an expected speed of 123 MiB/s.










share|improve this question
















I just plugged into USB A 3.0 / C 3.1 my new external HDD to Debian Buster system.



The disk was sold as LaCie 2.5" Porsche Design P'9227 2TB USB-C.



Here is the output of fdisk -l /dev/sdc:



Disk /dev/sdc: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: P9227 Slim
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes


I just read some articles about 4k-emulated drives (512e), this one should be the case.



I am confused as to how to format it with NTFS.



I tried to use my brain, and here is what I came with:




  • Start sector of the partition should probably start on 4096 sector (?)



    So I created a partition with gdisk like this:



    Device Start End Sectors Size Type
    /dev/sdc1 4096 3907029134 3907025039 1.8T Microsoft basic data



  • Sector size should probably be forced with the --sector-size option like I did (?) issuing:



    mkfs.ntfs --no-indexing --verbose --with-uuid --label EXTERNAL_2TB --quick --sector-size 4096 /dev/sdc1



EDIT1:



Windows 10 fully updated did not recognize the partition and asked me to format, I used my favorite tool for that, and back to Linux here is the output of fdisk -l /dev/sdc:



Device Start End Sectors Size Type
/dev/sdc1 2048 3907028991 3907026944 1,8T Microsoft basic data


So why it must start at sector 2048, I don't understand.




EDIT2:



I don't understand what I am doing wrong in terms of compatibility with Windows. Every time I re-partition it / re-format it and boot Windows and plug the drive in, it just offers me to Format it itself.



I am quite positive I tried everything from inside gdisk + mkfs.ntfs.



I would like to know why I am unable to do the same as Windows does from my Linux CLI.




I will answer all questions tomorrow morning as well as comments.



I am now running:



pv --progress --timer --eta --rate --average-rate --bytes -s 1953314876k < /dev/zero > /media/vlastimil/LACIE_2TB/zero


with an expected speed of 123 MiB/s.







partition hard-disk ntfs mkfs gdisk






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Jan 27 at 20:34







Vlastimil

















asked Jan 27 at 18:36









VlastimilVlastimil

8,0721464138




8,0721464138







  • 1





    The only weird thing here is the bogus "optimal io size". Nothing wrong with start sector 2048 or 4096 (MiB alignment). As to why Windows doesn't like it, who knows?

    – frostschutz
    Jan 27 at 19:37







  • 2





    There is no need for the partition to start on sector 4096. The only requirement for optimal alignment is that the starting sector number is divisible by eight. The physical sectors are aligned to logical sectors N*8. 4096/512 = 8.

    – Johan Myréen
    Jan 27 at 20:34











  • I fail to understand why the disk not working in Windows is a concern here, do not understand why you are finding it strange it starting at sector 2048 (it is pretty normal), and I think you are mixing up logical and physical sectors. However, you true problem is that you external adapter sees the disk in a different way than pluggin it in your Windows machine. There is another question that talks about that, but there are people here that are able to explain it better.

    – Rui F Ribeiro
    Jan 27 at 20:34













  • 1





    The only weird thing here is the bogus "optimal io size". Nothing wrong with start sector 2048 or 4096 (MiB alignment). As to why Windows doesn't like it, who knows?

    – frostschutz
    Jan 27 at 19:37







  • 2





    There is no need for the partition to start on sector 4096. The only requirement for optimal alignment is that the starting sector number is divisible by eight. The physical sectors are aligned to logical sectors N*8. 4096/512 = 8.

    – Johan Myréen
    Jan 27 at 20:34











  • I fail to understand why the disk not working in Windows is a concern here, do not understand why you are finding it strange it starting at sector 2048 (it is pretty normal), and I think you are mixing up logical and physical sectors. However, you true problem is that you external adapter sees the disk in a different way than pluggin it in your Windows machine. There is another question that talks about that, but there are people here that are able to explain it better.

    – Rui F Ribeiro
    Jan 27 at 20:34








1




1





The only weird thing here is the bogus "optimal io size". Nothing wrong with start sector 2048 or 4096 (MiB alignment). As to why Windows doesn't like it, who knows?

– frostschutz
Jan 27 at 19:37






The only weird thing here is the bogus "optimal io size". Nothing wrong with start sector 2048 or 4096 (MiB alignment). As to why Windows doesn't like it, who knows?

– frostschutz
Jan 27 at 19:37





2




2





There is no need for the partition to start on sector 4096. The only requirement for optimal alignment is that the starting sector number is divisible by eight. The physical sectors are aligned to logical sectors N*8. 4096/512 = 8.

– Johan Myréen
Jan 27 at 20:34





There is no need for the partition to start on sector 4096. The only requirement for optimal alignment is that the starting sector number is divisible by eight. The physical sectors are aligned to logical sectors N*8. 4096/512 = 8.

– Johan Myréen
Jan 27 at 20:34













I fail to understand why the disk not working in Windows is a concern here, do not understand why you are finding it strange it starting at sector 2048 (it is pretty normal), and I think you are mixing up logical and physical sectors. However, you true problem is that you external adapter sees the disk in a different way than pluggin it in your Windows machine. There is another question that talks about that, but there are people here that are able to explain it better.

– Rui F Ribeiro
Jan 27 at 20:34






I fail to understand why the disk not working in Windows is a concern here, do not understand why you are finding it strange it starting at sector 2048 (it is pretty normal), and I think you are mixing up logical and physical sectors. However, you true problem is that you external adapter sees the disk in a different way than pluggin it in your Windows machine. There is another question that talks about that, but there are people here that are able to explain it better.

– Rui F Ribeiro
Jan 27 at 20:34











1 Answer
1






active

oldest

votes


















3














A physical sector size of 4096 means that the data on the drive is laid out in units of 4096 bytes, i.e. disk comprised of sequential "compartments" of 4096 bytes, that have to be written atomically. For compatibility reasons, most disks with 4096 byte sectors present themselves as having traditional 512 byte "logical sectors", which means the addressing unit is a 512 byte block.



The practical implication of this emulation of a 512 sector drive with an underlying disk with 4096 byte sectors is a potential performance issue. When writing a single 512 byte sector to a 512e disk, the drive must read the whole 4096 byte sector containing the 512-byte sector, modify the sector in RAM (on the disk controller) by replacing the 512-byte sector with the new contents, and finally write the whole 4096 sector back to the disk. Things get worse if you are reading or writing a couple of consecutive 512 sectors that happen to cross a 4096 sector boundary.



File systems usually lay out their data structures well, i.e. they are aligned to multiples of at least 4096 bytes, so the bigger sector size normally does not present a problem. This all breaks down, however, if the partition containing the file system itself is not aligned properly. In the case of a 512e disk, the partitions should be aligned so that the first 512-byte logical sector number is a multiple of eight.






share|improve this answer























  • Good thinking. This is a known issue, specially when aligning VMs because all the improvement gains add up and benefit the infra-structure as a whole. +1

    – Rui F Ribeiro
    Jan 28 at 17:49











Your Answer








StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f497063%2fpartitioning-and-formatting-a-4k-emulated-512e-hdd%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









3














A physical sector size of 4096 means that the data on the drive is laid out in units of 4096 bytes, i.e. disk comprised of sequential "compartments" of 4096 bytes, that have to be written atomically. For compatibility reasons, most disks with 4096 byte sectors present themselves as having traditional 512 byte "logical sectors", which means the addressing unit is a 512 byte block.



The practical implication of this emulation of a 512 sector drive with an underlying disk with 4096 byte sectors is a potential performance issue. When writing a single 512 byte sector to a 512e disk, the drive must read the whole 4096 byte sector containing the 512-byte sector, modify the sector in RAM (on the disk controller) by replacing the 512-byte sector with the new contents, and finally write the whole 4096 sector back to the disk. Things get worse if you are reading or writing a couple of consecutive 512 sectors that happen to cross a 4096 sector boundary.



File systems usually lay out their data structures well, i.e. they are aligned to multiples of at least 4096 bytes, so the bigger sector size normally does not present a problem. This all breaks down, however, if the partition containing the file system itself is not aligned properly. In the case of a 512e disk, the partitions should be aligned so that the first 512-byte logical sector number is a multiple of eight.






share|improve this answer























  • Good thinking. This is a known issue, specially when aligning VMs because all the improvement gains add up and benefit the infra-structure as a whole. +1

    – Rui F Ribeiro
    Jan 28 at 17:49
















3














A physical sector size of 4096 means that the data on the drive is laid out in units of 4096 bytes, i.e. disk comprised of sequential "compartments" of 4096 bytes, that have to be written atomically. For compatibility reasons, most disks with 4096 byte sectors present themselves as having traditional 512 byte "logical sectors", which means the addressing unit is a 512 byte block.



The practical implication of this emulation of a 512 sector drive with an underlying disk with 4096 byte sectors is a potential performance issue. When writing a single 512 byte sector to a 512e disk, the drive must read the whole 4096 byte sector containing the 512-byte sector, modify the sector in RAM (on the disk controller) by replacing the 512-byte sector with the new contents, and finally write the whole 4096 sector back to the disk. Things get worse if you are reading or writing a couple of consecutive 512 sectors that happen to cross a 4096 sector boundary.



File systems usually lay out their data structures well, i.e. they are aligned to multiples of at least 4096 bytes, so the bigger sector size normally does not present a problem. This all breaks down, however, if the partition containing the file system itself is not aligned properly. In the case of a 512e disk, the partitions should be aligned so that the first 512-byte logical sector number is a multiple of eight.






share|improve this answer























  • Good thinking. This is a known issue, specially when aligning VMs because all the improvement gains add up and benefit the infra-structure as a whole. +1

    – Rui F Ribeiro
    Jan 28 at 17:49














3












3








3







A physical sector size of 4096 means that the data on the drive is laid out in units of 4096 bytes, i.e. disk comprised of sequential "compartments" of 4096 bytes, that have to be written atomically. For compatibility reasons, most disks with 4096 byte sectors present themselves as having traditional 512 byte "logical sectors", which means the addressing unit is a 512 byte block.



The practical implication of this emulation of a 512 sector drive with an underlying disk with 4096 byte sectors is a potential performance issue. When writing a single 512 byte sector to a 512e disk, the drive must read the whole 4096 byte sector containing the 512-byte sector, modify the sector in RAM (on the disk controller) by replacing the 512-byte sector with the new contents, and finally write the whole 4096 sector back to the disk. Things get worse if you are reading or writing a couple of consecutive 512 sectors that happen to cross a 4096 sector boundary.



File systems usually lay out their data structures well, i.e. they are aligned to multiples of at least 4096 bytes, so the bigger sector size normally does not present a problem. This all breaks down, however, if the partition containing the file system itself is not aligned properly. In the case of a 512e disk, the partitions should be aligned so that the first 512-byte logical sector number is a multiple of eight.






share|improve this answer













A physical sector size of 4096 means that the data on the drive is laid out in units of 4096 bytes, i.e. disk comprised of sequential "compartments" of 4096 bytes, that have to be written atomically. For compatibility reasons, most disks with 4096 byte sectors present themselves as having traditional 512 byte "logical sectors", which means the addressing unit is a 512 byte block.



The practical implication of this emulation of a 512 sector drive with an underlying disk with 4096 byte sectors is a potential performance issue. When writing a single 512 byte sector to a 512e disk, the drive must read the whole 4096 byte sector containing the 512-byte sector, modify the sector in RAM (on the disk controller) by replacing the 512-byte sector with the new contents, and finally write the whole 4096 sector back to the disk. Things get worse if you are reading or writing a couple of consecutive 512 sectors that happen to cross a 4096 sector boundary.



File systems usually lay out their data structures well, i.e. they are aligned to multiples of at least 4096 bytes, so the bigger sector size normally does not present a problem. This all breaks down, however, if the partition containing the file system itself is not aligned properly. In the case of a 512e disk, the partitions should be aligned so that the first 512-byte logical sector number is a multiple of eight.







share|improve this answer












share|improve this answer



share|improve this answer










answered Jan 28 at 7:15









Johan MyréenJohan Myréen

7,68911524




7,68911524












  • Good thinking. This is a known issue, specially when aligning VMs because all the improvement gains add up and benefit the infra-structure as a whole. +1

    – Rui F Ribeiro
    Jan 28 at 17:49


















  • Good thinking. This is a known issue, specially when aligning VMs because all the improvement gains add up and benefit the infra-structure as a whole. +1

    – Rui F Ribeiro
    Jan 28 at 17:49

















Good thinking. This is a known issue, specially when aligning VMs because all the improvement gains add up and benefit the infra-structure as a whole. +1

– Rui F Ribeiro
Jan 28 at 17:49






Good thinking. This is a known issue, specially when aligning VMs because all the improvement gains add up and benefit the infra-structure as a whole. +1

– Rui F Ribeiro
Jan 28 at 17:49


















draft saved

draft discarded
















































Thanks for contributing an answer to Unix & Linux Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f497063%2fpartitioning-and-formatting-a-4k-emulated-512e-hdd%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown






Popular posts from this blog

How to check contact read email or not when send email to Individual?

Bahrain

Postfix configuration issue with fips on centos 7; mailgun relay