Max transfer size in SG_IO/SG_ATA_16
Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
I need to pass a direct, raw ATA request to a hard drive (0x25, READ DMA EXT), to disobey max sector count (long story), and to bypass all possible OS caches, buffers, reorderings et al.
HDIO_DRIVE_TASKFILE IOCTL is no longer available due to libata.
I accomplished the goal with a SG_IO IOCTL with ATA pass-through (SG_ATA_16). Works perfectly except one problem: I can read a maximum of 8192 sectors in one command. I need to read a full of 32767 sectors.
max_hw_sectors_kb is 32767, so the drive supports this much transfer
max_sectors_kb was low, yet I brought it up to 32767 sectors, to no avail
scheduler is set to noop, no change.- Tried gather buffer (
iovec_count>0, properly setiovecsto consecutive buffer slices), no change.
Environment: Ubuntu 16.04/16.10/17.04/17.10 with standard kernels, SATA drive connected to standard AHCI interface on Intel chipset.
No matter what I do, starting with 8193 sectors, IOCTL bails out with "Invalid argument" error (precisely, with EINVAL errno). On another machine, it varies slightly from run to run (no reboot), usually settling at max 1344 sectors.
Where to look? What can cause a data transfer cap?
hard-disk read scsi sata ioctl
asked Dec 6 '17 at 15:47
Pawel Kraszewski
62
add a comment |Â
up vote
1
down vote
favorite
I need to pass a direct, raw ATA request to a hard drive (0x25, READ DMA EXT), to disobey max sector count (long story), and to bypass all possible OS caches, buffers, reorderings et al.
HDIO_DRIVE_TASKFILE IOCTL is no longer available due to libata.
I accomplished the goal with a SG_IO IOCTL with ATA pass-through (SG_ATA_16). Works perfectly except one problem: I can read a maximum of 8192 sectors in one command. I need to read a full of 32767 sectors.
max_hw_sectors_kb is 32767, so the drive supports this much transfer
max_sectors_kb was low, yet I brought it up to 32767 sectors, to no avail
scheduler is set to noop, no change.- Tried gather buffer (
iovec_count>0, properly setiovecsto consecutive buffer slices), no change.
Environment: Ubuntu 16.04/16.10/17.04/17.10 with standard kernels, SATA drive connected to standard AHCI interface on Intel chipset.
No matter what I do, starting with 8193 sectors, IOCTL bails out with "Invalid argument" error (precisely, with EINVAL errno). On another machine, it varies slightly from run to run (no reboot), usually settling at max 1344 sectors.
Where to look? What can cause a data transfer cap?
hard-disk read scsi sata ioctl
asked Dec 6 '17 at 15:47
Pawel Kraszewski
62
Now I'm very curious: Why do you need to read 32767 sectors at once?
– dirkt
Dec 6 '17 at 16:51
Can't go into much detail - device on the other end will not be a HDD, it just responds to enough ATA commands to be recognized as HDD. The main operation isREAD DMA EXT, which delivers a block of data I need, unique for specific starting LBA. I can't allow missed, cached or out-of-order read.
– Pawel Kraszewski
Dec 6 '17 at 18:08
1
Well, I gave up Linux. FreeBSD withMAXPHYSkernel option bumped-up, merrily conveys 32MB of data at once, using CAM subsystem (code based loosely onscsicmd()function ofcamcontrol(8)utility).
– Pawel Kraszewski
Dec 7 '17 at 15:54
add a comment |Â
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I need to pass a direct, raw ATA request to a hard drive (0x25, READ DMA EXT), to disobey max sector count (long story), and to bypass all possible OS caches, buffers, reorderings et al.
HDIO_DRIVE_TASKFILE IOCTL is no longer available due to libata.
I accomplished the goal with a SG_IO IOCTL with ATA pass-through (SG_ATA_16). Works perfectly except one problem: I can read a maximum of 8192 sectors in one command. I need to read a full of 32767 sectors.
max_hw_sectors_kb is 32767, so the drive supports this much transfer
max_sectors_kb was low, yet I brought it up to 32767 sectors, to no avail
scheduler is set to noop, no change.- Tried gather buffer (
iovec_count>0, properly setiovecsto consecutive buffer slices), no change.
Environment: Ubuntu 16.04/16.10/17.04/17.10 with standard kernels, SATA drive connected to standard AHCI interface on Intel chipset.
No matter what I do, starting with 8193 sectors, IOCTL bails out with "Invalid argument" error (precisely, with EINVAL errno). On another machine, it varies slightly from run to run (no reboot), usually settling at max 1344 sectors.
Where to look? What can cause a data transfer cap?
hard-disk read scsi sata ioctl
asked Dec 6 '17 at 15:47
Pawel Kraszewski
62
I need to pass a direct, raw ATA request to a hard drive (0x25, READ DMA EXT), to disobey max sector count (long story), and to bypass all possible OS caches, buffers, reorderings et al.
HDIO_DRIVE_TASKFILE IOCTL is no longer available due to libata.
I accomplished the goal with a SG_IO IOCTL with ATA pass-through (SG_ATA_16). Works perfectly except one problem: I can read a maximum of 8192 sectors in one command. I need to read a full of 32767 sectors.
max_hw_sectors_kb is 32767, so the drive supports this much transfer
max_sectors_kb was low, yet I brought it up to 32767 sectors, to no avail
scheduler is set to noop, no change.- Tried gather buffer (
iovec_count>0, properly setiovecsto consecutive buffer slices), no change.
Environment: Ubuntu 16.04/16.10/17.04/17.10 with standard kernels, SATA drive connected to standard AHCI interface on Intel chipset.
No matter what I do, starting with 8193 sectors, IOCTL bails out with "Invalid argument" error (precisely, with EINVAL errno). On another machine, it varies slightly from run to run (no reboot), usually settling at max 1344 sectors.
Where to look? What can cause a data transfer cap?
hard-disk read scsi sata ioctl
asked Dec 6 '17 at 15:47
Pawel Kraszewski
62
asked Dec 6 '17 at 15:47
Pawel Kraszewski
62
asked Dec 6 '17 at 15:47
Pawel Kraszewski
62
asked Dec 6 '17 at 15:47
Pawel Kraszewski
62
62
Now I'm very curious: Why do you need to read 32767 sectors at once?
– dirkt
Dec 6 '17 at 16:51
Can't go into much detail - device on the other end will not be a HDD, it just responds to enough ATA commands to be recognized as HDD. The main operation isREAD DMA EXT, which delivers a block of data I need, unique for specific starting LBA. I can't allow missed, cached or out-of-order read.
– Pawel Kraszewski
Dec 6 '17 at 18:08
1
Well, I gave up Linux. FreeBSD withMAXPHYSkernel option bumped-up, merrily conveys 32MB of data at once, using CAM subsystem (code based loosely onscsicmd()function ofcamcontrol(8)utility).
– Pawel Kraszewski
Dec 7 '17 at 15:54
add a comment |Â
Now I'm very curious: Why do you need to read 32767 sectors at once?
– dirkt
Dec 6 '17 at 16:51
Can't go into much detail - device on the other end will not be a HDD, it just responds to enough ATA commands to be recognized as HDD. The main operation isREAD DMA EXT, which delivers a block of data I need, unique for specific starting LBA. I can't allow missed, cached or out-of-order read.
– Pawel Kraszewski
Dec 6 '17 at 18:08
1
Well, I gave up Linux. FreeBSD withMAXPHYSkernel option bumped-up, merrily conveys 32MB of data at once, using CAM subsystem (code based loosely onscsicmd()function ofcamcontrol(8)utility).
– Pawel Kraszewski
Dec 7 '17 at 15:54
Now I'm very curious: Why do you need to read 32767 sectors at once?
– dirkt
Dec 6 '17 at 16:51
Now I'm very curious: Why do you need to read 32767 sectors at once?
– dirkt
Dec 6 '17 at 16:51
Can't go into much detail - device on the other end will not be a HDD, it just responds to enough ATA commands to be recognized as HDD. The main operation is
READ DMA EXT, which delivers a block of data I need, unique for specific starting LBA. I can't allow missed, cached or out-of-order read.– Pawel Kraszewski
Dec 6 '17 at 18:08
Can't go into much detail - device on the other end will not be a HDD, it just responds to enough ATA commands to be recognized as HDD. The main operation is
READ DMA EXT, which delivers a block of data I need, unique for specific starting LBA. I can't allow missed, cached or out-of-order read.– Pawel Kraszewski
Dec 6 '17 at 18:08
1
1
Well, I gave up Linux. FreeBSD with
MAXPHYS kernel option bumped-up, merrily conveys 32MB of data at once, using CAM subsystem (code based loosely on scsicmd() function of camcontrol(8) utility).– Pawel Kraszewski
Dec 7 '17 at 15:54
Well, I gave up Linux. FreeBSD with
MAXPHYS kernel option bumped-up, merrily conveys 32MB of data at once, using CAM subsystem (code based loosely on scsicmd() function of camcontrol(8) utility).– Pawel Kraszewski
Dec 7 '17 at 15:54
add a comment |Â
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f409244%2fmax-transfer-size-in-sg-io-sg-ata-16%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Now I'm very curious: Why do you need to read 32767 sectors at once?
– dirkt
Dec 6 '17 at 16:51
Can't go into much detail - device on the other end will not be a HDD, it just responds to enough ATA commands to be recognized as HDD. The main operation is
READ DMA EXT, which delivers a block of data I need, unique for specific starting LBA. I can't allow missed, cached or out-of-order read.– Pawel Kraszewski
Dec 6 '17 at 18:08
1
Well, I gave up Linux. FreeBSD with
MAXPHYSkernel option bumped-up, merrily conveys 32MB of data at once, using CAM subsystem (code based loosely onscsicmd()function ofcamcontrol(8)utility).– Pawel Kraszewski
Dec 7 '17 at 15:54