Does the destruction of sensitive information limit the choice of hard drives to non-flash based devices?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP












27















Working with a non-profit organization,it's common to reuse hard drives that have previously stored highly sensitive information such as medical and financial records. This is primarily driven by cost-saving measures to reduce purchasing new hard drives.



If the destruction of sensitive information is the first requirement, does this limit the choice in selecting the type of storage medium?



For example, do non-flash based devices provide a higher level of assurance in the destruction of data using ATA Secure Erase and a single wipe in comparison to SSDs including self-encrypting drives?










share|improve this question



















  • 1





    SATA is just a standard for communication and interfaces. SATA drives can either be spinning rust hard disks or solid state drives. Also the ATA command set is not unique to SATA. It also works with SAS (a similar standard more common in enterprise environments).

    – forest
    Jan 11 at 6:43












  • @forest - That is correct. I have updated the question.

    – Motivated
    Jan 11 at 6:48











  • Confusing: Are you concerned with hard drives you are buying, or hard drives you are disposing of?

    – Harper
    Jan 11 at 22:03











  • @Harper - It's both. If there is the option to re-use, it's often the default choice. If there isn't the option to re-use e.g. damaged drives, unavailable drives, etc, the choice is often limited to the most cost-effective device which is generally non-flash devices.

    – Motivated
    Jan 12 at 20:01






  • 1





    $56M nonprofit, sorry, that is not poverty, that is a Board with f'd up priorities, in which case the real question is "how do I convince my Board to take security seriously?"

    – Harper
    Jan 13 at 19:41
















27















Working with a non-profit organization,it's common to reuse hard drives that have previously stored highly sensitive information such as medical and financial records. This is primarily driven by cost-saving measures to reduce purchasing new hard drives.



If the destruction of sensitive information is the first requirement, does this limit the choice in selecting the type of storage medium?



For example, do non-flash based devices provide a higher level of assurance in the destruction of data using ATA Secure Erase and a single wipe in comparison to SSDs including self-encrypting drives?










share|improve this question



















  • 1





    SATA is just a standard for communication and interfaces. SATA drives can either be spinning rust hard disks or solid state drives. Also the ATA command set is not unique to SATA. It also works with SAS (a similar standard more common in enterprise environments).

    – forest
    Jan 11 at 6:43












  • @forest - That is correct. I have updated the question.

    – Motivated
    Jan 11 at 6:48











  • Confusing: Are you concerned with hard drives you are buying, or hard drives you are disposing of?

    – Harper
    Jan 11 at 22:03











  • @Harper - It's both. If there is the option to re-use, it's often the default choice. If there isn't the option to re-use e.g. damaged drives, unavailable drives, etc, the choice is often limited to the most cost-effective device which is generally non-flash devices.

    – Motivated
    Jan 12 at 20:01






  • 1





    $56M nonprofit, sorry, that is not poverty, that is a Board with f'd up priorities, in which case the real question is "how do I convince my Board to take security seriously?"

    – Harper
    Jan 13 at 19:41














27












27








27


11






Working with a non-profit organization,it's common to reuse hard drives that have previously stored highly sensitive information such as medical and financial records. This is primarily driven by cost-saving measures to reduce purchasing new hard drives.



If the destruction of sensitive information is the first requirement, does this limit the choice in selecting the type of storage medium?



For example, do non-flash based devices provide a higher level of assurance in the destruction of data using ATA Secure Erase and a single wipe in comparison to SSDs including self-encrypting drives?










share|improve this question
















Working with a non-profit organization,it's common to reuse hard drives that have previously stored highly sensitive information such as medical and financial records. This is primarily driven by cost-saving measures to reduce purchasing new hard drives.



If the destruction of sensitive information is the first requirement, does this limit the choice in selecting the type of storage medium?



For example, do non-flash based devices provide a higher level of assurance in the destruction of data using ATA Secure Erase and a single wipe in comparison to SSDs including self-encrypting drives?







storage deletion sensitive-data-exposure ssd sata






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Jan 11 at 7:30







Motivated

















asked Jan 11 at 6:41









MotivatedMotivated

520412




520412







  • 1





    SATA is just a standard for communication and interfaces. SATA drives can either be spinning rust hard disks or solid state drives. Also the ATA command set is not unique to SATA. It also works with SAS (a similar standard more common in enterprise environments).

    – forest
    Jan 11 at 6:43












  • @forest - That is correct. I have updated the question.

    – Motivated
    Jan 11 at 6:48











  • Confusing: Are you concerned with hard drives you are buying, or hard drives you are disposing of?

    – Harper
    Jan 11 at 22:03











  • @Harper - It's both. If there is the option to re-use, it's often the default choice. If there isn't the option to re-use e.g. damaged drives, unavailable drives, etc, the choice is often limited to the most cost-effective device which is generally non-flash devices.

    – Motivated
    Jan 12 at 20:01






  • 1





    $56M nonprofit, sorry, that is not poverty, that is a Board with f'd up priorities, in which case the real question is "how do I convince my Board to take security seriously?"

    – Harper
    Jan 13 at 19:41













  • 1





    SATA is just a standard for communication and interfaces. SATA drives can either be spinning rust hard disks or solid state drives. Also the ATA command set is not unique to SATA. It also works with SAS (a similar standard more common in enterprise environments).

    – forest
    Jan 11 at 6:43












  • @forest - That is correct. I have updated the question.

    – Motivated
    Jan 11 at 6:48











  • Confusing: Are you concerned with hard drives you are buying, or hard drives you are disposing of?

    – Harper
    Jan 11 at 22:03











  • @Harper - It's both. If there is the option to re-use, it's often the default choice. If there isn't the option to re-use e.g. damaged drives, unavailable drives, etc, the choice is often limited to the most cost-effective device which is generally non-flash devices.

    – Motivated
    Jan 12 at 20:01






  • 1





    $56M nonprofit, sorry, that is not poverty, that is a Board with f'd up priorities, in which case the real question is "how do I convince my Board to take security seriously?"

    – Harper
    Jan 13 at 19:41








1




1





SATA is just a standard for communication and interfaces. SATA drives can either be spinning rust hard disks or solid state drives. Also the ATA command set is not unique to SATA. It also works with SAS (a similar standard more common in enterprise environments).

– forest
Jan 11 at 6:43






SATA is just a standard for communication and interfaces. SATA drives can either be spinning rust hard disks or solid state drives. Also the ATA command set is not unique to SATA. It also works with SAS (a similar standard more common in enterprise environments).

– forest
Jan 11 at 6:43














@forest - That is correct. I have updated the question.

– Motivated
Jan 11 at 6:48





@forest - That is correct. I have updated the question.

– Motivated
Jan 11 at 6:48













Confusing: Are you concerned with hard drives you are buying, or hard drives you are disposing of?

– Harper
Jan 11 at 22:03





Confusing: Are you concerned with hard drives you are buying, or hard drives you are disposing of?

– Harper
Jan 11 at 22:03













@Harper - It's both. If there is the option to re-use, it's often the default choice. If there isn't the option to re-use e.g. damaged drives, unavailable drives, etc, the choice is often limited to the most cost-effective device which is generally non-flash devices.

– Motivated
Jan 12 at 20:01





@Harper - It's both. If there is the option to re-use, it's often the default choice. If there isn't the option to re-use e.g. damaged drives, unavailable drives, etc, the choice is often limited to the most cost-effective device which is generally non-flash devices.

– Motivated
Jan 12 at 20:01




1




1





$56M nonprofit, sorry, that is not poverty, that is a Board with f'd up priorities, in which case the real question is "how do I convince my Board to take security seriously?"

– Harper
Jan 13 at 19:41






$56M nonprofit, sorry, that is not poverty, that is a Board with f'd up priorities, in which case the real question is "how do I convince my Board to take security seriously?"

– Harper
Jan 13 at 19:41











4 Answers
4






active

oldest

votes


















50














Data destruction is a technique of last resort. If you are planning to use a new storage device, you should use full disk encryption. This allows you to either destroy the encrypted master key or simply forget the password, effectively rendering all data unrecoverable, despite no data actually being wiped. Encryption is a solution for both solid state and standard hard drives. Use a strong algorithm like AES.



If you absolutely need to use a hard drive without full disk encryption, you should get one which supports SED, which is transparent hardware encryption. SED transparently encrypts all data written to the drive, but keeps the encryption key stored in a special area. When you initiate secure erasure, this key is all that is destroyed. This feature is supported on most modern SSDs and HDDs. If you do not know if a drive supports it, you can often conclude that it is supported if the estimated ATA Secure Erase time is showing as only two minutes, regardless of how large the drive itself is.



There is nothing intrinsic to the data storage methods used by solid state media that makes it hard to perform data destruction, but their firmware makes it impossible for the operating system to overwrite specific sectors due to wear leveling, a feature that spreads writes around the drive to decrease the wear and tear on individual flash cells (each of which has a finite lifespan). This does mean that you cannot overwrite data on SSDs reliably. You can still use SED if the drive implements it, and you can use ATA Security Erase as well, but if you need to manually overwrite a range of sectors, use an HDD.



Note that, if you do use an SSD and are using full disk encryption and you have TRIM enabled, the drive will leak a limited amount of metadata, as explained in this excellent blog post. You can usually disable TRIM at a small performance penalty, but you will avoid metadata leakage. Whether or not the exact metadata leaked is problematic depends on your specific threat model.






share|improve this answer

























  • Comments are not for extended discussion; this conversation has been moved to chat.

    – Rory Alsop
    Jan 12 at 19:22


















13














Tl;dr: Because you can never trust all storage drives to securely wipe themselves, you must plan as if none of your drives can be securely wiped.



Placing a dependency on the type of media is not the right way to approach the problem, because the technology is always evolving and changing, and you can never be in 100% control of all IT spend. Remember that disks were never designed for secrecy first - they are designed for the opposite: reliable access. (Some disk makers like to maximize profits by selling their products as “security solutions”, but that still doesn’t make them the best choice for the job.)



For example, Shadow IT (aka the boss’s kid) is good at buying consumer equipment like SSDs, and installing it in the department desktops without asking permission. Or a non-profit might have to accept a generous donation of a hundred drives from some corporate sponsor (for political or marketing reasons), but that don’t support Secure Erase. Decent corporate laptops don’t even offer spinny disks as an option anymore, while wear-leveling algorithms ensure that SSDs always risk leaking some data in the slack spaces of the drive.



Instead, look to something that is designed to solve this exact security problem, and is something that you can control enterprise-wide, such as installing encrypted file systems that can be wiped as quickly as deleting the key. For example, in a Windows shop enforcing BitLocker via Group Policy would protect all the drives, not just the special ones you ordered.






share|improve this answer

























  • It's not uncommon for donated devices to be provisioned with non-flash devices. If so and since wear leveling algorithms have a risk of data leakage, it seems that non-flash devices offer a higher level of assurance when employing secure destruction methods such as ATA secure erase and overwriting.

    – Motivated
    Jan 11 at 7:34











  • @Motivated , you’re confirming my point. An unspecific “level of assurance” is nowhere near the same as the permanent and total destruction of information that you would get by destroying the file system’s encryption key. Hoping a random storage device does not contain residual sensitive data is just a roll of the dice away from a breach. This requires a planned approach.

    – John Deters
    Jan 11 at 13:32






  • 1





    Overwriting is not a guaranteed form of data destruction. Here’s proof: not a single disk vendor offers a copy of DBAN with their drives and says “this is a PCI compliant disk wiping solution.” They will never offer it because they know bits of data can live on in bad blocks. Don’t try to rationalize disk wiping; don’t even consider it because it leads down a false path. Instead, look for a cryptographic solution that is purpose-designed for secure data destruction, and is independent of the storage technology. Then you can keep safely using your salvaged hard drives.

    – John Deters
    Jan 11 at 16:55






  • 2





    We set up storage arrays with encrypted drives. Sensitive systems and all laptops use OS level encrypted file systems. Applications with sensitive data use app-level encryption because encrypting sensitive data at rest is a security policy requirement. When a drive is formatted, the key is destroyed. When drives are removed from service, we track them by serial number and document the process to assure they are sent through the shredder. Different processes for different stages in the lifecycle. Notice that the drive technology doesn’t matter - the data is always secured. It’s all planned.

    – John Deters
    Jan 11 at 17:13






  • 1





    @8bittree oops, yes, I meant “secrecy”, not “security”. Thanks, I fixed it.

    – John Deters
    Jan 12 at 4:22


















1














Firstly, my understanding is that an SSD that properly implements the secure erase command will erase the unallocated blocks, although it may be unable to erase retired/failed blocks (these are blocks that have worn out and no longer operate correctly) and in theory these could contain recoverable data.



Secondly, HDDs also include reserved space. Most notably, this is used when a sector on the disk fails (a "bad sector") and the data must be relocated elsewhere. The original data is left behind in the bad sector which is no longer in use. Some disks also use additional reserved space as working room to rearrange other sectors so that the data from the bad sector can be physically located in a more optimal place and the disk may not erase the reserved space that was used either. In theory though, an HDD that properly implements the secure erase command will erase both bad sectors (if possible) and reserved space.



However, as other answers have pointed out, this is all relying on the proper implementation of the secure erase command and the ability for failing/failed parts of the media to be erased. The best solution may lie in something that is not dependent on the drive's own firmware supporting a particular operation.



With full disk encryption (or even file-level encryption, although be careful of filenames revealing information) you don't need to erase the actual data, just the encryption keys. In that case, the data is encrypted before it is written to disk and the disk (should) only ever contain the encrypted data. As the encryption keys are required to decrypt the encrypted data, having access to the encrypted data on the disk is useless without the encryption keys. As long as your encryption keys are securely erased or, even better, stored on a separate disk/memory device (e.g. a smart card or hardware key) the data is effectively unreadable.



Note that a lot of HDDs and SSDs are now offering "self-encryption". This is where the disk's own firmware generates an encryption key and stores it in the disk controller's internal memory, and encrypts the data itself before it is written to disk and decrypts it again after reading. The computer sees the disk as an unencrypted disk but the data actually stored on the media is encrypted. Such disks typically implement the secure erase command by deleting the encryption key from the controller's memory rather than overwriting the disk. Personally I avoid self-encrypting disks because they have a bad history of firmware bugs leading to either gaping security vulnerabilities or data loss, but the concept is the same as OS-level full disk encryption.






share|improve this answer























  • As the OP mentioned in the question, they are running on donated or reused drives. Not all those drives will offer self-encryption, nor is there assurance that any of them will have a correct and secure implementation of Secure Erase. He really shouldn’t trust random hardware to meet his security needs.

    – John Deters
    Jan 11 at 13:37











  • @Michael Johson - If secure erase and self-encrypting drives are unreliable in implementation and taking into consideration for wear leveling as suggested by John Deter, it seems that flash devices offer a lower level of assurance.

    – Motivated
    Jan 11 at 16:49











  • you don't need to erase the actual data, just the encryption keys. While practical, not secure. Erase both. You never know when a backdoor or zero-day is found for that encryption standard.

    – cde
    Jan 13 at 0:11


















0














Solid state disks are definitively to be preferred. Note that they are not without their troubles either, since sometimes implementors just suck (and Windows/Bitlocker sucks, too).



Traditional disk drives have been "encrypting" (or rather mixing) data weakly since pretty much forever to distribute bits better, but this doesn't help much in protecting data. More recently, there exist harddrives which are self-encrypting disks (SED), but as harddisks they are kinda "prestige" products and outrageous in price. I haven't so far owned one.



Solid state disks are practically always SED, but the feature set, and more importantly, the quality of the implementation differs a lot. As you can read in the linked article, for example, earlier models from Crucial used an encryption that was total bollocks. The user's password is compared to a hash by the firmware to "unlock" the drive's encryption key as opposed to e.g. Samsung's drives which use PBKDF2 to derive the key from the password. Which, in terms of actual versus misleading security is worlds in between.



Luckily, in any case, and regardless of bad implementations, the security-while-used is much more affected than the security-after-erased. Well... luckily, I don't know if that's a good wording, actually systems should always be secure. But at least it doesn't suck beyond.



There exist the notion of "master password" in the ATA standard, so any such thing as unlocking verus deriving an encryption key is -- even not considering that someone might find a way to read out the storage -- catastrophic. It basically means nothing is encrypted at all in a meaningful way.



Secure erase on a SED means erasing the disk encryption key, rendering the contents of the complete disk unreadable. So, unless one assumes a maliciously-built drive (which tells you it did a secure erase, but secretly still holds a copy of the key), this is secure even in presence of a broken implementation, and even in presence of someone cracking open the controller chip or such.



Secure erase on a traditional harddisk means the disk will overwrite every sector. I've recently done that with a pre-fail (SMART showing errors) Seagate Barracuda that was to be RMAed.

And guess what, secure erase is all nice and well, but a pre-fail disk will simply refuse to do the job. It'll start, whack around for a few minutes, and terminate with "error blah blah" after erasing approximately 10% of the disk. That wasn't an issue in my case since the data on the disk was from a RAID with software encryption on top, so any contents was basically useless anyway (wiping not really necessary). But, you get the idea. If you didn't use an encrypted filesystem, there's now no way to erase the data!



Generally, wear-levelling (both on traditional disks and SSDs) may make overwriting stuff much less possible than you are maybe inclined to believe.



Also, restoring overwritten data on a magnetic disk is possible. Yes, it is much, much harder than it was 15-20 years ago when data density was much lower (back then, it was pretty much a routine job). But it is still... generally possible.



So, if the data is truly super-sensitive (as in medical records), either one should layer software encryption on top, which eliminates the need to wipe the disk (though it doesn't hurt to do it anyway), or one should not donate the drives but use one of these to be sure.






share|improve this answer























  • There's the question whether you want to trust a "self encrypting" drive. Lots of encryption is created by clueless dolts.

    – gnasher729
    Jan 13 at 11:35











  • @gnasher729: Well the encryption per se isn't the problem since they all use AES-256. It's what is readily supported in hardware, cheap, and standard. The problem is when they do stuff like on the Crucial drives pointed out in the article where the DEK is stored on the device and "unlocked" by comparing password strings (rather than deriving a key from user input). That's about as trivial to crack as the typical if(trial == true) ...; code found in a lot of software which takes a 12 year old three minutes to patch. But even so... for disposing this doesn't matter. Key erased is erased.

    – Damon
    Jan 13 at 22:51











Your Answer








StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "162"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsecurity.stackexchange.com%2fquestions%2f201257%2fdoes-the-destruction-of-sensitive-information-limit-the-choice-of-hard-drives-to%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























4 Answers
4






active

oldest

votes








4 Answers
4






active

oldest

votes









active

oldest

votes






active

oldest

votes









50














Data destruction is a technique of last resort. If you are planning to use a new storage device, you should use full disk encryption. This allows you to either destroy the encrypted master key or simply forget the password, effectively rendering all data unrecoverable, despite no data actually being wiped. Encryption is a solution for both solid state and standard hard drives. Use a strong algorithm like AES.



If you absolutely need to use a hard drive without full disk encryption, you should get one which supports SED, which is transparent hardware encryption. SED transparently encrypts all data written to the drive, but keeps the encryption key stored in a special area. When you initiate secure erasure, this key is all that is destroyed. This feature is supported on most modern SSDs and HDDs. If you do not know if a drive supports it, you can often conclude that it is supported if the estimated ATA Secure Erase time is showing as only two minutes, regardless of how large the drive itself is.



There is nothing intrinsic to the data storage methods used by solid state media that makes it hard to perform data destruction, but their firmware makes it impossible for the operating system to overwrite specific sectors due to wear leveling, a feature that spreads writes around the drive to decrease the wear and tear on individual flash cells (each of which has a finite lifespan). This does mean that you cannot overwrite data on SSDs reliably. You can still use SED if the drive implements it, and you can use ATA Security Erase as well, but if you need to manually overwrite a range of sectors, use an HDD.



Note that, if you do use an SSD and are using full disk encryption and you have TRIM enabled, the drive will leak a limited amount of metadata, as explained in this excellent blog post. You can usually disable TRIM at a small performance penalty, but you will avoid metadata leakage. Whether or not the exact metadata leaked is problematic depends on your specific threat model.






share|improve this answer

























  • Comments are not for extended discussion; this conversation has been moved to chat.

    – Rory Alsop
    Jan 12 at 19:22















50














Data destruction is a technique of last resort. If you are planning to use a new storage device, you should use full disk encryption. This allows you to either destroy the encrypted master key or simply forget the password, effectively rendering all data unrecoverable, despite no data actually being wiped. Encryption is a solution for both solid state and standard hard drives. Use a strong algorithm like AES.



If you absolutely need to use a hard drive without full disk encryption, you should get one which supports SED, which is transparent hardware encryption. SED transparently encrypts all data written to the drive, but keeps the encryption key stored in a special area. When you initiate secure erasure, this key is all that is destroyed. This feature is supported on most modern SSDs and HDDs. If you do not know if a drive supports it, you can often conclude that it is supported if the estimated ATA Secure Erase time is showing as only two minutes, regardless of how large the drive itself is.



There is nothing intrinsic to the data storage methods used by solid state media that makes it hard to perform data destruction, but their firmware makes it impossible for the operating system to overwrite specific sectors due to wear leveling, a feature that spreads writes around the drive to decrease the wear and tear on individual flash cells (each of which has a finite lifespan). This does mean that you cannot overwrite data on SSDs reliably. You can still use SED if the drive implements it, and you can use ATA Security Erase as well, but if you need to manually overwrite a range of sectors, use an HDD.



Note that, if you do use an SSD and are using full disk encryption and you have TRIM enabled, the drive will leak a limited amount of metadata, as explained in this excellent blog post. You can usually disable TRIM at a small performance penalty, but you will avoid metadata leakage. Whether or not the exact metadata leaked is problematic depends on your specific threat model.






share|improve this answer

























  • Comments are not for extended discussion; this conversation has been moved to chat.

    – Rory Alsop
    Jan 12 at 19:22













50












50








50







Data destruction is a technique of last resort. If you are planning to use a new storage device, you should use full disk encryption. This allows you to either destroy the encrypted master key or simply forget the password, effectively rendering all data unrecoverable, despite no data actually being wiped. Encryption is a solution for both solid state and standard hard drives. Use a strong algorithm like AES.



If you absolutely need to use a hard drive without full disk encryption, you should get one which supports SED, which is transparent hardware encryption. SED transparently encrypts all data written to the drive, but keeps the encryption key stored in a special area. When you initiate secure erasure, this key is all that is destroyed. This feature is supported on most modern SSDs and HDDs. If you do not know if a drive supports it, you can often conclude that it is supported if the estimated ATA Secure Erase time is showing as only two minutes, regardless of how large the drive itself is.



There is nothing intrinsic to the data storage methods used by solid state media that makes it hard to perform data destruction, but their firmware makes it impossible for the operating system to overwrite specific sectors due to wear leveling, a feature that spreads writes around the drive to decrease the wear and tear on individual flash cells (each of which has a finite lifespan). This does mean that you cannot overwrite data on SSDs reliably. You can still use SED if the drive implements it, and you can use ATA Security Erase as well, but if you need to manually overwrite a range of sectors, use an HDD.



Note that, if you do use an SSD and are using full disk encryption and you have TRIM enabled, the drive will leak a limited amount of metadata, as explained in this excellent blog post. You can usually disable TRIM at a small performance penalty, but you will avoid metadata leakage. Whether or not the exact metadata leaked is problematic depends on your specific threat model.






share|improve this answer















Data destruction is a technique of last resort. If you are planning to use a new storage device, you should use full disk encryption. This allows you to either destroy the encrypted master key or simply forget the password, effectively rendering all data unrecoverable, despite no data actually being wiped. Encryption is a solution for both solid state and standard hard drives. Use a strong algorithm like AES.



If you absolutely need to use a hard drive without full disk encryption, you should get one which supports SED, which is transparent hardware encryption. SED transparently encrypts all data written to the drive, but keeps the encryption key stored in a special area. When you initiate secure erasure, this key is all that is destroyed. This feature is supported on most modern SSDs and HDDs. If you do not know if a drive supports it, you can often conclude that it is supported if the estimated ATA Secure Erase time is showing as only two minutes, regardless of how large the drive itself is.



There is nothing intrinsic to the data storage methods used by solid state media that makes it hard to perform data destruction, but their firmware makes it impossible for the operating system to overwrite specific sectors due to wear leveling, a feature that spreads writes around the drive to decrease the wear and tear on individual flash cells (each of which has a finite lifespan). This does mean that you cannot overwrite data on SSDs reliably. You can still use SED if the drive implements it, and you can use ATA Security Erase as well, but if you need to manually overwrite a range of sectors, use an HDD.



Note that, if you do use an SSD and are using full disk encryption and you have TRIM enabled, the drive will leak a limited amount of metadata, as explained in this excellent blog post. You can usually disable TRIM at a small performance penalty, but you will avoid metadata leakage. Whether or not the exact metadata leaked is problematic depends on your specific threat model.







share|improve this answer














share|improve this answer



share|improve this answer








edited Jan 12 at 8:56

























answered Jan 11 at 6:56









forestforest

34.6k16114120




34.6k16114120












  • Comments are not for extended discussion; this conversation has been moved to chat.

    – Rory Alsop
    Jan 12 at 19:22

















  • Comments are not for extended discussion; this conversation has been moved to chat.

    – Rory Alsop
    Jan 12 at 19:22
















Comments are not for extended discussion; this conversation has been moved to chat.

– Rory Alsop
Jan 12 at 19:22





Comments are not for extended discussion; this conversation has been moved to chat.

– Rory Alsop
Jan 12 at 19:22













13














Tl;dr: Because you can never trust all storage drives to securely wipe themselves, you must plan as if none of your drives can be securely wiped.



Placing a dependency on the type of media is not the right way to approach the problem, because the technology is always evolving and changing, and you can never be in 100% control of all IT spend. Remember that disks were never designed for secrecy first - they are designed for the opposite: reliable access. (Some disk makers like to maximize profits by selling their products as “security solutions”, but that still doesn’t make them the best choice for the job.)



For example, Shadow IT (aka the boss’s kid) is good at buying consumer equipment like SSDs, and installing it in the department desktops without asking permission. Or a non-profit might have to accept a generous donation of a hundred drives from some corporate sponsor (for political or marketing reasons), but that don’t support Secure Erase. Decent corporate laptops don’t even offer spinny disks as an option anymore, while wear-leveling algorithms ensure that SSDs always risk leaking some data in the slack spaces of the drive.



Instead, look to something that is designed to solve this exact security problem, and is something that you can control enterprise-wide, such as installing encrypted file systems that can be wiped as quickly as deleting the key. For example, in a Windows shop enforcing BitLocker via Group Policy would protect all the drives, not just the special ones you ordered.






share|improve this answer

























  • It's not uncommon for donated devices to be provisioned with non-flash devices. If so and since wear leveling algorithms have a risk of data leakage, it seems that non-flash devices offer a higher level of assurance when employing secure destruction methods such as ATA secure erase and overwriting.

    – Motivated
    Jan 11 at 7:34











  • @Motivated , you’re confirming my point. An unspecific “level of assurance” is nowhere near the same as the permanent and total destruction of information that you would get by destroying the file system’s encryption key. Hoping a random storage device does not contain residual sensitive data is just a roll of the dice away from a breach. This requires a planned approach.

    – John Deters
    Jan 11 at 13:32






  • 1





    Overwriting is not a guaranteed form of data destruction. Here’s proof: not a single disk vendor offers a copy of DBAN with their drives and says “this is a PCI compliant disk wiping solution.” They will never offer it because they know bits of data can live on in bad blocks. Don’t try to rationalize disk wiping; don’t even consider it because it leads down a false path. Instead, look for a cryptographic solution that is purpose-designed for secure data destruction, and is independent of the storage technology. Then you can keep safely using your salvaged hard drives.

    – John Deters
    Jan 11 at 16:55






  • 2





    We set up storage arrays with encrypted drives. Sensitive systems and all laptops use OS level encrypted file systems. Applications with sensitive data use app-level encryption because encrypting sensitive data at rest is a security policy requirement. When a drive is formatted, the key is destroyed. When drives are removed from service, we track them by serial number and document the process to assure they are sent through the shredder. Different processes for different stages in the lifecycle. Notice that the drive technology doesn’t matter - the data is always secured. It’s all planned.

    – John Deters
    Jan 11 at 17:13






  • 1





    @8bittree oops, yes, I meant “secrecy”, not “security”. Thanks, I fixed it.

    – John Deters
    Jan 12 at 4:22















13














Tl;dr: Because you can never trust all storage drives to securely wipe themselves, you must plan as if none of your drives can be securely wiped.



Placing a dependency on the type of media is not the right way to approach the problem, because the technology is always evolving and changing, and you can never be in 100% control of all IT spend. Remember that disks were never designed for secrecy first - they are designed for the opposite: reliable access. (Some disk makers like to maximize profits by selling their products as “security solutions”, but that still doesn’t make them the best choice for the job.)



For example, Shadow IT (aka the boss’s kid) is good at buying consumer equipment like SSDs, and installing it in the department desktops without asking permission. Or a non-profit might have to accept a generous donation of a hundred drives from some corporate sponsor (for political or marketing reasons), but that don’t support Secure Erase. Decent corporate laptops don’t even offer spinny disks as an option anymore, while wear-leveling algorithms ensure that SSDs always risk leaking some data in the slack spaces of the drive.



Instead, look to something that is designed to solve this exact security problem, and is something that you can control enterprise-wide, such as installing encrypted file systems that can be wiped as quickly as deleting the key. For example, in a Windows shop enforcing BitLocker via Group Policy would protect all the drives, not just the special ones you ordered.






share|improve this answer

























  • It's not uncommon for donated devices to be provisioned with non-flash devices. If so and since wear leveling algorithms have a risk of data leakage, it seems that non-flash devices offer a higher level of assurance when employing secure destruction methods such as ATA secure erase and overwriting.

    – Motivated
    Jan 11 at 7:34











  • @Motivated , you’re confirming my point. An unspecific “level of assurance” is nowhere near the same as the permanent and total destruction of information that you would get by destroying the file system’s encryption key. Hoping a random storage device does not contain residual sensitive data is just a roll of the dice away from a breach. This requires a planned approach.

    – John Deters
    Jan 11 at 13:32






  • 1





    Overwriting is not a guaranteed form of data destruction. Here’s proof: not a single disk vendor offers a copy of DBAN with their drives and says “this is a PCI compliant disk wiping solution.” They will never offer it because they know bits of data can live on in bad blocks. Don’t try to rationalize disk wiping; don’t even consider it because it leads down a false path. Instead, look for a cryptographic solution that is purpose-designed for secure data destruction, and is independent of the storage technology. Then you can keep safely using your salvaged hard drives.

    – John Deters
    Jan 11 at 16:55






  • 2





    We set up storage arrays with encrypted drives. Sensitive systems and all laptops use OS level encrypted file systems. Applications with sensitive data use app-level encryption because encrypting sensitive data at rest is a security policy requirement. When a drive is formatted, the key is destroyed. When drives are removed from service, we track them by serial number and document the process to assure they are sent through the shredder. Different processes for different stages in the lifecycle. Notice that the drive technology doesn’t matter - the data is always secured. It’s all planned.

    – John Deters
    Jan 11 at 17:13






  • 1





    @8bittree oops, yes, I meant “secrecy”, not “security”. Thanks, I fixed it.

    – John Deters
    Jan 12 at 4:22













13












13








13







Tl;dr: Because you can never trust all storage drives to securely wipe themselves, you must plan as if none of your drives can be securely wiped.



Placing a dependency on the type of media is not the right way to approach the problem, because the technology is always evolving and changing, and you can never be in 100% control of all IT spend. Remember that disks were never designed for secrecy first - they are designed for the opposite: reliable access. (Some disk makers like to maximize profits by selling their products as “security solutions”, but that still doesn’t make them the best choice for the job.)



For example, Shadow IT (aka the boss’s kid) is good at buying consumer equipment like SSDs, and installing it in the department desktops without asking permission. Or a non-profit might have to accept a generous donation of a hundred drives from some corporate sponsor (for political or marketing reasons), but that don’t support Secure Erase. Decent corporate laptops don’t even offer spinny disks as an option anymore, while wear-leveling algorithms ensure that SSDs always risk leaking some data in the slack spaces of the drive.



Instead, look to something that is designed to solve this exact security problem, and is something that you can control enterprise-wide, such as installing encrypted file systems that can be wiped as quickly as deleting the key. For example, in a Windows shop enforcing BitLocker via Group Policy would protect all the drives, not just the special ones you ordered.






share|improve this answer















Tl;dr: Because you can never trust all storage drives to securely wipe themselves, you must plan as if none of your drives can be securely wiped.



Placing a dependency on the type of media is not the right way to approach the problem, because the technology is always evolving and changing, and you can never be in 100% control of all IT spend. Remember that disks were never designed for secrecy first - they are designed for the opposite: reliable access. (Some disk makers like to maximize profits by selling their products as “security solutions”, but that still doesn’t make them the best choice for the job.)



For example, Shadow IT (aka the boss’s kid) is good at buying consumer equipment like SSDs, and installing it in the department desktops without asking permission. Or a non-profit might have to accept a generous donation of a hundred drives from some corporate sponsor (for political or marketing reasons), but that don’t support Secure Erase. Decent corporate laptops don’t even offer spinny disks as an option anymore, while wear-leveling algorithms ensure that SSDs always risk leaking some data in the slack spaces of the drive.



Instead, look to something that is designed to solve this exact security problem, and is something that you can control enterprise-wide, such as installing encrypted file systems that can be wiped as quickly as deleting the key. For example, in a Windows shop enforcing BitLocker via Group Policy would protect all the drives, not just the special ones you ordered.







share|improve this answer














share|improve this answer



share|improve this answer








edited Jan 12 at 4:21

























answered Jan 11 at 7:09









John DetersJohn Deters

26.6k24088




26.6k24088












  • It's not uncommon for donated devices to be provisioned with non-flash devices. If so and since wear leveling algorithms have a risk of data leakage, it seems that non-flash devices offer a higher level of assurance when employing secure destruction methods such as ATA secure erase and overwriting.

    – Motivated
    Jan 11 at 7:34











  • @Motivated , you’re confirming my point. An unspecific “level of assurance” is nowhere near the same as the permanent and total destruction of information that you would get by destroying the file system’s encryption key. Hoping a random storage device does not contain residual sensitive data is just a roll of the dice away from a breach. This requires a planned approach.

    – John Deters
    Jan 11 at 13:32






  • 1





    Overwriting is not a guaranteed form of data destruction. Here’s proof: not a single disk vendor offers a copy of DBAN with their drives and says “this is a PCI compliant disk wiping solution.” They will never offer it because they know bits of data can live on in bad blocks. Don’t try to rationalize disk wiping; don’t even consider it because it leads down a false path. Instead, look for a cryptographic solution that is purpose-designed for secure data destruction, and is independent of the storage technology. Then you can keep safely using your salvaged hard drives.

    – John Deters
    Jan 11 at 16:55






  • 2





    We set up storage arrays with encrypted drives. Sensitive systems and all laptops use OS level encrypted file systems. Applications with sensitive data use app-level encryption because encrypting sensitive data at rest is a security policy requirement. When a drive is formatted, the key is destroyed. When drives are removed from service, we track them by serial number and document the process to assure they are sent through the shredder. Different processes for different stages in the lifecycle. Notice that the drive technology doesn’t matter - the data is always secured. It’s all planned.

    – John Deters
    Jan 11 at 17:13






  • 1





    @8bittree oops, yes, I meant “secrecy”, not “security”. Thanks, I fixed it.

    – John Deters
    Jan 12 at 4:22

















  • It's not uncommon for donated devices to be provisioned with non-flash devices. If so and since wear leveling algorithms have a risk of data leakage, it seems that non-flash devices offer a higher level of assurance when employing secure destruction methods such as ATA secure erase and overwriting.

    – Motivated
    Jan 11 at 7:34











  • @Motivated , you’re confirming my point. An unspecific “level of assurance” is nowhere near the same as the permanent and total destruction of information that you would get by destroying the file system’s encryption key. Hoping a random storage device does not contain residual sensitive data is just a roll of the dice away from a breach. This requires a planned approach.

    – John Deters
    Jan 11 at 13:32






  • 1





    Overwriting is not a guaranteed form of data destruction. Here’s proof: not a single disk vendor offers a copy of DBAN with their drives and says “this is a PCI compliant disk wiping solution.” They will never offer it because they know bits of data can live on in bad blocks. Don’t try to rationalize disk wiping; don’t even consider it because it leads down a false path. Instead, look for a cryptographic solution that is purpose-designed for secure data destruction, and is independent of the storage technology. Then you can keep safely using your salvaged hard drives.

    – John Deters
    Jan 11 at 16:55






  • 2





    We set up storage arrays with encrypted drives. Sensitive systems and all laptops use OS level encrypted file systems. Applications with sensitive data use app-level encryption because encrypting sensitive data at rest is a security policy requirement. When a drive is formatted, the key is destroyed. When drives are removed from service, we track them by serial number and document the process to assure they are sent through the shredder. Different processes for different stages in the lifecycle. Notice that the drive technology doesn’t matter - the data is always secured. It’s all planned.

    – John Deters
    Jan 11 at 17:13






  • 1





    @8bittree oops, yes, I meant “secrecy”, not “security”. Thanks, I fixed it.

    – John Deters
    Jan 12 at 4:22
















It's not uncommon for donated devices to be provisioned with non-flash devices. If so and since wear leveling algorithms have a risk of data leakage, it seems that non-flash devices offer a higher level of assurance when employing secure destruction methods such as ATA secure erase and overwriting.

– Motivated
Jan 11 at 7:34





It's not uncommon for donated devices to be provisioned with non-flash devices. If so and since wear leveling algorithms have a risk of data leakage, it seems that non-flash devices offer a higher level of assurance when employing secure destruction methods such as ATA secure erase and overwriting.

– Motivated
Jan 11 at 7:34













@Motivated , you’re confirming my point. An unspecific “level of assurance” is nowhere near the same as the permanent and total destruction of information that you would get by destroying the file system’s encryption key. Hoping a random storage device does not contain residual sensitive data is just a roll of the dice away from a breach. This requires a planned approach.

– John Deters
Jan 11 at 13:32





@Motivated , you’re confirming my point. An unspecific “level of assurance” is nowhere near the same as the permanent and total destruction of information that you would get by destroying the file system’s encryption key. Hoping a random storage device does not contain residual sensitive data is just a roll of the dice away from a breach. This requires a planned approach.

– John Deters
Jan 11 at 13:32




1




1





Overwriting is not a guaranteed form of data destruction. Here’s proof: not a single disk vendor offers a copy of DBAN with their drives and says “this is a PCI compliant disk wiping solution.” They will never offer it because they know bits of data can live on in bad blocks. Don’t try to rationalize disk wiping; don’t even consider it because it leads down a false path. Instead, look for a cryptographic solution that is purpose-designed for secure data destruction, and is independent of the storage technology. Then you can keep safely using your salvaged hard drives.

– John Deters
Jan 11 at 16:55





Overwriting is not a guaranteed form of data destruction. Here’s proof: not a single disk vendor offers a copy of DBAN with their drives and says “this is a PCI compliant disk wiping solution.” They will never offer it because they know bits of data can live on in bad blocks. Don’t try to rationalize disk wiping; don’t even consider it because it leads down a false path. Instead, look for a cryptographic solution that is purpose-designed for secure data destruction, and is independent of the storage technology. Then you can keep safely using your salvaged hard drives.

– John Deters
Jan 11 at 16:55




2




2





We set up storage arrays with encrypted drives. Sensitive systems and all laptops use OS level encrypted file systems. Applications with sensitive data use app-level encryption because encrypting sensitive data at rest is a security policy requirement. When a drive is formatted, the key is destroyed. When drives are removed from service, we track them by serial number and document the process to assure they are sent through the shredder. Different processes for different stages in the lifecycle. Notice that the drive technology doesn’t matter - the data is always secured. It’s all planned.

– John Deters
Jan 11 at 17:13





We set up storage arrays with encrypted drives. Sensitive systems and all laptops use OS level encrypted file systems. Applications with sensitive data use app-level encryption because encrypting sensitive data at rest is a security policy requirement. When a drive is formatted, the key is destroyed. When drives are removed from service, we track them by serial number and document the process to assure they are sent through the shredder. Different processes for different stages in the lifecycle. Notice that the drive technology doesn’t matter - the data is always secured. It’s all planned.

– John Deters
Jan 11 at 17:13




1




1





@8bittree oops, yes, I meant “secrecy”, not “security”. Thanks, I fixed it.

– John Deters
Jan 12 at 4:22





@8bittree oops, yes, I meant “secrecy”, not “security”. Thanks, I fixed it.

– John Deters
Jan 12 at 4:22











1














Firstly, my understanding is that an SSD that properly implements the secure erase command will erase the unallocated blocks, although it may be unable to erase retired/failed blocks (these are blocks that have worn out and no longer operate correctly) and in theory these could contain recoverable data.



Secondly, HDDs also include reserved space. Most notably, this is used when a sector on the disk fails (a "bad sector") and the data must be relocated elsewhere. The original data is left behind in the bad sector which is no longer in use. Some disks also use additional reserved space as working room to rearrange other sectors so that the data from the bad sector can be physically located in a more optimal place and the disk may not erase the reserved space that was used either. In theory though, an HDD that properly implements the secure erase command will erase both bad sectors (if possible) and reserved space.



However, as other answers have pointed out, this is all relying on the proper implementation of the secure erase command and the ability for failing/failed parts of the media to be erased. The best solution may lie in something that is not dependent on the drive's own firmware supporting a particular operation.



With full disk encryption (or even file-level encryption, although be careful of filenames revealing information) you don't need to erase the actual data, just the encryption keys. In that case, the data is encrypted before it is written to disk and the disk (should) only ever contain the encrypted data. As the encryption keys are required to decrypt the encrypted data, having access to the encrypted data on the disk is useless without the encryption keys. As long as your encryption keys are securely erased or, even better, stored on a separate disk/memory device (e.g. a smart card or hardware key) the data is effectively unreadable.



Note that a lot of HDDs and SSDs are now offering "self-encryption". This is where the disk's own firmware generates an encryption key and stores it in the disk controller's internal memory, and encrypts the data itself before it is written to disk and decrypts it again after reading. The computer sees the disk as an unencrypted disk but the data actually stored on the media is encrypted. Such disks typically implement the secure erase command by deleting the encryption key from the controller's memory rather than overwriting the disk. Personally I avoid self-encrypting disks because they have a bad history of firmware bugs leading to either gaping security vulnerabilities or data loss, but the concept is the same as OS-level full disk encryption.






share|improve this answer























  • As the OP mentioned in the question, they are running on donated or reused drives. Not all those drives will offer self-encryption, nor is there assurance that any of them will have a correct and secure implementation of Secure Erase. He really shouldn’t trust random hardware to meet his security needs.

    – John Deters
    Jan 11 at 13:37











  • @Michael Johson - If secure erase and self-encrypting drives are unreliable in implementation and taking into consideration for wear leveling as suggested by John Deter, it seems that flash devices offer a lower level of assurance.

    – Motivated
    Jan 11 at 16:49











  • you don't need to erase the actual data, just the encryption keys. While practical, not secure. Erase both. You never know when a backdoor or zero-day is found for that encryption standard.

    – cde
    Jan 13 at 0:11















1














Firstly, my understanding is that an SSD that properly implements the secure erase command will erase the unallocated blocks, although it may be unable to erase retired/failed blocks (these are blocks that have worn out and no longer operate correctly) and in theory these could contain recoverable data.



Secondly, HDDs also include reserved space. Most notably, this is used when a sector on the disk fails (a "bad sector") and the data must be relocated elsewhere. The original data is left behind in the bad sector which is no longer in use. Some disks also use additional reserved space as working room to rearrange other sectors so that the data from the bad sector can be physically located in a more optimal place and the disk may not erase the reserved space that was used either. In theory though, an HDD that properly implements the secure erase command will erase both bad sectors (if possible) and reserved space.



However, as other answers have pointed out, this is all relying on the proper implementation of the secure erase command and the ability for failing/failed parts of the media to be erased. The best solution may lie in something that is not dependent on the drive's own firmware supporting a particular operation.



With full disk encryption (or even file-level encryption, although be careful of filenames revealing information) you don't need to erase the actual data, just the encryption keys. In that case, the data is encrypted before it is written to disk and the disk (should) only ever contain the encrypted data. As the encryption keys are required to decrypt the encrypted data, having access to the encrypted data on the disk is useless without the encryption keys. As long as your encryption keys are securely erased or, even better, stored on a separate disk/memory device (e.g. a smart card or hardware key) the data is effectively unreadable.



Note that a lot of HDDs and SSDs are now offering "self-encryption". This is where the disk's own firmware generates an encryption key and stores it in the disk controller's internal memory, and encrypts the data itself before it is written to disk and decrypts it again after reading. The computer sees the disk as an unencrypted disk but the data actually stored on the media is encrypted. Such disks typically implement the secure erase command by deleting the encryption key from the controller's memory rather than overwriting the disk. Personally I avoid self-encrypting disks because they have a bad history of firmware bugs leading to either gaping security vulnerabilities or data loss, but the concept is the same as OS-level full disk encryption.






share|improve this answer























  • As the OP mentioned in the question, they are running on donated or reused drives. Not all those drives will offer self-encryption, nor is there assurance that any of them will have a correct and secure implementation of Secure Erase. He really shouldn’t trust random hardware to meet his security needs.

    – John Deters
    Jan 11 at 13:37











  • @Michael Johson - If secure erase and self-encrypting drives are unreliable in implementation and taking into consideration for wear leveling as suggested by John Deter, it seems that flash devices offer a lower level of assurance.

    – Motivated
    Jan 11 at 16:49











  • you don't need to erase the actual data, just the encryption keys. While practical, not secure. Erase both. You never know when a backdoor or zero-day is found for that encryption standard.

    – cde
    Jan 13 at 0:11













1












1








1







Firstly, my understanding is that an SSD that properly implements the secure erase command will erase the unallocated blocks, although it may be unable to erase retired/failed blocks (these are blocks that have worn out and no longer operate correctly) and in theory these could contain recoverable data.



Secondly, HDDs also include reserved space. Most notably, this is used when a sector on the disk fails (a "bad sector") and the data must be relocated elsewhere. The original data is left behind in the bad sector which is no longer in use. Some disks also use additional reserved space as working room to rearrange other sectors so that the data from the bad sector can be physically located in a more optimal place and the disk may not erase the reserved space that was used either. In theory though, an HDD that properly implements the secure erase command will erase both bad sectors (if possible) and reserved space.



However, as other answers have pointed out, this is all relying on the proper implementation of the secure erase command and the ability for failing/failed parts of the media to be erased. The best solution may lie in something that is not dependent on the drive's own firmware supporting a particular operation.



With full disk encryption (or even file-level encryption, although be careful of filenames revealing information) you don't need to erase the actual data, just the encryption keys. In that case, the data is encrypted before it is written to disk and the disk (should) only ever contain the encrypted data. As the encryption keys are required to decrypt the encrypted data, having access to the encrypted data on the disk is useless without the encryption keys. As long as your encryption keys are securely erased or, even better, stored on a separate disk/memory device (e.g. a smart card or hardware key) the data is effectively unreadable.



Note that a lot of HDDs and SSDs are now offering "self-encryption". This is where the disk's own firmware generates an encryption key and stores it in the disk controller's internal memory, and encrypts the data itself before it is written to disk and decrypts it again after reading. The computer sees the disk as an unencrypted disk but the data actually stored on the media is encrypted. Such disks typically implement the secure erase command by deleting the encryption key from the controller's memory rather than overwriting the disk. Personally I avoid self-encrypting disks because they have a bad history of firmware bugs leading to either gaping security vulnerabilities or data loss, but the concept is the same as OS-level full disk encryption.






share|improve this answer













Firstly, my understanding is that an SSD that properly implements the secure erase command will erase the unallocated blocks, although it may be unable to erase retired/failed blocks (these are blocks that have worn out and no longer operate correctly) and in theory these could contain recoverable data.



Secondly, HDDs also include reserved space. Most notably, this is used when a sector on the disk fails (a "bad sector") and the data must be relocated elsewhere. The original data is left behind in the bad sector which is no longer in use. Some disks also use additional reserved space as working room to rearrange other sectors so that the data from the bad sector can be physically located in a more optimal place and the disk may not erase the reserved space that was used either. In theory though, an HDD that properly implements the secure erase command will erase both bad sectors (if possible) and reserved space.



However, as other answers have pointed out, this is all relying on the proper implementation of the secure erase command and the ability for failing/failed parts of the media to be erased. The best solution may lie in something that is not dependent on the drive's own firmware supporting a particular operation.



With full disk encryption (or even file-level encryption, although be careful of filenames revealing information) you don't need to erase the actual data, just the encryption keys. In that case, the data is encrypted before it is written to disk and the disk (should) only ever contain the encrypted data. As the encryption keys are required to decrypt the encrypted data, having access to the encrypted data on the disk is useless without the encryption keys. As long as your encryption keys are securely erased or, even better, stored on a separate disk/memory device (e.g. a smart card or hardware key) the data is effectively unreadable.



Note that a lot of HDDs and SSDs are now offering "self-encryption". This is where the disk's own firmware generates an encryption key and stores it in the disk controller's internal memory, and encrypts the data itself before it is written to disk and decrypts it again after reading. The computer sees the disk as an unencrypted disk but the data actually stored on the media is encrypted. Such disks typically implement the secure erase command by deleting the encryption key from the controller's memory rather than overwriting the disk. Personally I avoid self-encrypting disks because they have a bad history of firmware bugs leading to either gaping security vulnerabilities or data loss, but the concept is the same as OS-level full disk encryption.







share|improve this answer












share|improve this answer



share|improve this answer










answered Jan 11 at 8:55









Micheal JohnsonMicheal Johnson

1,2981614




1,2981614












  • As the OP mentioned in the question, they are running on donated or reused drives. Not all those drives will offer self-encryption, nor is there assurance that any of them will have a correct and secure implementation of Secure Erase. He really shouldn’t trust random hardware to meet his security needs.

    – John Deters
    Jan 11 at 13:37











  • @Michael Johson - If secure erase and self-encrypting drives are unreliable in implementation and taking into consideration for wear leveling as suggested by John Deter, it seems that flash devices offer a lower level of assurance.

    – Motivated
    Jan 11 at 16:49











  • you don't need to erase the actual data, just the encryption keys. While practical, not secure. Erase both. You never know when a backdoor or zero-day is found for that encryption standard.

    – cde
    Jan 13 at 0:11

















  • As the OP mentioned in the question, they are running on donated or reused drives. Not all those drives will offer self-encryption, nor is there assurance that any of them will have a correct and secure implementation of Secure Erase. He really shouldn’t trust random hardware to meet his security needs.

    – John Deters
    Jan 11 at 13:37











  • @Michael Johson - If secure erase and self-encrypting drives are unreliable in implementation and taking into consideration for wear leveling as suggested by John Deter, it seems that flash devices offer a lower level of assurance.

    – Motivated
    Jan 11 at 16:49











  • you don't need to erase the actual data, just the encryption keys. While practical, not secure. Erase both. You never know when a backdoor or zero-day is found for that encryption standard.

    – cde
    Jan 13 at 0:11
















As the OP mentioned in the question, they are running on donated or reused drives. Not all those drives will offer self-encryption, nor is there assurance that any of them will have a correct and secure implementation of Secure Erase. He really shouldn’t trust random hardware to meet his security needs.

– John Deters
Jan 11 at 13:37





As the OP mentioned in the question, they are running on donated or reused drives. Not all those drives will offer self-encryption, nor is there assurance that any of them will have a correct and secure implementation of Secure Erase. He really shouldn’t trust random hardware to meet his security needs.

– John Deters
Jan 11 at 13:37













@Michael Johson - If secure erase and self-encrypting drives are unreliable in implementation and taking into consideration for wear leveling as suggested by John Deter, it seems that flash devices offer a lower level of assurance.

– Motivated
Jan 11 at 16:49





@Michael Johson - If secure erase and self-encrypting drives are unreliable in implementation and taking into consideration for wear leveling as suggested by John Deter, it seems that flash devices offer a lower level of assurance.

– Motivated
Jan 11 at 16:49













you don't need to erase the actual data, just the encryption keys. While practical, not secure. Erase both. You never know when a backdoor or zero-day is found for that encryption standard.

– cde
Jan 13 at 0:11





you don't need to erase the actual data, just the encryption keys. While practical, not secure. Erase both. You never know when a backdoor or zero-day is found for that encryption standard.

– cde
Jan 13 at 0:11











0














Solid state disks are definitively to be preferred. Note that they are not without their troubles either, since sometimes implementors just suck (and Windows/Bitlocker sucks, too).



Traditional disk drives have been "encrypting" (or rather mixing) data weakly since pretty much forever to distribute bits better, but this doesn't help much in protecting data. More recently, there exist harddrives which are self-encrypting disks (SED), but as harddisks they are kinda "prestige" products and outrageous in price. I haven't so far owned one.



Solid state disks are practically always SED, but the feature set, and more importantly, the quality of the implementation differs a lot. As you can read in the linked article, for example, earlier models from Crucial used an encryption that was total bollocks. The user's password is compared to a hash by the firmware to "unlock" the drive's encryption key as opposed to e.g. Samsung's drives which use PBKDF2 to derive the key from the password. Which, in terms of actual versus misleading security is worlds in between.



Luckily, in any case, and regardless of bad implementations, the security-while-used is much more affected than the security-after-erased. Well... luckily, I don't know if that's a good wording, actually systems should always be secure. But at least it doesn't suck beyond.



There exist the notion of "master password" in the ATA standard, so any such thing as unlocking verus deriving an encryption key is -- even not considering that someone might find a way to read out the storage -- catastrophic. It basically means nothing is encrypted at all in a meaningful way.



Secure erase on a SED means erasing the disk encryption key, rendering the contents of the complete disk unreadable. So, unless one assumes a maliciously-built drive (which tells you it did a secure erase, but secretly still holds a copy of the key), this is secure even in presence of a broken implementation, and even in presence of someone cracking open the controller chip or such.



Secure erase on a traditional harddisk means the disk will overwrite every sector. I've recently done that with a pre-fail (SMART showing errors) Seagate Barracuda that was to be RMAed.

And guess what, secure erase is all nice and well, but a pre-fail disk will simply refuse to do the job. It'll start, whack around for a few minutes, and terminate with "error blah blah" after erasing approximately 10% of the disk. That wasn't an issue in my case since the data on the disk was from a RAID with software encryption on top, so any contents was basically useless anyway (wiping not really necessary). But, you get the idea. If you didn't use an encrypted filesystem, there's now no way to erase the data!



Generally, wear-levelling (both on traditional disks and SSDs) may make overwriting stuff much less possible than you are maybe inclined to believe.



Also, restoring overwritten data on a magnetic disk is possible. Yes, it is much, much harder than it was 15-20 years ago when data density was much lower (back then, it was pretty much a routine job). But it is still... generally possible.



So, if the data is truly super-sensitive (as in medical records), either one should layer software encryption on top, which eliminates the need to wipe the disk (though it doesn't hurt to do it anyway), or one should not donate the drives but use one of these to be sure.






share|improve this answer























  • There's the question whether you want to trust a "self encrypting" drive. Lots of encryption is created by clueless dolts.

    – gnasher729
    Jan 13 at 11:35











  • @gnasher729: Well the encryption per se isn't the problem since they all use AES-256. It's what is readily supported in hardware, cheap, and standard. The problem is when they do stuff like on the Crucial drives pointed out in the article where the DEK is stored on the device and "unlocked" by comparing password strings (rather than deriving a key from user input). That's about as trivial to crack as the typical if(trial == true) ...; code found in a lot of software which takes a 12 year old three minutes to patch. But even so... for disposing this doesn't matter. Key erased is erased.

    – Damon
    Jan 13 at 22:51
















0














Solid state disks are definitively to be preferred. Note that they are not without their troubles either, since sometimes implementors just suck (and Windows/Bitlocker sucks, too).



Traditional disk drives have been "encrypting" (or rather mixing) data weakly since pretty much forever to distribute bits better, but this doesn't help much in protecting data. More recently, there exist harddrives which are self-encrypting disks (SED), but as harddisks they are kinda "prestige" products and outrageous in price. I haven't so far owned one.



Solid state disks are practically always SED, but the feature set, and more importantly, the quality of the implementation differs a lot. As you can read in the linked article, for example, earlier models from Crucial used an encryption that was total bollocks. The user's password is compared to a hash by the firmware to "unlock" the drive's encryption key as opposed to e.g. Samsung's drives which use PBKDF2 to derive the key from the password. Which, in terms of actual versus misleading security is worlds in between.



Luckily, in any case, and regardless of bad implementations, the security-while-used is much more affected than the security-after-erased. Well... luckily, I don't know if that's a good wording, actually systems should always be secure. But at least it doesn't suck beyond.



There exist the notion of "master password" in the ATA standard, so any such thing as unlocking verus deriving an encryption key is -- even not considering that someone might find a way to read out the storage -- catastrophic. It basically means nothing is encrypted at all in a meaningful way.



Secure erase on a SED means erasing the disk encryption key, rendering the contents of the complete disk unreadable. So, unless one assumes a maliciously-built drive (which tells you it did a secure erase, but secretly still holds a copy of the key), this is secure even in presence of a broken implementation, and even in presence of someone cracking open the controller chip or such.



Secure erase on a traditional harddisk means the disk will overwrite every sector. I've recently done that with a pre-fail (SMART showing errors) Seagate Barracuda that was to be RMAed.

And guess what, secure erase is all nice and well, but a pre-fail disk will simply refuse to do the job. It'll start, whack around for a few minutes, and terminate with "error blah blah" after erasing approximately 10% of the disk. That wasn't an issue in my case since the data on the disk was from a RAID with software encryption on top, so any contents was basically useless anyway (wiping not really necessary). But, you get the idea. If you didn't use an encrypted filesystem, there's now no way to erase the data!



Generally, wear-levelling (both on traditional disks and SSDs) may make overwriting stuff much less possible than you are maybe inclined to believe.



Also, restoring overwritten data on a magnetic disk is possible. Yes, it is much, much harder than it was 15-20 years ago when data density was much lower (back then, it was pretty much a routine job). But it is still... generally possible.



So, if the data is truly super-sensitive (as in medical records), either one should layer software encryption on top, which eliminates the need to wipe the disk (though it doesn't hurt to do it anyway), or one should not donate the drives but use one of these to be sure.






share|improve this answer























  • There's the question whether you want to trust a "self encrypting" drive. Lots of encryption is created by clueless dolts.

    – gnasher729
    Jan 13 at 11:35











  • @gnasher729: Well the encryption per se isn't the problem since they all use AES-256. It's what is readily supported in hardware, cheap, and standard. The problem is when they do stuff like on the Crucial drives pointed out in the article where the DEK is stored on the device and "unlocked" by comparing password strings (rather than deriving a key from user input). That's about as trivial to crack as the typical if(trial == true) ...; code found in a lot of software which takes a 12 year old three minutes to patch. But even so... for disposing this doesn't matter. Key erased is erased.

    – Damon
    Jan 13 at 22:51














0












0








0







Solid state disks are definitively to be preferred. Note that they are not without their troubles either, since sometimes implementors just suck (and Windows/Bitlocker sucks, too).



Traditional disk drives have been "encrypting" (or rather mixing) data weakly since pretty much forever to distribute bits better, but this doesn't help much in protecting data. More recently, there exist harddrives which are self-encrypting disks (SED), but as harddisks they are kinda "prestige" products and outrageous in price. I haven't so far owned one.



Solid state disks are practically always SED, but the feature set, and more importantly, the quality of the implementation differs a lot. As you can read in the linked article, for example, earlier models from Crucial used an encryption that was total bollocks. The user's password is compared to a hash by the firmware to "unlock" the drive's encryption key as opposed to e.g. Samsung's drives which use PBKDF2 to derive the key from the password. Which, in terms of actual versus misleading security is worlds in between.



Luckily, in any case, and regardless of bad implementations, the security-while-used is much more affected than the security-after-erased. Well... luckily, I don't know if that's a good wording, actually systems should always be secure. But at least it doesn't suck beyond.



There exist the notion of "master password" in the ATA standard, so any such thing as unlocking verus deriving an encryption key is -- even not considering that someone might find a way to read out the storage -- catastrophic. It basically means nothing is encrypted at all in a meaningful way.



Secure erase on a SED means erasing the disk encryption key, rendering the contents of the complete disk unreadable. So, unless one assumes a maliciously-built drive (which tells you it did a secure erase, but secretly still holds a copy of the key), this is secure even in presence of a broken implementation, and even in presence of someone cracking open the controller chip or such.



Secure erase on a traditional harddisk means the disk will overwrite every sector. I've recently done that with a pre-fail (SMART showing errors) Seagate Barracuda that was to be RMAed.

And guess what, secure erase is all nice and well, but a pre-fail disk will simply refuse to do the job. It'll start, whack around for a few minutes, and terminate with "error blah blah" after erasing approximately 10% of the disk. That wasn't an issue in my case since the data on the disk was from a RAID with software encryption on top, so any contents was basically useless anyway (wiping not really necessary). But, you get the idea. If you didn't use an encrypted filesystem, there's now no way to erase the data!



Generally, wear-levelling (both on traditional disks and SSDs) may make overwriting stuff much less possible than you are maybe inclined to believe.



Also, restoring overwritten data on a magnetic disk is possible. Yes, it is much, much harder than it was 15-20 years ago when data density was much lower (back then, it was pretty much a routine job). But it is still... generally possible.



So, if the data is truly super-sensitive (as in medical records), either one should layer software encryption on top, which eliminates the need to wipe the disk (though it doesn't hurt to do it anyway), or one should not donate the drives but use one of these to be sure.






share|improve this answer













Solid state disks are definitively to be preferred. Note that they are not without their troubles either, since sometimes implementors just suck (and Windows/Bitlocker sucks, too).



Traditional disk drives have been "encrypting" (or rather mixing) data weakly since pretty much forever to distribute bits better, but this doesn't help much in protecting data. More recently, there exist harddrives which are self-encrypting disks (SED), but as harddisks they are kinda "prestige" products and outrageous in price. I haven't so far owned one.



Solid state disks are practically always SED, but the feature set, and more importantly, the quality of the implementation differs a lot. As you can read in the linked article, for example, earlier models from Crucial used an encryption that was total bollocks. The user's password is compared to a hash by the firmware to "unlock" the drive's encryption key as opposed to e.g. Samsung's drives which use PBKDF2 to derive the key from the password. Which, in terms of actual versus misleading security is worlds in between.



Luckily, in any case, and regardless of bad implementations, the security-while-used is much more affected than the security-after-erased. Well... luckily, I don't know if that's a good wording, actually systems should always be secure. But at least it doesn't suck beyond.



There exist the notion of "master password" in the ATA standard, so any such thing as unlocking verus deriving an encryption key is -- even not considering that someone might find a way to read out the storage -- catastrophic. It basically means nothing is encrypted at all in a meaningful way.



Secure erase on a SED means erasing the disk encryption key, rendering the contents of the complete disk unreadable. So, unless one assumes a maliciously-built drive (which tells you it did a secure erase, but secretly still holds a copy of the key), this is secure even in presence of a broken implementation, and even in presence of someone cracking open the controller chip or such.



Secure erase on a traditional harddisk means the disk will overwrite every sector. I've recently done that with a pre-fail (SMART showing errors) Seagate Barracuda that was to be RMAed.

And guess what, secure erase is all nice and well, but a pre-fail disk will simply refuse to do the job. It'll start, whack around for a few minutes, and terminate with "error blah blah" after erasing approximately 10% of the disk. That wasn't an issue in my case since the data on the disk was from a RAID with software encryption on top, so any contents was basically useless anyway (wiping not really necessary). But, you get the idea. If you didn't use an encrypted filesystem, there's now no way to erase the data!



Generally, wear-levelling (both on traditional disks and SSDs) may make overwriting stuff much less possible than you are maybe inclined to believe.



Also, restoring overwritten data on a magnetic disk is possible. Yes, it is much, much harder than it was 15-20 years ago when data density was much lower (back then, it was pretty much a routine job). But it is still... generally possible.



So, if the data is truly super-sensitive (as in medical records), either one should layer software encryption on top, which eliminates the need to wipe the disk (though it doesn't hurt to do it anyway), or one should not donate the drives but use one of these to be sure.







share|improve this answer












share|improve this answer



share|improve this answer










answered Jan 12 at 16:57









DamonDamon

2,857715




2,857715












  • There's the question whether you want to trust a "self encrypting" drive. Lots of encryption is created by clueless dolts.

    – gnasher729
    Jan 13 at 11:35











  • @gnasher729: Well the encryption per se isn't the problem since they all use AES-256. It's what is readily supported in hardware, cheap, and standard. The problem is when they do stuff like on the Crucial drives pointed out in the article where the DEK is stored on the device and "unlocked" by comparing password strings (rather than deriving a key from user input). That's about as trivial to crack as the typical if(trial == true) ...; code found in a lot of software which takes a 12 year old three minutes to patch. But even so... for disposing this doesn't matter. Key erased is erased.

    – Damon
    Jan 13 at 22:51


















  • There's the question whether you want to trust a "self encrypting" drive. Lots of encryption is created by clueless dolts.

    – gnasher729
    Jan 13 at 11:35











  • @gnasher729: Well the encryption per se isn't the problem since they all use AES-256. It's what is readily supported in hardware, cheap, and standard. The problem is when they do stuff like on the Crucial drives pointed out in the article where the DEK is stored on the device and "unlocked" by comparing password strings (rather than deriving a key from user input). That's about as trivial to crack as the typical if(trial == true) ...; code found in a lot of software which takes a 12 year old three minutes to patch. But even so... for disposing this doesn't matter. Key erased is erased.

    – Damon
    Jan 13 at 22:51

















There's the question whether you want to trust a "self encrypting" drive. Lots of encryption is created by clueless dolts.

– gnasher729
Jan 13 at 11:35





There's the question whether you want to trust a "self encrypting" drive. Lots of encryption is created by clueless dolts.

– gnasher729
Jan 13 at 11:35













@gnasher729: Well the encryption per se isn't the problem since they all use AES-256. It's what is readily supported in hardware, cheap, and standard. The problem is when they do stuff like on the Crucial drives pointed out in the article where the DEK is stored on the device and "unlocked" by comparing password strings (rather than deriving a key from user input). That's about as trivial to crack as the typical if(trial == true) ...; code found in a lot of software which takes a 12 year old three minutes to patch. But even so... for disposing this doesn't matter. Key erased is erased.

– Damon
Jan 13 at 22:51






@gnasher729: Well the encryption per se isn't the problem since they all use AES-256. It's what is readily supported in hardware, cheap, and standard. The problem is when they do stuff like on the Crucial drives pointed out in the article where the DEK is stored on the device and "unlocked" by comparing password strings (rather than deriving a key from user input). That's about as trivial to crack as the typical if(trial == true) ...; code found in a lot of software which takes a 12 year old three minutes to patch. But even so... for disposing this doesn't matter. Key erased is erased.

– Damon
Jan 13 at 22:51


















draft saved

draft discarded
















































Thanks for contributing an answer to Information Security Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsecurity.stackexchange.com%2fquestions%2f201257%2fdoes-the-destruction-of-sensitive-information-limit-the-choice-of-hard-drives-to%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown






Popular posts from this blog

How to check contact read email or not when send email to Individual?

Displaying single band from multi-band raster using QGIS

How many registers does an x86_64 CPU actually have?