1984 wrote:I was under the impression that programs with "secure delete" options would not just delete from the filesystem but would actually locate the space on the drive where the data was located and then overwrite it?
Allegedly, yes. If somebody wrote a program which performed an ordinary delete and claimed it was secure, could you tell the difference?
The only way to know with these programs is by disassembling them. Reverse engineers typically don't bother because they don't rely on such software to delete things securely.
Even if a program did locate the physical sectors a file is on and overwrote them, that may still not be enough. Journaling filesystems like NTFS write operations to the journal to allow recovery in case of unexpected data loss. This creates data duplicity in the journal which may contain sensitive information which a user was under the impression had been securely deleted.
NTFS journals can be deleted, Linux livecds can be booted and followed by shred and all available space of a drive can be cleaned with
dd if=/dev/null, and when the OP messes up and deletes more data than he intended or sensitive data which he wanted gone remained and was recovered it will be our fault for giving him advice that went over his head. I prefer giving advice which is easy to follow to laymen and has the intended results, and in my point of view the easiest and most reliable way to solve this problem is to follow the advice I originally gave.
As for the possibility to recover data that has been properly deleted (overwritten), that is a myth that stems from a 1996 research paper called
Secure Deletion of Data from Magnetic and Solid-State Memory. The myth is spread by people who have not read the paper or did not understand that those techniques applied to 30+ year old technology at the time the paper was written. Quoting Peter Gutmann, the author, himself (emphasis added):
In the time since this paper was published, some people have treated the 35-pass overwrite technique described in it more as a kind of voodoo incantation to banish evil spirits than the result of a technical analysis of drive encoding techniques. As a result, they advocate applying the voodoo to PRML and EPRML drives even though it will have no more effect than a simple scrubbing with random data. In fact performing the full 35-pass overwrite is pointless for any drive since it targets a blend of scenarios involving all types of (normally-used) encoding technology, which covers everything back to 30+-year-old MFM methods (if you don't understand that statement, re-read the paper). If you're using a drive which uses encoding technology X, you only need to perform the passes specific to X, and you never need to perform all 35 passes. For any modern PRML/EPRML drive, a few passes of random scrubbing is the best you can do. As the paper says, "A good scrubbing with random data will do about as well as can be expected". This was true in 1996, and is still true now.
The US government standardized voodoo deletion via
its data erasure policies like DoD 5220.22-M, which many of these "secure delete" programs follow. They also melt drives that have been used to store data of the highest level of confidentiality. That doesn't mean that it's necessary, or that logos needs to throw his drive in lava.
As a matter of fact, nobody has ever recovered data from a modern drive that has been properly overwritten only once. There is a lot of hocus pocus told by data recovery firms and echoed by people who have not done their research or ever tried to perform data recovery themselves.
There are many misunderstandings. Apologies if I sound like a total besserwisser.
Bless
[quote="1984"]I was under the impression that programs with "secure delete" options would not just delete from the filesystem but would actually locate the space on the drive where the data was located and then overwrite it?[/quote]
Allegedly, yes. If somebody wrote a program which performed an ordinary delete and claimed it was secure, could you tell the difference?
The only way to know with these programs is by disassembling them. Reverse engineers typically don't bother because they don't rely on such software to delete things securely.
Even if a program did locate the physical sectors a file is on and overwrote them, that may still not be enough. Journaling filesystems like NTFS write operations to the journal to allow recovery in case of unexpected data loss. This creates data duplicity in the journal which may contain sensitive information which a user was under the impression had been securely deleted.
NTFS journals can be deleted, Linux livecds can be booted and followed by shred and all available space of a drive can be cleaned with [i]dd if=/dev/null[/i], and when the OP messes up and deletes more data than he intended or sensitive data which he wanted gone remained and was recovered it will be our fault for giving him advice that went over his head. I prefer giving advice which is easy to follow to laymen and has the intended results, and in my point of view the easiest and most reliable way to solve this problem is to follow the advice I originally gave.
As for the possibility to recover data that has been properly deleted (overwritten), that is a myth that stems from a 1996 research paper called [i]Secure Deletion of Data from Magnetic and Solid-State Memory[/i]. The myth is spread by people who have not read the paper or did not understand that those techniques applied to 30+ year old technology at the time the paper was written. Quoting Peter Gutmann, the author, himself (emphasis added):
[quote]In the time since this paper was published, some people have treated the 35-pass overwrite technique described in it more as a kind of voodoo incantation to banish evil spirits than the result of a technical analysis of drive encoding techniques. As a result, they advocate applying the voodoo to PRML and EPRML drives even though [b]it will have no more effect than a simple scrubbing with random data[/b]. In fact [b]performing the full 35-pass overwrite is pointless for any drive since it targets a blend of scenarios involving all types of (normally-used) encoding technology[/b], which covers everything back to 30+-year-old MFM methods (if you don't understand that statement, re-read the paper). If you're using a drive which uses encoding technology X, you only need to perform the passes specific to X, and [b]you never need to perform all 35 passes[/b]. For any modern PRML/EPRML drive, a few passes of random scrubbing is the best you can do. [b]As the paper says, "A good scrubbing with random data will do about as well as can be expected". This was true in 1996, and is still true now.[/b][/quote]
The US government standardized voodoo deletion via [url=https://en.wikipedia.org/wiki/Data_erasure#Standards]its data erasure policies[/url] like DoD 5220.22-M, which many of these "secure delete" programs follow. They also melt drives that have been used to store data of the highest level of confidentiality. That doesn't mean that it's necessary, or that logos needs to throw his drive in lava.
As a matter of fact, nobody has ever recovered data from a modern drive that has been properly overwritten only once. There is a lot of hocus pocus told by data recovery firms and echoed by people who have not done their research or ever tried to perform data recovery themselves.
There are many misunderstandings. Apologies if I sound like a total besserwisser.
Bless