Yeah, there’s a good deal of abstraction between the OS and the physical storage with solid state media.
This is the same reason capacities can be spoofed and people think they are buying 60TB portable SSDs for $10 (and they remain adamant that it’s real and not a scam because they plugged it in and looked at the ‘properties’).
I’ve never bought a faked capacity drive.
I wonder if the makers of those things have a way to fool the OS into writing the stated capacity, even though most of the incoming bits go straight into the bucket. IOW, where the drive would allow you to seemingly fill it with 60TB of backups, and you’d be none the wiser until you tried to read much of it back out and got only “read error” in return?
Or do they just do the easy thing where if the drive’s real capacity is 1TB, the drive accurately reports “Drive full” when it hits 1 TB written?
This stuff is scary. They’re not just putting a small drive in a larger housing, they’re actually making malevolent storage controllers that will allow you to write terabytes onto the drive, keeping the latest 64 GB but erasing the older data in FIFO manner. Which, for many people, means they won’t notice the missing data for a long while.
As I understand it, though, a FULL format that can be done through the computer itself sets everything to zeros. That should be sufficient, should it not? There are also “kill disk” CDs that can do the same thing.
It’s true that a full format will erase substantially all the data such that ordinary software tools can recover nothing. It’s also true that on a large drive, a “full format” can takes hours to run.
The whole rest of this thread is about whether e.g. the NSA could recover some or all of the full-format erased data by reading “the shadows of where the data had been.” The consensus is that if that ever was a realistic concern, it’s even less so today.
Now as to who among us have data that the NSA would be willing to painstakingly reconstruct from shadows is a different question. Even assuming it was possible for them to do so.
As always when discussing defensive measures, first you need to decide who / what you’re defending against.
Thanks for making a long story short, LOL ![]()
Yeah, with pretty much all storage media, the capacity is what the controller onboard the storage device says it is - there is a handshake during which the OS asks ‘what the hell are you?’ and the device responds with its identity and properties. It can claim to be anything.
Typically a storage device with spoofed capacity will accept files beyond the limit of its true physical storage capacity, but what happens next is not always the same and depends on the file system used and the programming of the onboard controller.
In the typical case, the file system is one where the metadata is stored in the lowest addresses in the storage (which corresponds with the real physical media); the file contents are stored elsewhere - so you write files to the drive - they appear to be stored and their filenames and sizes are reported correctly, but when the limit of actual storage was reached, the file contents just began to be tipped into the void; when you try to retrieve these files (but only then), you get a big load of zeroes back; typically resulting in the file being reported as corrupted but in the case of simple things like text files, just a big blank file.
In some cases though, the onboard controller will have been programmed to cycle around when the limit of physical storage is reached - this is devious because it means that if you verify files as you write them, the most recently-written file appears intact and successfully copied, but it has overwritten the contents of files you wrote earlier (the metadata may still look good here if the looping is only happening in a higher part of the storage).
In a few cases I have seen, the onboard controller needs some of the flash storage for its own code or other internal use and if these do the thing where writing loops back to the start when it reaches the end of the physical storage, this overwrites the system data and/or filesystem metadata and the device stops working at that point and may be bricked.
So we have the “good”, “better”, “best” model of fakery. Just explained by you in better, best, good order. ![]()
Evil bastards should be strung up by their 'nads.
FWIW, the protocol to “lie” about the capacity can be a good thing. In my ancient past, I wrote what was effectively SCSI firmware for a system that used a big Unix box as back end storage. The back end storage had lots of drives, and I could configure them as various RAID levels or plain partitions, and present them as individual SCSI devices or LUNs. I had to calculate capacity at run time so I could correctly answer the inquiry related commands. I wasn’t actually lying in this case, but I was using the mechanism that allowed others to lie if they wanted to.
Side note - it was more than just capacity. I was reporting number of heads and cylinders too. And I stumbled on a bug in one of the major Unix vendors at the time that assumed that drives were not extremely tall and thin
They were ordering certain parts of their code based on physical architecture of the drive for performance reasons, and had an if statement backwards in an extremely unlikely real world case.
So your “drive” with 1 cylinder of 1,000,000 tracks of 100 bytes each provoked some … indigestion? ![]()
Perhaps the most telling case was of one of the 9-11 hijackers (or rather, the guy they arrested who appeared to be training like them) who had been using the public computer at a copy store. The FBI took the computer for analysis, but found nothing. It had been wiped several times by then, IIRC.
How about a steamroller?
Beyond the theoretical exercise of recovering data, what is it that you (or anyone) is worried about? Unless you are a spy, 99.999% of the data on your drive is worthless. The only data I would worry about is passwords to bank accounts, and those are already in an encrypted file. Everything else is personal data that has little value. So, it’s an interesting exercise, but really doesn’t require worrying about.
Oh, and - SSDs? They are basically impossible to recover erased data from these days. Any SSD that uses the TRIM command is going to wipe a deleted block very quickly. The upshot of which is: keep a backup! If you accidentally erase an important file, assume that it’s gone forever.
Ah yes. I remember all that complicated ‘placement optimization’ code in Berkeley UFS! Became irrelevent once SCSI essentially arrived as the de-facto disk interconnect of the time, and was just a legacy pain that had to be maintained…
You have to hand it to him, though: heating the platters past the Curie point elegantly disrupts the magnetic domains while reconstructing the structure from a puddle of slag seems even harder than reassembling a bent-up jigsaw puzzle.
One has to admit though, there’s something wonderfully Victorian about a steamroller!
I expect Rob Wilkins was pretty scrupulous about keeping backups, but I hope he keeps them severely under his hat. We don’t want any Pratchett ‘fan fiction’ emerging. I have in fact seen a couple of unauthorized examples, and they were absolutely cringeworthy: totally devoid of the spark and humour of Terry’s work!
No need for the FBI/NSA to recover your deleted files. Soon as you started typing a key word or two, they put you under surveillance.
Ref: The Paws of Mr. Claus - Arlo Guthrie
“..And within two minutes, and not two minutes from when he hangs up the phone, but two minutes from when he first put the dime in, they got 30,000 feet of tape rollin, files on tape, pictures, movies, dramas, actions on tape.”
they got 30,000 feet of tape rollin, files on tape, pictures, movies, dramas, actions on tape
And of course the 8 by 10 glossy photographs… with circles and arrows and a paragraph on the back of each one explaining….
It’s actually harder on an SSD to reliably unrecoverably delete data. Read @DPRK’s cite in post 2 for more.
The SSD is lying to the motherboard and the motherboard is lying to the OS. Which might be lying to you, the end user.
Getting past all the lying to force the SSD to really truly delete everywhere the data is or ever was is a tallish order.
All this is vastly more complicated now than back in the 1980s w a small spinning HD on a PC.
Kinda? It depends on your definition of delete. It’s pretty easy to Bitlocker an entire SSD and then delete the key.
FWIW, the protocol to “lie” about the capacity can be a good thing.
In general, it is. When you plug a storage device into your machine you don’t really want to wait while it counts and verifies that every bit is really there - the host just trusts that the device is what it says it is, because the whole idea of that came about in a time where nobody was really thinking about how to use computers to scam people.
Same thing with email protocols - they were created in a more innocent time where the headers just honestly stated who you are because why else would they exist?