Modern Windows O.S. Inquiry About Deleting A File

Factual Questions
1

This is a question more about data integrity than anything else.

In a Windows environment, many computer users know that when you delete a file using the trash the file data is not erased, but is simply marked as unavailable by the Windows O.S. (This implies the user has confirmed that Windows can indeed delete the file[s].)

But I know a little how an operating system works. Once a file is marked as deleted the disc sectors/blocks/clusters are returned to the pool of available storage units. Windows is a multitasking environment. There could be many applications running needing to write disc units. The question is this: If I delete a file via the trash, and confirming deletion, how many SECONDS might that data actually remain intact? I speculate less than thirty.

Thus, a conventional deletion via the trash is nearly guaranteed to erase the actual data stored on disc.

Agree?

2

Impossible to say without more information. However, it’s entirely possible for erased information to hang around for years. Hence it’s recommended that you scrub or destroy drives before passing them on to untrustworthy parties.

3

This.

Generally speaking, drives try and write things in contiguous blocks if at all possible, but that doesn’t always happen- defragmentation, new files, file deletions, etc… all cause drive space to be written/overwritten constantly. But the thing is that if you have lots of free space, hard drives tend to write files contiguously and sequentially in an effort not to fragment files.

This scenario is entirely possible- your drive is 65% full and defragmented. Then you write a specific file. That file is going to be at the 65% point on your drive, however it organizes such things. Then if say… 30% of your drive space is freed up- you delete a bunch of pictures of your ex or something.

And then you delete your file. Now you basically have the first 35% of your drive full (figuratively speaking- see note below), and that file you just deleted is somewhere where the drive isn’t likely to write until you fill up another 30% of the space.

So that file is going to be “deleted”, but in actuality sit out there until it’s overwritten, which may take a while depending on how fast you fill up your drive.

*note- with multiple platters and heads, it’s not like a CD where it reads from the rim to the hub, but logically speaking it’s close enough.

I used to do computer forensics, and we used to find ancient deleted files and fragments of older ones ALL the time on computers.

4

Also, modern drives keep track of how many times each sector has been used (written to). Needed for error detection & correction of failing sectors. But this can also be used to ‘spread the wear’, by writing new or updated files into sectors that haven’t been used lately. So the original copy is still there in it’s original location, intact, but that space is marked as available.

With the ever increasing capacity of drives now, it’s common for people to have drives with, say only 1/3 or 1/4 of the space actually used. And with that much empty available space, and the system trying to write to sectors that haven’t been used recently, that just deleted file may sit there, intact but un-indexed, for a long time until that space is over-written with a new file. It could easily be years.

5

Just so we’re clear : if I select a file and press the Delete key, I may get a prompt asking “Are you sure you want to send Kangaroo.jpg to the Recycle Bin ?” . If I click on Yes, the file will be kept intact on disk, but its directory entry will be moved to the Recycle Bin area of the disk. I can open the Recycle Bin tool in 3 years and get the file back, intact, with its original file name in its original directory. No other file will overwrite its contents by random chance: it’s still a file whose integrity is protected by the file system.

The file will get removed from the Recycle Bin if:

  • I go to the Recycle Bin tool and choose to erase the file permanently (or to empty the Recycle Bin entirely).
  • The disk space taken up by the Recycle Bin entries exceeds a certain fraction of the size of the disk, at which point Windows may erase files from the Recycle Bin at random (I think).

At that point, the file is deleted : its name is forgotten and the disk sectors containing the image become available for other uses, and eventually overwritten.

If I want to delete Giraffe.jpg without the Recycle Bin purgatory, I can press Shift-Delete instead. I get a different prompt to approve, and the file will be deleted “for real” right away. Its sectors will get overwritten by other files eventually.

In both cases, once the file is deleted, there’s no way to tell when its sectors will get overwritten. Sure, there’s always another file getting written by the OS or some other application, but it will use the “next available” sectors based on the filesystem’s algorithm. If that file I’ve just deleted weighed 200 MB, most of its sectors can remain intact on disk for a long time.

ETA : This was a discussion about a non-removable, rotating hard disk. For removable drives, I think Windows doesn’t maintain a Recycle Bin at all. And a SSD drive may have its own policies about overwriting free sectors, independently from Windows itself, to even out the number of times a given “sector” of the SSD is written.

6

SSD’s complicate things because of TRIM/garbage collection. The SSD controller knows which blocks have data on them, even if it doesn’t know what that data is. Spinning hard drives aren’t like that. When a file is deleted by the operating system (Windows, macOS, etc.), its entry in the drive catalog (think of it as an index) is removed, so the drive will just blithely write over any old de-cataloged data. As others have said, this can take seconds, minutes, days, or years.

While a spinning hard drive can just write over old data, an SSD has to erase/reset those sectors at the lowest level before the new data can be written on them. This has a significant speed penalty, hence TRIM/garbage collection. So when the operating system deletes files, it also marks them in such a way that the SSD controller understands they’re old/junk data and may be disposed of. Then when the SSD controller goes through its garbage collection routine, it erases those sectors while nothing else is going on so they’re ready to be written without needing to wait for them to be erased first. This may only happen daily, or some data may not get properly marked so it can also sit there for an extended period.

7

I think this is the closest to what I had in mind. You empty the recycle bin, Windows marks the file as deleted and returns the disk storage units to the “available” pool. I agree as well that there is no way to know how long the actual data will remain. The key phrase used by Heracles is “it will use the ‘next available’ sectors based on the filesystem’s algorithm.”

I contend that in a normal Windows environment, with not only the O.S. doing stuff, there are user applications also doing stuff. This signals to me that recently freed storage units (sectors) will be reused quickly.

I’ll concede that if the O.S. is being intelligent about fragmentation those recently freed sectors might remain free for a long time.

8

I’m not following the logic here. While I don’t know the algorithm that Windows uses, it generally writes to the largest available free chunk to avoid file fragmentation. If you free up a small chunk in the middle somewhere, that chunk is going to be pretty far down on the list of desirable space.

There is also a performance issue. Track to track head seeks take a lot of time comparatively (well, on physical spinny drives at least), so Windows is going to write things in such a way as to minimize head movement. SSD drives don’t have a physical head to move around but following this type of algorithm helps to load level the writes to different parts of the disk so that ends up working well for SSD drives.

Recently freed sectors aren’t likely to be in large blocks and aren’t likely to be next to other free sectors, so it seems to me that they aren’t likely at all to be reused quickly.

9

How much of that stuff involves writing to disk?

10

Here’s what I remember from decades gone by:

If you delete a file, it is “moved” to the “recycle bin”.

A directory (or folder) entry is a record in the directory containing the metadata of the file - name, size, date written, updated, start sector (track, block, platter) of the file.
Each sector on the disk contains the data for the file, and if there is more data, a pointer to the next sector in the file. The last sector contains an “End of file” marker.
(This is another source of insecurity, In the good old days, the sector was only written to the end-of-file marker, and there could be data from a previous file in the remainder of that sector; if say sectors are 10K and your file ends at 3K, there’s 7K that could be a fragment from a previous erased file)

At one time, in FAT file systems, and DOS, deleting a file simply put a “~” in the first character of the filename, indicating the entry in the folder was now free to be reused.

(A folder is essentially just another file - a chain of sectors containing filename entries. If the current sector is full, extend the folder chain to a new sector to add more files. this is also the advantage of the “~” marking an unused folder entry; the next file you write in that folder will use the next empty space, so the directory does not grow uncontrollably as you delete and add. )

there is also a sector table, indicating which sectors are actually free - ie. not part of a directory or file.

So a corresponding entry for the “deleted” file is created in the directory called “Recycle Bin”, pointing to the first sector of the file data, plus metadata of where the file was erased from, in case you want it restored. The chain of data remains intact. Your file is recoverable, but from what you can see from Windows, it is “free space”.

Should the disk run out of actual free space, there is an Operating System algorithm to decide which files in the recycle bin get used in what order. (oldest first?)

Otherwise, the files are there until you “Empty Recycle Bin”. Then, the entry in the Recycle Bin folder is now marked free to be reused; the data chain for the file is marked as free space in the sector table. When the data disappears (answer to OP) depends on how much data is written to the disk, how often and how fast.

The operating system does not necessarily overwrite any data automatically. Worst case, you “erase all” (DEL . ) and in the root of the disk system all files are marked as deleted, all sectors marked as free. If nothing further is written to the disk, an enterprising data recovery tech can get almost anything back.

(This is based on FAT technology. Not sure what refinements NTFS or assorted Linux filesystems have come up with. Computers are sufficiently faster and have so much more space than the Good Old Days so OS’s are often tweaked for speed and security.)

A few notes -

Writing a new file involves putting the necessary entry (or updating the entry) in the directory. Then the blocks of data are written in sequence; as each new block needs to be written, an OS algorithm decides with free sector to add to the new chain of data.

Except for databases (with pre-allocated data space, usually) a lot of “File - Save” usually involves writing a new file, delete the old then renaming the new file. This way if the write fails, the old file is till intact. So it involves writing extra data.

A USB stick (AFAIK) works the same, except sometimes the recycle bin is on the stick too, so the whole system is self-contained. If there is no recycle bin, then deleted files go straight to sectors marked “free” and directory entries marked as deleted. But again, deleted data is still actually on the disk.

Some security options in filesystems will erase data when deleted, but generally they don’t. no need to.

SSD (and sometimes, USB sticks - nonvolatile RAM in these devices have a very large but finite number of write cycles before they stop working, so the hardware includes algorithms to allocate writes to least use sectors so no sector is overused with writes. Again, this helps preserve sectors full of deleted data, and auto-erasing data sectors simply halves the lifetime of the device, so less common. Note your phone also contains NVRAM.

I don’t know how recoverable data is if, say, a recovery program finds a few framents of a file but not the whole thing. Text files - obviously. Word files and email files are usually text with plenty of extra control data. Spreadsheets - usually, you can decode a series of cells. I have no idea how recoverable a chunk of a JPG is. It depends how important this is to whoever is looking. the point is, each sector that is not blanked out point to the next in the chain, so a decent sized fragment chain can be identified as a single file for analysis.

This is why there are eraser programs that go through and erase all empty sectors and end-of-file space. Some will also clean deleted directory entries (Since file name, size, and date may be sensitive info too)

the first step in forensic analysis is to make a sector by sector mirror of a disk so it can be analyzed without destroying the original

11

Even more observations -

There are processes that use empty space (i.e. shadow copies, so you can recover previous versions of files) This actually uses disk space, the OS just pretends it’s not there. If your computer has been running a while, it has probably already pre-allocated and used its full quote.

A lot of the “stuff running in the background” is doing low disk use work - writing logs, capturing performance data, system data (i.e. when you close a program, often it remembers the location and size of the window on the screen; last used file for that program, etc. etc. But these are very small. The simple rule of thumb would be - don’t expect your fully deleted file to still be there, but don’t rely on IS activity to overwrite it.

Memory paging, where RAM is freed up from processes that don’t really need it any more - can be a lot of disk IO, but usually is confined to a specific pre-allocated file area. (Pagefile.sys) Unless your pagefile is set to auto-expand, it won’t grab more disk space. If it does, it also has an upper limit and usually is at it already.

12

Let’s start with this because I agree with it. So we end up with a few sectors here and there. Windows won’t use them because Windows is looking for bigger contiguous storage chunks. The actual data will remain intact.

Granted that most user interactive applications are not disk intensive until a file is saved or closed. But see the second quote from md-2000 below.

Totally agree with this, and I contend that the Windows O.S. does a lot of disc writes in the background often

This is the bullseye response to me. “A lot of the “stuff running in the background” is doing low disk use work - writing logs, capturing performance data, system data…”

Low disc work is exactly the kind of data that the O.S. can fit into those few and far between sectors. Thus I continue to contend that in a normal sort of Windows usage environment the data in a “empty the trash” deleted file is overwritten quickly.

13

That doesn’t make sense. Drives are big these days. If you delete a few hundred gigabytes of stuff, almost all of that is going to stick around unless you intentionally fill the drive back up again. The logging and other stuff only uses a minuscule portion of that.

If you only delete a few small files, it depends on where they are on the disk. Maybe you get lucky and they are overwritten quickly, but that’s not at all guaranteed.

14

This is a minor point but I think worth making. On my workstation a cluster is the smallest data storage unit. A cluster on my workstation is 16k bytes. I cringe every time I save a 1000 character file, knowing how much disc space is being dedicated but unused. A lot of us are accustomed to audio, video and still photgraph files. They ARE big. But my e-mails, personal notes, correspondence are trifling and almost never fill a single cluster.

15

There are software utilities that clears empty sectors on a disk.

I’ve used CCleaner for over 20 years. It’s main purpose is to find and remove cache and temp files. Does a great job clearing the web browser cache.

There’s a menu option to write zeroes to unused disk sectors. It can take a long time on a 1TB drive.

16

Dr Strangelove I agree that if you delete gigabytes of data most of it will remain until replaced with gigabytes more. I can’t argue with your scenario.

17

That was a lot worse in the old days of FAT16, where you got 65,536 clusters total. I remember getting a 2 GB drive (huge at the time), which gave me 32k clusters. For significant portions of my data, it was no upgrade at all because most of the files were well under the cluster size.

18

Is this true? I often have some non-saved text file or another open on my computer (OSX), and even when the computer crashes or is forced to reboot, those text files are usually still open, with all of their data intact. The only way that could be true would be if the application and/or the OS are regularly saving the contents of those windows to disk, in some sort of temporary file at least.

19

It depends entirely on the program. Text files are tiny and it’s no effort for the program to regularly save them to disk (though they’ll only bother when the contents change). For, say, a giant video file? Much less likely (though in a video editor it’s likely just saving the instructions needed to recreate the file).

20

Modern file systems can do tail packing, where multiple small files, or the “tails” of larger files, are saved together in a single cluster to avoid this problem.