I was using a USB drive and went to delete some files on it and it is taking well over 30 minutes. The system (Windows 10) was deleting one file at a time. In fact it measured its speed as files deleted/second which averaged around 30/s (give or take). When deleting over 100,000 files it takes a while.
This seem incredibly inefficient. I thought the way a system deleted files was to just tell the file allocation table (or the equivalent used today) that the space on the drive was free. That should only take moments.
I am not using safe erase or anything so why the slow, file-by-file delete?
Doing anything with a USB drive (copy files onto it, copy files off of it, delete files, etc) takes a zillion times longer than doing something to files on your local hard drive.
I do think Windows is pokier at deleting files in general. It’s just not as noticeable until / unless you’re doing it to files on a USB drive or some other slow-ass equivalent.
ETA: the file allocation table in question is the one on the USB drive itself. Otherwise the files would still show up if you inserted the stick into someone else’s USB port. So it has to go find the entry for each file. Then modify that entry to de-allocate the space and remove it as a visible / accessible file.
I’m just repeating what I’ve heard … but the slow down is driving the cute little graphic of the pages floating through the air or something … that takes a lot of computational power … the actual deleting is very quick and easy …
Your system is having to call up the file name, display it on the screen, calculate the remaining sectors, display THAT … then go to the next file … but yeah, the file isn’t even erased, just the sector is listed as available to use …
My suspicion is that you’re right, but there’s also a lot of bind-up possible in the speed of the medium and its connection.
No stopwatch test done, but I recall that deleting a bunch of files from the command prompt was quicker to finish than the same operation by the Explorer UI.
OTOH, if the I/O operations to delete a file have to be done over a slow connection (old USB, or network), latency piles up and the deletion itself takes longer per transaction.
I suspect it’s in large part due to the way flash memory devices work. Because flash memory can wear out if it’s used a lot, there’s a low-level controller on the thumb drive that goes to a lot of effort to try and balance the use of all the different sections of memory on the drive – essentially there’s a tiny, slow, computer in the thumb drive that’s doing a lot of work every time the main computer adds or deletes things from the drive. I imagine it’s actually the controller inside the thumb drive that’s slowing things down, not Windows putting up file names on the screen.
If it were Windows doing the cute display of pages flipping past on the screen, it would be slow to delete a similar folder’s worth of files from your local hard disk.
I believe it’s also true that flash memory can only be written a block at a time - so you can’t just surgically change a few specific bits - you have to read the block into memory, make the changes, then write the whole block back.
I’m not sure who told you that but I’d take it with a grain of salt. Not least of which because that animation hasn’t even existed on Windows in something like 10 years… if you’re seeing it in 2018, you badly need to upgrade.
The real problem here is that most USB drives are slow as snot at doing random-access operations. EDIT: probably for the reason Quercus mentioned.
Much of this is because of stuff added to Windows (& hardware) to satisfy user desires or compensate for users dumb mistakes.
Before deleting files, Windows copies them to the Recycle Bin (so they can be recovered when you realize that you made a dumb mistake & deleted some valuable file). This copy can take longer than the actual deletion. (You can skip this – Ctrl-delete I think.)
Because users got impatient/concerned when waiting for files to be deleted, Windows added various fancy displays of pages flipping or percentage bars filling up across the screen or file names being listed – all this takes time away from actually deleting files, just to show you pretty pictures.
Hardware (especially USB drives) has various safety provisions to deal safely with a user unplugging it in the middle of a file operation, or to compensate for users constantly writing/rewriting files on the drive (load leveling), etc. All these provisions add a bit of time to file operations. Not much, but when dealing with 100,000 files, it adds up.
Note that you can avoid much of this by opening up a DOS command window and issuing a command like
This will be much faster.
But that is expert-level: you better know what you’re doing. No pretty pictures while deleting, and no Recycle option – the files are just gone. If you made a dumb mistake and deleted too many files, better hope you have a current backup.
A normal delete does not “erase” the file. Nor does deleting into the recycle bin “copy” it there. A standard delete leaves the file’s data right where it is on the disk and then removes the file’s entry from the centralized index of files. That’s why recovery utilities can work. Recycling a file to the bin is no different than renaming it to a new location on (the same) disk, it is a change to the index only. There are delete utilities that erase a file by overwriting each binary bit with zeros, sometimes in multiple passes. Those truly delete the file and take much longer.
If you look around, (or google and stuff), you will be able to find the place where you can enable write-caching for the drive (may depend on OS version). That will speed it up a bit.
If the write caching converts all the single write transactions into larger block transactions, it will speed it up a lot - I don’t know if it does that.
The procedure for writing to flash is much like the procedure for writing to a hard drive: you have to get the heads in place before you can start writing In the case of flash, you have to notify the flash in advance which row you want to read, so it can queue it up – this is why sequential reads are faster than random reads.
Also each write cycle starts by erasing the block, which slows things down a bit (or speads them up, depending on how you consider it: the special feature of “flash” memory is that you can erase a whole block before writing, rather than having to do each individual cell. That’s only an advantage when you plan to write blocks).
The part of the GUI that takes up the most time is identifying the size of every file so it can show you a percentage. People used to yell at how imprecise the estimates were, so Microsoft made it better. On a USB flash drive, that takes a whole lot longer.
I’m not sure about the Recycle Bin issue, as that should just be stored on the USB drive. At least, that’s how it works on hard drives. The whole point of the Recycle Bin is that the file is not actually deleted, but left where it was, but the name gets changed to be in the Recycle Bin. Then, when needed, the Recycle Bin will remove the name, which means the file can be overwritten.
Granted, flash memory is a little weird, in that blocks have to be blanked out in order for files to be written. Maybe that’s done ahead of time? Seems a bad idea.
Windows enables write caching on internal hard drives/SSDs. What this means is it stores/caches pending write operations to a disk in system memory, and executes them later as disk activity allows. It gives the appearance of having written the files quickly, but they may not actually be committed to disk until later.
USB drives are flagged as “removable” by Windows, which disables write caching on them. reason being, if you copy or save a bunch of data to a USB drive, if the writes were cached and you unplug the drive before they can be committed to the disk, that data will never make it to the drive. If you do it while Windows is flushing the cache to disk, the files you copied might even be corrupted. So, with a removable drive Windows will write (or erase) the files as you issue the command, and you have to wait until it is finished.
second, you’re talking about deleting 100,000 files. as you said, “deleting” a file removes it from the table, but IIRC Windows also changes the first few bytes of the file to mark it as a “deleted file.” It has to do that for each file you want deleted, and each of those requires a certain number of write I/O operations* (IOPs as gnoitall mentions.) USB 2.0 is reasonably fast when you’re copying a bulk data stream e.g. a 1 GB video file, but its latency is high when you’re doing anything with a bunch of small files. all of those IOPs just pile up and it takes time to work through them.
doing anything with lots of small files over USB just plain sucks, period. not much you can do about it. And enabling write caching won’t help; all that means is that you must use the “Safely Remove Hardware” function before disconnecting it, and 2) you’re going to be waiting a while to “Safely Remove” it while it’s flushing all of those pending writes to the drive.
Either way, you’re gonna wait.
(* as an aside, this is one reason internal SSDs have become so wicked fast. not only do they distribute writes in parallel over lots of flash memory chips, they have extremely fast controllers which can handle hundreds of thousands of I/O operations per second. Even the best, fastest spinny disk hard drives can only handle at most a couple hundred IOPS largely due to their mechanical and rotational latency.)