The Vista algorithm was the worst. When you’d start a drag-and-drop copy of a large number of files, it would sit there and calculate for many minutes before it would actually do anything. I usually gave up and started a command prompt and used XCOPY.
I see that behavior in W7 a little, but not as bad.
Windows 8 reworks this whole setup.
Phenomenally slower. Like, 100s of times slower to copy files that occupy a single filesystem block than to copy one big file that spans many.
This is what’s going on.
Imagine that you’re in stop-and-go traffic and you’re trying to estimate how long it’s going to take you to get somewhere with no memory of what came before. You get moving, and think “Sweet, we’re going 40mph and still accelerating! We’re going to be there in 20 minutes!”. They you have to slow down: “Oh, crap, now we’re all inching along. It’s going to be 17 hours”.
Generally, when copying a big bunch of files, Windows is going to copy them in something approaching alphabetical order. Hit some big files, and it’s going to give you a short estimate. Hit some little ones, and it’s going to tell you it’ll take forever.
The main issue is that you have lots of individual files (more than a few hundred.)
What Windows does (at a high level) is for each file, there are 2 steps: it copies the directory entry for that one file (analogous to a Table-of-Contents entry), and it also copies all the associated data (the actual megabytes of data). These are in 2 separate areas of the media. If you had were copying to a spinning harddrive, you could actually hear the arm flapping back and forth between the inside and outside of the disk platters as it switches from writing directory entries to the actual data. As for a USB flash drive, it’s silent so you can’t hear the thrashing of mechanical parts but the principles of slowness are the same – the USB flash has to constantly rewrite the same area where the directory entries reside
For discussion purposes… if you had just 1 file that was 38GB in size, the copy operation would run much master.
Anyways, Windows is very good at estimating the writing of data blocks (1 file of 38GB) but not at the writing of thousands of directory entries (10000 files totaling 38GB).
Defarg won’t help this type of thing. If it’s not important that all 10,000 files must be copied as 10,000 for individual access, you can zip them into 1 file first, and then copy that zip file. Less “thrashing” that way.
But then again, you’ve moved to the “slowness of 10,000” files down the chain: whoever opens that zip file will wait a little while as the zip utility “scans” all 10,000 entries before showing any of them so he can extract that one file. 
I decided to test test TeraCopy against the standard Windows copy command. The data consisted of 9,517 files in 1,990 folders. I wiped the flash drive clean by reformatting it before each test.
The time difference was statistically significant (I think), but not very substantial. TeraCopy 150 minutes to do the job, and Windows Copy took 159 minutes. So it’s about 6% faster. Also, I could tell by reading the file names as they came up that the two programs were copying the files in exactly the same order as each other, which corresponded to the order they appeared in the directory tree. Or at least, it appeared that way from the display I was seeing.
These are all music files, the .mp3 are pretty consistently sized from 2-3 megs, and there are a substantial number of AAC files that are probably between 10 and 14 megs. Each folder was an artist, and contained all the music I had by that artist.
I haven’t used Teracopy but based on the description at that website, it does appear to resolve one of the more annoying problems with Windows file copies. That is, what happens when there is an error. Windows will just abort the whole copy process, without any indication of how far it’s gotten. (So I have no idea of what’s been copied and what hasn’t.) Teracopy, according to the website, will just skip the file in question.
But how would that change the practical problem faced by the user who wants some sort of idea of how the transfer is progressing?
After all, what is your “percentage complete” number going to be based on? Number of files, or megabytes of data?
If it’s based on number of files, it could be wildly inaccurate if you have files of varying sizes, from very small to very large. Similarly, if it’s based on the amount of data copied, that won’t help very much if the copying of a few large files is considerably faster than the copying of lots of small files.
Not only that, but zipping 10,000 files in the first place takes a bit of time, even if you don’t choose any compression and just pack them as uncompressed files. If you want to minimize the size of the ZIP file, and choose the compression type required to do this, it will take even longer.
This is correct but the “penalty” depends on the specific use case.
For example, I have an old client folder with 100,000 files in it. It is on a harddrive that is backed up every day to multiple external drives (and sometimes the cloud). Archiving it all into one big zip file saves enormous time on all subsequent copies and synchronizations. Even if you’re using a “smart” copy utility that avoids copying the same files repeatedly, the sync utility still has to compare 100,000 files between the 2 devices and that takes a long time. Multiply by 10 clients each with 100k files and I’m drowning in 1 million files instead of just 10 big zip files.
The OP is copying MP3s and he needs each file to be visible at the OS level so my rationale won’t work for his situation.
On any modern computer, the CPU is so many thousands of times faster than the storage mechanism, that ZIP compression time is (for all practical purposes) zero. There’s no point to putting files in a .zip with zero compression.
Your CPU can compress all the needed data in a small fraction of the time it sits there grumbling, “come onnnnn! I’m ready to copy more data!!! hurry up!” to your disk. Even for a Flash drive. Hell, even for RAM.
Either would work. The percentage complete is just an indicator that the task is running and not hung up. It’s accurate because it tells you how much of the task is complete. It has nothing to do with time which, as we’ve covered, is not possible to calculate accurately.
Just as long as the percentage don’t work like QuickBook’s updaters. If you’ve ever watched those updates install, you know that you can take 2 minutes to go from 0% to 100%… and then it will sit for 2 minutes at 100% with no explanation of what’s happening. That window will disappear… to be replaced by a new progress bar for update step 2, starting at 0%.
So even with percentages, computers can still provide confusing/meaningless information for the user.
One other thing to consider: On my older Asus computer I sometimes forget and use the USB 1 port. As soon as I see that the transfer will take two hours I quickly abort and use the USB 2 port. (Why they put both on my computer I’ll never know.)
I’ve never timed it but the USB 2 port is at least 10 times faster. (I’ve read that USB 2 can be up to 40x faster.)
This is probably not an issue with newer Win 7 computers but I’m posting for others who might be using older computers. (However, if your computer has both USB 2 and USB 3 ports use the ‘3’ port.)
The copy has been going for 2 years?
Now, that’s slow!
Just use Grsync.
I found out a few weeks ago that copying large files to a flash drive was much slower than copying them to a Class 10 SD Card. I place the SD Card into a USB adapter to do this.
Yes, they do in fact. The process is called BGColonCleanse. The ratio of impactation (ROI) can skew progress bar results.
Now that was worth a two year wait. Hilarious!