# Does Windows pull copying speed estimates out of Bill Gate's ass, or what? **URL:** \ **Category:** Factual Questions\ **Created:** [May 10, 2012, 3:04am UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468 "2012-05-10T03:04:04Z")\ **Posts on this page:** 18\ **Page:** 2
**Author:** ![control-z](https://avatars.discourse-cdn.com/v4/letter/c/eada6e/32.png) [@control-z](https://boards.straightdope.com/u/control-z)\ **Post date:** [May 10, 2012, 3:39pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/21 "2012-05-10T15:39:41Z")
The Vista algorithm was the worst. When you’d start a drag-and-drop copy of a large number of files, it would sit there and calculate for many minutes before it would actually do anything. I usually gave up and started a command prompt and used XCOPY. I see that behavior in W7 a little, but not as bad. ---
**Author:** ![DWMarch](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dwmarch/32/1010_2.png) [@DWMarch](https://boards.straightdope.com/u/DWMarch)\ **Post date:** [May 10, 2012, 6:01pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/22 "2012-05-10T18:01:42Z")
Windows 8 reworks this whole setup. [http://blogs.msdn.com/b/b8/archive/2011/08/23/improving-our-file-management-basics-copy-move-rename-and-delete.aspx](http://blogs.msdn.com/b/b8/archive/2011/08/23/improving-our-file-management-basics-copy-move-rename-and-delete.aspx) ---
**Author:** ![iamthewalrus\_3](https://avatars.discourse-cdn.com/v4/letter/i/258eb7/32.png) [@iamthewalrus\_3](https://boards.straightdope.com/u/iamthewalrus_3)\ **Post date:** [May 10, 2012, 6:03pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/23 "2012-05-10T18:03:56Z")
> [@Postariti](#): > > No factual answer, but I’ve noticed speed seems to vary with the size of the files. If I remember, copying lots of small files is slower than copying large files. Phenomenally slower. Like, 100s of times slower to copy files that occupy a single filesystem block than to copy one big file that spans many. > [@Francis\_Vaughan](#): > > The algorithm used seems to use a moving average which uses a very poorly chosen (too small) window. So simple purtubations in the progress (for instance due to disk seeking, operating system activity) are amplified. This is what’s going on. Imagine that you’re in stop-and-go traffic and you’re trying to estimate how long it’s going to take you to get somewhere with no memory of what came before. You get moving, and think “Sweet, we’re going 40mph and still accelerating! We’re going to be there in 20 minutes!”. They you have to slow down: “Oh, crap, now we’re all inching along. It’s going to be 17 hours”. Generally, when copying a big bunch of files, Windows is going to copy them in something approaching alphabetical order. Hit some big files, and it’s going to give you a short estimate. Hit some little ones, and it’s going to tell you it’ll take forever. ---
**Author:** ![Ruminator](https://avatars.discourse-cdn.com/v4/letter/r/b9bd4f/32.png) [@Ruminator](https://boards.straightdope.com/u/Ruminator)\ **Post date:** [May 10, 2012, 6:34pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/24 "2012-05-10T18:34:55Z")
> [@Boyo\_Jim](#): > > I’m copying a rather large amount of data (10,000 files, about 38 GB) The main issue is that you have lots of individual files (more than a few hundred.) What Windows does (at a high level) is for **each file** , there are 2 steps: it copies the **directory entry** for that one file (analogous to a Table-of-Contents entry), _and_ it also copies all the associated data (the actual megabytes of data). These are in 2 separate areas of the media. If you had were copying to a spinning harddrive, you could actually hear the arm flapping back and forth between the inside and outside of the disk platters as it switches from writing directory entries to the actual data. As for a USB flash drive, it’s silent so you can’t hear the thrashing of mechanical parts but the principles of slowness are the same – the USB flash has to constantly rewrite the same area where the directory entries reside For discussion purposes… if you had just 1 file that was 38GB in size, the copy operation would run much master. Anyways, Windows is very good at estimating the writing of data blocks (1 file of 38GB) but not at the writing of thousands of directory entries (10000 files totaling 38GB). > [@](#): > > Oh, one other thing. I defragged my disk before starting in the hopes that it might speed the process up – not having to jump all around the drive and all. All this data is contained within one folder and a set of subfolders. Defarg won’t help this type of thing. If it’s not important that all 10,000 files must be copied as 10,000 for individual access, you can zip them **into 1 file** first, and then copy that zip file. Less “thrashing” that way. But then again, you’ve moved to the “slowness of 10,000” files down the chain: whoever opens that zip file will wait a little while as the zip utility “scans” all 10,000 entries before showing any of them so he can extract that one file. 🙂 ---
**Author:** ![Boyo\_Jim](https://avatars.discourse-cdn.com/v4/letter/b/87869e/32.png) [@Boyo\_Jim](https://boards.straightdope.com/u/Boyo_Jim)\ **Post date:** [May 10, 2012, 7:18pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/25 "2012-05-10T19:18:05Z")
> [@Duckster](#): > > Use this —\> [TeraCopy for Windows - Code Sector](http://codesector.com/teracopy) > [@LionelHutz405](#): > > I second the recommendation to use TeraCopy. > > I find it interesting that Microsoft chose to display an inaccurate time estimate rather than display the percentage complete which is reasonable feedback for a job you simply can’t know ahead of time how long it will take. I decided to test test TeraCopy against the standard Windows copy command. The data consisted of 9,517 files in 1,990 folders. I wiped the flash drive clean by reformatting it before each test. The time difference was statistically significant (I think), but not very substantial. TeraCopy 150 minutes to do the job, and Windows Copy took 159 minutes. So it’s about 6% faster. Also, I could tell by reading the file names as they came up that the two programs were copying the files in exactly the same order as each other, which corresponded to the order they appeared in the directory tree. Or at least, it appeared that way from the display I was seeing. These are all music files, the .mp3 are pretty consistently sized from 2-3 megs, and there are a substantial number of AAC files that are probably between 10 and 14 megs. Each folder was an artist, and contained all the music I had by that artist. ---
**Author:** ![Dewey\_Finn](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dewey_finn/32/4222_2.png) [@Dewey\_Finn](https://boards.straightdope.com/u/Dewey_Finn)\ **Post date:** [May 10, 2012, 7:31pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/26 "2012-05-10T19:31:37Z")
I haven’t used Teracopy but based on the description at that website, it does appear to resolve one of the more annoying problems with Windows file copies. That is, what happens when there is an error. Windows will just abort the whole copy process, without any indication of how far it’s gotten. (So I have no idea of what’s been copied and what hasn’t.) Teracopy, according to the website, will just skip the file in question. ---
**Author:** ![mhendo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mhendo/32/3159_2.png) [@mhendo](https://boards.straightdope.com/u/mhendo)\ **Post date:** [May 10, 2012, 7:38pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/27 "2012-05-10T19:38:58Z")
> [@LionelHutz405](#): > > I find it interesting that Microsoft chose to display an inaccurate time estimate rather than display the percentage complete which is reasonable feedback for a job you simply can’t know ahead of time how long it will take. But how would that change the practical problem faced by the user who wants some sort of idea of how the transfer is progressing? After all, what is your “percentage complete” number going to be based on? Number of files, or megabytes of data? If it’s based on number of files, it could be wildly inaccurate if you have files of varying sizes, from very small to very large. Similarly, if it’s based on the amount of data copied, that won’t help very much if the copying of a few large files is considerably faster than the copying of lots of small files. ---
**Author:** ![mhendo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mhendo/32/3159_2.png) [@mhendo](https://boards.straightdope.com/u/mhendo)\ **Post date:** [May 10, 2012, 7:40pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/28 "2012-05-10T19:40:43Z")
> [@Ruminator](#): > > But then again, you’ve moved to the “slowness of 10,000” files down the chain: whoever opens that zip file will wait a little while as the zip utility “scans” all 10,000 entries before showing any of them so he can extract that one file. 🙂 Not only that, but zipping 10,000 files in the first place takes a bit of time, even if you don’t choose any compression and just pack them as uncompressed files. If you want to minimize the size of the ZIP file, and choose the compression type required to do this, it will take even longer. ---
**Author:** ![Ruminator](https://avatars.discourse-cdn.com/v4/letter/r/b9bd4f/32.png) [@Ruminator](https://boards.straightdope.com/u/Ruminator)\ **Post date:** [May 10, 2012, 8:26pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/29 "2012-05-10T20:26:48Z")
> [@mhendo](#): > > Not only that, but zipping 10,000 files in the first place takes a bit of time, even if you don’t choose any compression and just pack them as uncompressed files. This is correct but the “penalty” depends on the specific use case. For example, I have an old client folder with 100,000 files in it. It is on a harddrive that is backed up every day to multiple external drives (and sometimes the cloud). Archiving it all into one big zip file saves enormous time on _all subsequent_ copies and synchronizations. Even if you’re using a “smart” copy utility that avoids copying the same files repeatedly, the sync utility still has to _compare_ 100,000 files between the 2 devices and that takes a long time. Multiply by 10 clients each with 100k files and I’m drowning in 1 million files instead of just 10 big zip files. The OP is copying MP3s and he needs each file to be visible at the OS level so my rationale won’t work for his situation. ---
**Author:** ![Blakeyrat](https://avatars.discourse-cdn.com/v4/letter/b/ecd19e/32.png) [@Blakeyrat](https://boards.straightdope.com/u/Blakeyrat)\ **Post date:** [May 10, 2012, 8:38pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/30 "2012-05-10T20:38:24Z")
> [@mhendo](#): > > Not only that, but zipping 10,000 files in the first place takes a bit of time, even if you don’t choose any compression and just pack them as uncompressed files. If you want to minimize the size of the ZIP file, and choose the compression type required to do this, it will take even longer. On any modern computer, the CPU is so many thousands of times faster than the storage mechanism, that ZIP compression time is (for all practical purposes) zero. There’s no point to putting files in a .zip with zero compression. Your CPU can compress all the needed data in a small fraction of the time it sits there grumbling, “come onnnnn! I’m ready to copy more data!!! hurry up!” to your disk. Even for a Flash drive. Hell, even for RAM. ---
**Author:** ![LionelHutz405](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lionelhutz405/32/536_2.png) [@LionelHutz405](https://boards.straightdope.com/u/LionelHutz405)\ **Post date:** [May 10, 2012, 9:09pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/31 "2012-05-10T21:09:37Z")
> [@mhendo](#): > > But how would that change the practical problem faced by the user who wants some sort of idea of how the transfer is progressing? > > After all, what is your “percentage complete” number going to be based on? Number of files, or megabytes of data? > > If it’s based on number of files, it could be wildly inaccurate if you have files of varying sizes, from very small to very large. Similarly, if it’s based on the amount of data copied, that won’t help very much if the copying of a few large files is considerably faster than the copying of lots of small files. Either would work. The percentage complete is just an indicator that the task is running and not hung up. It’s accurate because it tells you how much of the task is complete. It has nothing to do with time which, as we’ve covered, is not possible to calculate accurately. ---
**Author:** ![dracoi](https://avatars.discourse-cdn.com/v4/letter/d/90db22/32.png) [@dracoi](https://boards.straightdope.com/u/dracoi)\ **Post date:** [May 11, 2012, 5:23am UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/32 "2012-05-11T05:23:57Z")
> [@LionelHutz405](#): > > Either would work. The percentage complete is just an indicator that the task is running and not hung up. It’s accurate because it tells you how much of the task is complete. It has nothing to do with time which, as we’ve covered, is not possible to calculate accurately. Just as long as the percentage don’t work like QuickBook’s updaters. If you’ve ever watched those updates install, you know that you can take 2 minutes to go from 0% to 100%… and then it will sit for 2 minutes at 100% with no explanation of what’s happening. That window will disappear… to be replaced by a new progress bar for update step 2, starting at 0%. So even with percentages, computers can still provide confusing/meaningless information for the user. ---
**Author:** ![madel](https://avatars.discourse-cdn.com/v4/letter/m/97f17d/32.png) [@madel](https://boards.straightdope.com/u/madel)\ **Post date:** [April 20, 2014, 3:23pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/33 "2014-04-20T15:23:20Z")
One other thing to consider: On my older Asus computer I sometimes forget and use the USB 1 port. As soon as I see that the transfer will take two hours I quickly abort and use the USB 2 port. (Why they put both on my computer I’ll never know.) I’ve never timed it but the USB 2 port is at least 10 times faster. (I’ve read that USB 2 can be up to 40x faster.) This is probably not an issue with newer Win 7 computers but I’m posting for others who might be using older computers. (However, if your computer has both USB 2 and USB 3 ports use the ‘3’ port.) ---
**Author:** ![beowulff](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/beowulff/32/542_2.png) [@beowulff](https://boards.straightdope.com/u/beowulff)\ **Post date:** [April 20, 2014, 3:40pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/34 "2014-04-20T15:40:02Z")
The copy has been going for _2 years?_ Now, _that’s slow!_ ---
**Author:** ![Claverhouse](https://avatars.discourse-cdn.com/v4/letter/c/13edae/32.png) [@Claverhouse](https://boards.straightdope.com/u/Claverhouse)\ **Post date:** [April 20, 2014, 3:43pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/35 "2014-04-20T15:43:55Z")
Just use Grsync. ---
**Author:** ![Duke\_of\_York](https://avatars.discourse-cdn.com/v4/letter/d/f6c823/32.png) [@Duke\_of\_York](https://boards.straightdope.com/u/Duke_of_York)\ **Post date:** [April 21, 2014, 12:48am UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/36 "2014-04-21T00:48:13Z")
I found out a few weeks ago that copying large files to a flash drive was much slower than copying them to a Class 10 SD Card. I place the SD Card into a USB adapter to do this. ---
**Author:** ![Doctor\_Jackson](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/doctor_jackson/32/32_2.png) [@Doctor\_Jackson](https://boards.straightdope.com/u/Doctor_Jackson)\ **Post date:** [April 21, 2014, 8:22pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/37 "2014-04-21T20:22:46Z")
> [@Boyo\_Jim](#): > > Does Windows pull copying speed estimates out of Bill Gate’s ass, or what? Yes, they do in fact. The process is called BGColonCleanse. The ratio of impactation (ROI) can skew progress bar results. ---
**Author:** ![KarlGauss](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/karlgauss/32/3713_2.png) [@KarlGauss](https://boards.straightdope.com/u/KarlGauss)\ **Post date:** [April 21, 2014, 8:45pm UTC](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468/38 "2014-04-21T20:45:29Z")
> [@Francis\_Vaughan](#): > > The obligatory [XKCD](http://xkcd.com/612/). Now _that_ was worth a two year wait. Hilarious! [Previous page](https://boards.straightdope.com/t/does-windows-pull-copying-speed-estimates-out-of-bill-gates-ass-or-what/621468.md?page=1)