I have five hard drives in my PC, four IDEs and a SATA.
One of the IDEs is inordinately slow in showing its contents in Windows Explorer, and always has been.
For testing purposes, I compared it against the drive to which it’s most similar. The problem drive is a Western Digital 1600JB, a 160GB drive running at 7200RPM with an 8MB cache. The other is a WD 1200JB, 120GB, 7200RPM, 8MB cache.
According to a testing program, the lookup speed for 10,000 files on the problem drive is 3.024 seconds, while the other drive with specs that are identical for everything except the size of the drive, clocks in at 1.402 seconds. This difference is much more evident in Windows Explorer, where it takes far longer than the numbers would indicate to show a directory listing.
All of the other speed tests (copy, read, and write for both small and large files) are nearly identical for the two drives.
Though I do use a combination IDE/SATA PCI card for two drives, neither one of the drives in question is connected through the card. Both are plugged straight into the motherboard.
It may simply be a slow drive, but perhaps there’re bad blocks – er, make that failing blocks, since “bad” would imply non-functioning – on the 160GB disk? That’s about all I got…
However, there are other questions that may be relevant…and I stress may be, while adding that they’re more preemptive for others than anything. What file system is on each disk? Has it been reformatted since you got it? Is one a master and one a slave? Are they both operating with the latest firmware?
One last thing: I googled a bit, trying to find performance comparisons that might support the contention that it’s just a slow disk. The best I found is a test of 21 160GB hard disks that shows WD drives don’t compare very well to other manufacturers. Note that they don’t explicitly mention whether these are IDE or SATA, but I’d assume IDE. Also, there’s a 120GB hard drive performance test found in the linked page shows comparable performance for both, indicating that the problem isn’t just the make/model, but either the particular disk or your system’s configuration.
Both drives have been formatted multiple times, always with NTFS and are using whatever firmware they shipped with. They’ve switched Master/Slave designations and IDE channels multiple times with the problems remaining the same the whole time. I think I’m just going to throw the problem drive into my old Xbox, where directory lookup speed won’t be an issue.
It matters which files the drive is looking up. The platters that hold data on a hard disk spin like records, so access times are faster towards the outside of the platters.
It seems like this is an obvious thing to control for in a performance checking tool, though. There are lots and lots of factors that could be different, though, between the two hard drives. The controllers on the hard drives would be the most obvious difference, and they can actually matter quite a bit. Take a look at Storage Review’s guide to hard drives for more (especially look at the sections on internal and external performance factors): http://www.storagereview.com/guide/index_perf_perf.html
But are those sequential files or random files? As Robcaro points out, there is much more to effective drive speed than just access & spin stats.
The software used to access the drive (opsys), the method of storing pointers (FAT), the fragmentation of a drive and interleave values will overwhelm any raw speed factors.
I can show you an optimized 286 machine with a simple FAT that will outperform a Windows Vista, if you allow me to tweak the parameters on each.
Another thing…most speed tests don’t reveal the number of retries (some of which may be internal). If a drive is failing, or has one or more bad spots that are being accessed, it will affect your overall throughput. Unlike the other factors I mentioned, this is one you should be concerned about, since a failing drive usually continues to fail rather than fixing itself.
Well, if it’s failing, then it’s been failing at a glacial rate since I got it three years ago, since it’s always been this way.
This is the program I used to test with. I’m not sure precisely what methods it uses, since the documentation is sparse.
That’s the strange thing; directory lookup speed is the only slow thing about it. The results of more tests against my other drives show that the copy, read, and write speeds are quite close, and extracting large files and other disk read/write intensive operations over the years have confirmed that.
I’m preparing it for the Xbox now. If the drive does fail, better it do so with backed up games that I still have the originals of on it than data that may or may not be backed up.
Directories are stored in the same way as files, and they, too, can become scattered and fragmented. This would be more noticeable if you have a very large number of very small files, since each would take a minimum of one directory entry. Unbalanced trees (and a directory structure is a tree structure) can cause slow reads, too, as searches (and a directory read is a search) need to traverse the entire tree from limb to limb.
Is there any way you can copy the one slow-reading disk to another of similar size and re-test? Do this as a file-by-file copy, not image. This should restructure all files and directories as cleanly as possible. A good defrag program (not the Microsoft default) will also do this.
Sorry I don’t have a better answer, but copying data to another drive might be eduational.
Thanks for the tips. I already copied all the files with the directory structure unchanged to my 320GB drive (Western Digital, IDE, 7200RM, 16MB cache) in the process of getting the 160GB drive ready for my Xbox. With 23,000-odd files from the old drive and 12,000 of its own, the 320GB drive takes 1.408 seconds on the directory lookup diagnostic. I don’t know if the problem was bad blocks in the part of the hard drive that stores the filenames and directory structure or what, but I guess it doesn’t matter all that much anymore.
That’s what struck me; I figured that a test suite would account for variations/branching in the directory structure. Since the read/write/copy numbers were similar, I figured it either had to a specific area on the disk OR some difference with the different data busses (e.g., the motherboard doesn’t yield the same performance for slave drives as masters). Since you say that the disks have been reformatted and used in various configurations, I’m kind of at a loss as to what the issue can be, especially given the long time period. I’m pretty curious about the actual reason, if only to store it among the anecdotes that all tech workers build up over time.
For instance, back in '94, the company I worked for used a network of 386s that ran DOS 6.1 from floppy (no harddisk, just RAM drives). On one project, we were getting errors that would halt the machines. It turns out that there was a hard limit of 128 subdirectories – but only when located in the root directory. Simply moving the entire filesystem structure from C:\ to C:\blah\ solved the problem, at the cost of days (!?!) of (sporadic) downtime.
In this case, I can’t even come up with a truly plausible explanation…
Certainly if your highest priority is directory access/search, that’s a good idea. But if file data is your ultimate goal, it will have less benefit.
I haven’t used Diskeeper, but I suspect it is superior to the MS XP default defrag program, which doesn’t even make all file blocks contiguous, but leaves large holes. At least that’s the way I’ve observed it to work.
Yep, that limitation was inherited from earlier DOSes, and was well known in hacker circles. We got used to keeping the root directory as lean as possible. Isn’t it amazing how Microsoft manages to ignore such bugs for so long and keep releasing new versions without fixing them?
And I used defrag programs in the 1980’s that were superior to MS XP’s default.
I just thought of something that might relate to the OP (imagine that!). Although I don’t know the method MS uses for directory searches in 2007, I do know the method used 10 or 20 years ago (and given the MS record, it probably hasn’t changed much). That was a linear search, and if you had a very, very large directory, the search was inefficient and become exponentially less efficient as the directory grew. (Example: A single directory of 100,000 entries may take longer to traverse than 10 directories of 10,000 entries each.) This, coupled with fragmentation, may have been the cause of the OP’s problem.
This can be alleviated by a file copy to an empty disk, and it seems you have observed exactly that.
It’s sad that there were better, faster, and more sophisticated implementations of computer techniques available from MS competitors like Novell in the 1980’s, but MS pushed them out of the market and we now rely on increased hardware speed to make up for poor software implementation.
I’m sure someone will disagree with me, so I’ll just say this is MHO.
This is just a WAG, but what about cluster size? If the cluster size is different, that’d make the FAT size different, which could affect how quickly it loads, right?
The default MS defrag is based on the Diskeeper engine that was available at the time XP launched. Presumably the commercial Diskeeper has evolved since then.
I am not a techie, so this may or may not be applicable-- just a WAG.
One of my recently added drives was annoyingly slower than the others in read/write operations. Trying to copy files to it (I intended it for mass storage) was a travail-- you could go to lunch, while waiting. I never got many files onto it, so I’m not sure if directory was also slow.
I did all the above and other tests as well. All no help. Drive manufacturer wanted to blame software. Software (Microsoft, WinXP) insisted it was a hardware issue.
Finally I discovered that the drive had, unknown to me, been installed by XP with “compression” enabled. Unchecked the compression box and, voila! problem resolved.
All of my hard drives were formatted in XP, so default cluster size on them, and I’ve never been hard up enough for space to turn on file compression.
Now that the drive is up and running in my Xbox, I’ve noticed that the list of menu options on my dashboard is slow to come up - this despite the fact that it’s using an entirely different file system! Ripping DVDs to the drive doesn’t seem any slower than the stock Xbox hard drive that was in there before.
Sounds to me like it has to be hardware or firmware on the hard drive, which narrows it considerably. I think it’s the controller - the little gizmo that regulates the data flow out of the hard drive. It could be that the controller’s software is just old and bad.
The fact that read/write stuff is fine rules out most mechanical problems, like an RPM under what is advertised or a slow actuator on the arm. My money is on the controller.
That is strange. Now, I know nothing about the Xbox, so I could be way off base. However, if the “dashboard” code is located in the same disk area (a big “if”?), it indicates to me that it may be a problem with that disk area. The thing is that if a disk block were failing, I wouldn’t expect it to do so for 3+ years. So that (most likely) shoots that down.
It was a nice thought, and provided the impetus to me to do a bit of larnin’. And I can say that I never want to look into NTFS again. :eek: NTFS uses a b+ tree for storing it’s directories, so that’s not it. However, I came across this article (and something similar at another site, independently), which says that for a directory structure containing >70,000 directories, the log file buffer will fill up (recording “last access times”) and need to be flushed for a full enumeration of names, which may lock the machine up for a time. The workaround is a registry modification, changing the value of NtfsDisableLastAccessUpdate from 0 (default) to 1.* This shouldn’t affect this drive, as neutron star has indicated only 23,000+ files (not directories). But it’s the only thing I’ve come across thus far, and my fatigue is currently outweighing my curiosity.
*: why one would include “Disable” in the variable name, when the value indicates exactly that is beyond me. This puts us in the position that “0”, which is usually taken to mean “false” or “off” actually means “on” in this case. Ah, how I do loves me some double negatives…