How do I diagnose this hard drive problem?

Although I would be delighted if we could solve this here, I suspect I may need specialized help. I welcome suggestions of sites I can go to get targeted advice for this situation.

Windows 10 Home
4TB hard drive, 678GB free. It’s the D: drive, not the system drive. Data only.

Symptoms:

  • File Explorer sometimes takes very long to show the content (green progress bar across the top can take up to two minutes before showing top-level files and folders)
  • Some other processes seem to be bogging down (coincidence?), even ones that I would not expect to touch this drive. For example, Excel hangs and Task Manager cannot end the process. I try to End Process and nothing happens. The app is marked as not responding. I cannot see how this could be related but both problems started at about the same time
  • Windows runs CHKDSK on the drive automatically when rebooting

Problems seem to point to file system corruption but no signs of physical failure.

Belarc Advisor is hanging at 24%, with 0% CPU and 0 disk. Previously reported on January 24:
ST4000DM006-2G5107 [Hard drive] (4000.79 GB) – drive 0, s/n ZC1ASBGG, rev DN04, SMART Status: Healthy

wmic reports OK on all drives.

Following are the results of running CHKDSK three times. It is finding file system problems but no bad sectors.

C:\Windows\system32>chkdsk d:
The type of the file system is NTFS.
The shadow copy provider had an error. Check the System and Application event logs for more information.

Volume label is Big 4.

WARNING! /F parameter not specified.
Running CHKDSK in read-only mode.

Stage 1: Examining basic file system structure …
1648128 file records processed.
File verification completed.
Phase duration (File record verification): 12.11 seconds.
118 large file records processed.
Phase duration (Orphan file record recovery): 0.00 milliseconds.
0 bad file records processed.
Phase duration (Bad file record checking): 0.28 milliseconds.

Stage 2: Examining file name linkage …
6382 reparse records processed.
1918908 index entries processed.
Index verification completed.
Phase duration (Index verification): 2.90 minutes.
CHKDSK is scanning unindexed files for reconnect to their original directory.
Detected orphaned file ShadowEditFiles (1B100), should be recovered into directory file 26.
Detected orphaned file Rabab’s 8mm conversion (17F882), should be recovered into directory file 26.
Detected orphaned file SnapShot.jpg (17F973), should be recovered into directory file 26.
Detected orphaned file Zoom Videos (19244D), should be recovered into directory file 26.
Detected orphaned file New folder (3) (19257C), should be recovered into directory file 26.
6 unindexed files scanned.
Detected orphaned file 4Paws (19257D), should be recovered into directory file 26.
6 unindexed files recovered to original directory.
Phase duration (Orphan reconnection): 0.00 milliseconds.
0 unindexed files recovered to lost and found.
Phase duration (Orphan recovery to lost and found): 0.58 milliseconds.
6382 reparse records processed.
Phase duration (Reparse point and Object ID verification): 18.97 milliseconds.

Stage 3: Examining security descriptors …
Examining an index entry with id 13F in index $SII of file 9.
Examining an index entry with id 13F in index $SDH of file 9.
Security descriptor verification completed.
Phase duration (Security descriptor verification): 24.92 milliseconds.
135391 data files processed.
Phase duration (Data attribute verification): 0.39 milliseconds.
CHKDSK is verifying Usn Journal…
Usn Journal verification completed.
Detected errors in the Master File Table (MFT) mirror.
The master file table’s (MFT) BITMAP attribute is incorrect.
The Volume Bitmap is incorrect.
Windows has checked the file system and found problems.
Please run chkdsk /scan to find the problems and queue them for repair.

3815429 MB total disk space.
3167696 MB in 1512548 files.
477304 KB in 135392 indexes.
0 KB in bad sectors.
1872351 KB in use by the system.
65536 KB occupied by the log file.
660929504 KB available on disk.

  4096 bytes in each allocation unit.

976750079 total allocation units on disk.
165232376 allocation units available on disk.
Total duration: 3.26 minutes (196001 ms).

C:\Windows\system32>chkdsk d: /scan
The type of the file system is NTFS.
The shadow copy provider had an error. Check the System and Application event logs for more information.

A snapshot error occured while scanning this drive. You can try again, but if this problem persists, run an offline scan and fix.

C:\Windows\system32>chkdsk d: /X /scan /forceofflinefix
The type of the file system is NTFS.
Volume label is Big 4.

Stage 1: Examining basic file system structure …
1648128 file records processed.
File verification completed.
Phase duration (File record verification): 12.78 seconds.
118 large file records processed.
Phase duration (Orphan file record recovery): 0.00 milliseconds.
0 bad file records processed.
Phase duration (Bad file record checking): 0.60 milliseconds.

Stage 2: Examining file name linkage …
6382 reparse records processed.
1918908 index entries processed.
Index verification completed.
Phase duration (Index verification): 2.88 minutes.
CHKDSK is scanning unindexed files for reconnect to their original directory.
Recovering orphaned file ShadowEditFiles (1B100) into directory file 26.
Recovering orphaned file Rabab’s 8mm conversion (17F882) into directory file 26.
Recovering orphaned file SnapShot.jpg (17F973) into directory file 26.
Recovering orphaned file Zoom Videos (19244D) into directory file 26.
Recovering orphaned file New folder (3) (19257C) into directory file 26.
6 unindexed files scanned.
Recovering orphaned file 4Paws (19257D) into directory file 26.
6 unindexed files recovered to original directory.
Phase duration (Orphan reconnection): 0.00 milliseconds.
An unspecified error occurred (696e647863686b2e 1f7f).
An unspecified error occurred (6e74667363686b2e 17b4).

C:\Windows\system32>

Is everything on this hard drive backed up? If not backing up the data should be your first priority.

I do not have the entire drive marked for backup, but critical files are backed up. Some very recent backups may have failed. I haven’t picked through all the logs but I use iDrive cloud backup.

I googled for those two “unspecified error” codes. Most of the results suggest that the drive is failing, which is what I would have guessed from the symptoms. However there was this thread in which the user reported that the problem went away when he disabled ransomware protection in Windows Defender. That seems unlikely to me but it might be worth looking into.

You said wmic reports ok, but if you haven’t done so, try

wmic /namespace:\root\wmi path MSStorageDriver_FailurePredictStatus

Without further info, my guess is the drive is on the verge of failure. You should make sure everything important is backed up, and prepare to get a new drive if no other solutions turn up.

ETA: there should be two backslashes after “namespace:”. Discourse ate one of them.

Windows can get stuck when any drive is not responding, even a drive that is not obviously needed by the stuck process. I experienced this just the other day with a failing USB memory stick.

Like others say, it does sound like the drive is failing. Backup up your data, and replace it.

You can also try downloading the SeaTools program, or whatever it is called this decade. That should let you run SMART tests, and possibly do a destructive write test.

I don’t know what the equivalent Windows commands are, but on Linux to prove to myself that a disk is dead I run badblks on it (write and then read/verify the entire drive) and then run a SMART slow test (read the entire drive). Badblks is a destructive (to the data) test. The combination of those two will almost always fail if the drive is bad.

Basically your data is worth many thousands of dollars (even if you pulled it off the internet for free it probably took hundreds of hours to acquire). On the other hand a new 4TB drive is in the $100 range. So either look into adding an additional 4TB drive to your computer or add a 4TB external portable drive and get everything copied over. [As to your iDrive backup it would take many, many hours to restore from there–and anyway you should have your important data preferably in 3 places.]

Note SSD drives are a lot faster, but also more expensive.

I just disconnected the drive and ordered a new drive. Hopefully I’ll be able to copy the data over. I could probably go on forever diagnosing and fixing but I think in the long run it’s worth it to just replace the drive.

I haven’t had a hard drive failure in a long time. I bought this drive in August 2019 and would not expect a failure this soon, but it’s also out of warranty. It’s a Seagate.

I remember the first time I bought a hard drive in the 1980s. It was industry standard to ship it with a list of bad sectors. Can you believe that we just accepted that a drive new in the box was going to ship with bad sectors?

P.S. Even with the drive disconnected I’m getting an Excel process that hangs and cannot be stopped by Task Manager. I got the PID and tried to kill it in a cmd window and that didn’t work either. So nothing to do with the hard drive problem.

It’s still the industry standard to ship with a list of bad sectors. It’s just hidden by the drive firmware from the user. Whenever a sector starts reading bad, the drive copies the data from that sector to another one preallocated for the purpose, updates a translation table it keeps internally, and keeps going with minimal loss of performance. You mostly start to notice when you run low on preallocated sectors.

Is it possible your Excel process was expecting to be able to access something stored on the drive you just took offline?

Yeah, bad sectors are, and always have been, inevitable. The technology has just gotten better at hiding them from the user. My very first job was working for Memorex in the 1970s, writing software to test newly manufactured drives and find the bad sectors. I also wrote what they called a “scratch fill” program: it looked at all the sectors that tested as bad and tried to determine if any of them were physically in a straight line on the platter. The assumption was that such bad sectors were caused by a scratch on the platter, so we would go ahead and mark as bad all other sectors in the same line, on the assumption that they were probably scratched too and would eventually fail.

I think it was a corrupted file. The problem occurred when a macro tried to open a second file. I restored the second file from a previous version and the problem went away. The odd thing was that Windows could not kill the process.

Replacing the HHD is the obvious best option in your case.

For future information or just out of curiosity have a look at this program that has helped me out on a few occasions.

HDD Regenerator is a unique program for regeneration of physically damaged disk drives. It does not hide bad sectors , it really restores them!***

HDD Regenerator

Once you have the new drive installed, and all your data copied over to it (and a few random files checked to make sure the copy worked), then take that old drive, and see if it can still be used. Reformat it, test all the blocks, etc. This will exclude the bad blocks, so it will be smaller than 4Tbytes now, but still usable for backup copies or something.
Or if you don’t want to spend the time, donate it to Free Geeks or similar, they will do that and then get it to some needy person.

Though if you’re going to replace the hard drive, I recommend getting a larger one.

I loved this program back in the day. Haven’t used it in a while given the immense time it takes to run on a decent sized drive by today’s standards. Plus SMART capabilities subsume most of its purpose.

I just recently found an ext. HDD was failing. SMART is out of spare sectors. I’m just going to try to copy what I can to a new drive. The cost and time of HDD Regen. isn’t worth it.

Thanks to all for your help.

I ordered a Seagate FireCuda 8TB, and got it today. I formatted it and am in the process of copying 3.5 TB of data to it.

Then we’ll see if the old drive can be salvaged for anything.