Replacement option(s) for chkdsk

[Appletax]

I've had plenty of times where the popup for safe to remove comes up and I still might get a chkdsk error next time it put it into a Windows machine. But this seemed more prevalent back in the XP days.

You no longer have to do a safe eject. But, I still prefer to use it just in case!

Windows default media removal policy

Manage default media removal policy in Windows.

docs.microsoft.com

Windows defines two main policies, Quick removal and Better performance, that control how the system interacts with external storage devices such as USB thumb drives.

In earlier versions of Windows, the default policy was Better performance.

Beginning in Windows 10 version 1809 (October 2018 Update), the default policy is Quick removal.

 

[britechguy]

Quick Removal has actually been the default since the advent of Windows 7.

USB devices were touted as "hot swappable" almost from day one and, at least to me, that always meant so long as there was no active write activity occurring you could pull the plug at any time. You could kill a read if reading was going on, but that would have no effect on the drive.

In the early days tons of people were losing things they did not expect to lose when they just pulled a USB storage device. So the move away from write caching as the default was very rapid indeed.

I've been just yanking (straight out, no wiggling) USB thumb drives, external backup drives, etc., literally for decades now with nary a second thought and no ill effects. That's what the whole point of Quick Removal is.

But those who wish to eject should do so. I'm only reporting what I do and what my experience has been. To each his or her own.

 

[Appletax]

Quick Removal has actually been the default since the advent of Windows 7.

USB devices were touted as "hot swappable" almost from day one and, at least to me, that always meant so long as there was no active write activity occurring you could pull the plug at any time. You could kill a read if reading was going on, but that would have no effect on the drive.

In the early days tons of people were losing things they did not expect to lose when they just pulled a USB storage device. So the move away from write caching as the default was very rapid indeed.

I've been just yanking (straight out, no wiggling) USB thumb drives, external backup drives, etc., literally for decades now with nary a second thought and no ill effects. That's what the whole point of Quick Removal is.

But those who wish to eject should do so. I'm only reporting what I do and what my experience has been. To each his or her own.

But the link I shared direct from M$ says Quick Removal became default in Windows 10 in 2018.

That must mean that prior versions were set to Better Performance?

Easy way to test: check it out in Windows 7 or 8.

Disk Management > Right-click the external device > Properties > Policies.

I would probably only enable Better Performance on an external drive that rarely gets removed, such as my external 8TB HDD.

 

[britechguy]

But the link I shared direct from M$ says Quick Removal became default in Windows 10 in 2018.

That must mean that prior versions were set to Better Performance?

Easy way to test: check it out in Windows 7 or 8.

Disk Management > Right-click the external device > Properties > Policies.

I would probably only enable Better Performance on an external drive that rarely gets removed, such as my external 8TB HDD.

You can find the Windows 7 documentation that says the same thing.

However, the acid test is checking on a Win7 machine for yourself. I can assure you that it will be Quick Removal as the default. If you find a machine on which that has been changed it will be:

1. Intentional on the part of someone who has charge of it.

2. Rare as hen's teeth.

And, for me, no performance improvement is worth the possibility of data loss due to something as simple as a power outage or just a "dip." When I write something, I want that action to occur now. And in the age of SSDs, there would be no appreciable performance gain. I'd say there's little with the HDDs that have been the norm for many years now, too. Some of these settings are anachronisms that hang about from the very early days.

 

[Appletax]

You can find the Windows 7 documentation that says the same thing.

However, the acid test is checking on a Win7 machine for yourself. I can assure you that it will be Quick Removal as the default. If you find a machine on which that has been changed it will be:

1. Intentional on the part of someone who has charge of it.

2. Rare as hen's teeth.

And, for me, no performance improvement is worth the possibility of data loss due to something as simple as a power outage or just a "dip." When I write something, I want that action to occur now. And in the age of SSDs, there would be no appreciable performance gain. I'd say there's little with the HDDs that have been the norm for many years now, too. Some of these settings are anachronisms that hang about from the very early days.

Good point! Would only consider if running on a UPS. Even still, it does not seem to be much benefit.

 

[britechguy]

I'm going to fire up my Windows 7 machine, which has never been tweaked much at all since I clean installed it (as it's the only machine I ever encountered that got an official "can't update to Windows 10" message due to some hardware limitation) and check again on Quick Removal. But there are other fish to fry today, first.

 

[fincoder]

What I'd like to know is:
Why does Microsoft still present the safely remove option? The OS knows if Quick Removal is enabled. Maybe it's there for legacy reasons.
Why does the OS often fail to release the drive? If Quick Removal is enabled, surely it should return that it's safe to remove, every time without delay.
And if pulled out without safe eject, why does it sometimes mark the drive as bad and suggest a chkdsk operation when next used?

I think the answer to these questions is:
There is still some risk removing drives without eject, regardless of what that linked Microsoft article says.

If instance if File History is turned on for the drive, it's not so easy to be certain the drive isn't in use. There could theoretically be other background processes writing to the drive as well. These scenarios have nothing to do with the Quick Removal option, they would need the user to safely remove before yanking.

 

[GTP]

There is still some risk removing drives without eject, regardless of what that linked Microsoft article says.

I'll continue to wait for the "Safe to remove drive" option or just shutdown if I'm in a rush.
I have about 120 gb of tools OS's and other software on a couple of 128GB drives.
I've lost so many USB drives that suddenly wont show up in Windows after being "pulled."
I do have backups of course but copying 120GB to a USB takes time.

 

[Joep]

Chkdsk is a file system consistence checker and repair tool, no more no less. I think 'data recover people' discourage use of chkdsk in scenario where access to data was lost already. I think anyone working in data recovery can give a number of examples where chkdsk turned a simple file system recovery into something more complex. It is also very possible that seemingly logical data loss issues have a physical cause in which case the last thing you may want is some tool running that sends r/w heads all over the place. In the end, chkdsk is not a data recovery tool.

I'd also like to add that chkdsk is an 'evolving' tool and IMO pretty robust as it is today. But in the past I have seen chkdsk really mess up file systems where NTFS drives suddenly became RAW or even FAT12 after running it (I am talking NT4 era). But my impression is, that it is quite good in doing what it's supposed to do.

AFAIK there's no replacement or alternative. The error checking (right click > properties > tools) tool, any GUI that Windows may offer to check a drive are in the end the same chkdsk tool. 3rd party disk doctor like tools are merely wrappers around chkdsk. And TBH if they were not I would not let it near my drive.

About USB flash drive file system corruption, I saw it discussed in the thread and I have been thinking about this too. I think most probable cause is the fact that Windows can control access to the drive from a Windows perspective. It can flush it's cache, make certain file system is in consistent state and then give it's blessing for pulling it. But we need to consider a USB flash drive is a computer in it's own right, complete with it's own ARM CPU. If you're not writing to the UFD, no files opened and all that, then even pulling it without waiting for 'safely remove' message, chances are file system is okay and consistent. But the UFD may be doing it's own maintenance, moving data around, erasing blocks and updating it's own file system (FTL) and Windows wouldn't know a thing about it. This may leave a small window for corruption when pulling the UFD. I am no 100% certain, but it would not surprise me if it's something like this.

 

[NETWizz]

NTFS has built in corrective function, Windows builds on that. There is no need to run chkdsk unless something has gone horribly wrong, and the same applies to repair-volume. The only time you should be using these tools is after you've imaged the data onto a new disk in an attempt to repair the volume damaged by the dying drive during the imaging process.

Note, DISM and SFC repair WINDOWS, chkdsk and repair-volume repair the NTFS volumes that are holding windows. So these tools have nothing to do with each other... Well other than the former set will cause the latter problem to get much larger if run on a damaged disk.

No it does NOT have any built-in corrective functions except for a journal. It is NOT like Re-FS, ZFS, BTRFS many of which have Extents, B-Trees, copy on write, snapshotting, transactional writes etc. NTFS unfortunately is at its heart still a 1990's filesystem.

As for running Chkdsk, I just run it WITHOUT checking for bad sectors. It either completes uneventfully in less than a minute, or it starts going crazy with really nasty outputs that scroll for 10+ minutes. If the later happens, replace the SSD/HDD; it is a hardware problem.

 

[britechguy]

Why does Microsoft still present the safely remove option?

To make a certain segment of the user base feel comfortable and "have what I'm used to."

Windows (and other OSes, too) are full of anachronisms that serve this purpose only, in many cases. Think about the hew and cry that goes up any time something familiar is removed, even if its removal is entirely logical because it serves no functional purpose due to design changes "under the hood."

 

[nlinecomputers]

I'll continue to wait for the "Safe to remove drive" option or just shutdown if I'm in a rush.
I have about 120 gb of tools OS's and other software on a couple of 128GB drives.
I've lost so many USB drives that suddenly wont show up in Windows after being "pulled."
I do have backups of course but copying 120GB to a USB takes time.

This. For as automatic as it is supposed to be, it sure fraks up a lot. Mostly related to Antivirus from what I have seen.

 

[britechguy]

For as automatic as it is supposed to be, it sure fraks up a lot.

I am not going to argue with you about your personal experience, but based on what has transpired on this topic as a whole I'd say that assessment is an exception case for a few, rather than the general experience of the many.

There are more reports of the, "I've just been pulling the drive when it's not actively being written to and never had any problem," class.

If one wishes to use the Safely Eject option, more power to you!

 

[Sky-Knight]

@NETWizz Self Healing NTFS was added with Windows 7 and Server 2008. That is literally the feature name... Google it. Basically, NTFS runs chkdsk for you automatically when it detects issues. The OS has scheduled tasks that also run chkdsk periodically built in.

So one should never have to run it themselves, and if they do... the drive is dead because you're in that 2nd category you've listed. If you see errors when running chkdsk bad things are happening to your data due to storage issues.

@britechguy I haven't had a USB storage device suffer data-loss on disconnect without the USB storage device itself FAILING at that same time in over a decade. The only time I use the eject button in Windows is when I'm working with magnetic external storage, to make sure it's done writing. USB storage is done writing when the file copy progress window goes away. You can still screw up files if you unplug during a file operation obviously, but in most cases those operations are obvious and people just don't do that. So yeah, I too don't bother with the eject button for USB keys... waste of time. Just unplug it when the writes are done.

 

[nlinecomputers]

@NETWizz Self Healing NTFS was added with Windows 7 and Server 2008. That is literally the feature name... Google it. Basically, NTFS runs chkdsk for you automatically when it detects issues. The OS has scheduled tasks that also run chkdsk periodically built in.

So one should never have to run it themselves, and if they do... the drive is dead because you're in that 2nd category you've listed. If you see errors when running chkdsk bad things are happening to your data due to storage issues.

Yeah, but it isn't the same as the file systems he is speaking of. They correct on the fly not flag CHKDSK for a scan on the next boot/task/etc. It's not as bad as 1990 NTFS but not as robust as ZFS et al.

 

[Sky-Knight]

Yeah, but it isn't the same as the file systems he is speaking of. They correct on the fly not flag CHKDSK for a scan on the next boot/task/etc. It's not as bad as 1990 NTFS but not as robust as ZFS et al.

No... that's what ReFS was supposed to be. And sort of is? ReFS is largely interchangeable with ZFS. But yeah, I'm not saying NTFS is as good as a modern filesystem, I'm just saying that you don't need to be running chkdsk unless something has gone horribly wrong. The platform is doing all the junk you need to keep NTFS healthy on its own these days.

 

[fincoder]

To make a certain segment of the user base feel comfortable and "have what I'm used to."

OK given that theory that I aready covered with the phrase 'legacy reasons', what about the next questions I posed:

Why does the OS often fail to release the drive? If Quick Removal is enabled, surely it should return that it's safe to remove, every time without delay.
And if pulled out without safe eject, why does it sometimes mark the drive as bad and suggest a chkdsk operation when next used?

And what about the specific case I mentioned of File History being active? At any particular moment, File History could start writing backup data in the background. Third-party backup and sync software could also write to the drive in the background. As I said, these scenarios have nothing to do with the Quick Removal setting. The possibility of background processes writing to a USB drive is the reason, in my opinion, that Safely Remove is still a thing, and the reason as techs we still need to advise customers to safely eject USB drives as a general rule. On our own machines we may decide that background access is not probable (e.g. automatic backup not activated) so we can take the risk.

 

[britechguy]

At any particular moment, File History could start writing backup data in the background. Third-party backup and sync software could also write to the drive in the background.

Only if you keep the drive connected, and these sorts of utilities running in the background. Neither of these practices is best practice, so I don't do them, and I would presume that they should not apply to any technician here.

I'm done arguing this. Hot swapping USB drives, not currently being written to, has been going on without incident for millions of users for decades now. I know a lot of people who have never even known about eject/safely remove because they were never taught to use it because it has been unnecessary under the Quick Removal paradigm.

One can think of many situations where "just yanking the plug" would be a horrible idea, but all of them involve active write activity to the drive at the moment the plug was pulled. Otherwise, it just works fine.

A very great many, I'd say most, USB thumb drives have activity lights that flash when either read/write activity is occurring. Every USB external hard drive I've ever touched has one. Telling an end user, "never pull the plug when that light is flashing," is all the instruction they are ever likely to need in this regard. This is not rocket science.

 

[GTP]

Mostly related to Antivirus from what I have seen.

At any particular moment, File History could start writing backup data in the background. Third-party backup and sync software could also write to the drive in the background. As I said, these scenarios have nothing to do with the Quick Removal setting

I just tried to safely eject a 4TB backup HDD and got a "Delayed write Failed" error. I didnt think delayed write was an issue now with fast drives and modern OS's?
I have a feeling that if I'd just yanked the drive without doing the safe eject option I would have lost about 3.3TB of backups.
I left it for about 20 mins and tried the safely eject option again, this time there was no problem and the "Safe to remove hardware" option appeared.
I checked the drive with CDI and CSmartControl both of which reported no problems.

 

[britechguy]

I have a feeling that if I'd just yanked the drive without doing the safe eject option I would have lost about 3.3TB of backups.

And I have the feeling, and "stupid play experience," to suggest you wouldn't. Generally if something gets screwed up it is the file under active write activity, nothing else.

I have never seen an entire drive be completely unreadable, or anything close to it, even when pulled in the middle of furious write activity.