Why are we using ntfs in 2020

[Galdorf]

Ntfs is a horrible file system for this point in time why are we still using it for mechanical hard drives it just makes things worse in benchmarks ntfs is dead last optimization is poor and well using SSD/NVME looses speed and wear leveling we need a file system that handles wear leveling.

Sure the if its not broken don't fix it but there has to be a time when you need to retire and old file system due to changes in technology a proper file system that handles wear leveling for SSD would be a step in the right direction as well as more optimal system for mechanical drives to prevent fragmentation.

2020 is not far away does it not seem like new innovations for computers are slowing down why are we still using mechanical drives why have we not seen leaps in storage space you would think at this point in time we would be at 100's of terabytes and much faster cpu speeds.

Why is system restore in windows so bad in windows 10 i have seen it fail 90% of the time i think Microsoft really needs to fix this issue still after working with timeshift and Macs time machine puts system restore to shame i can't even remember a single failure in these programs.

I think it all boils down to this:
America has become so anti-innovation – it's economic suicide

 

[sapphirescales]

@Galdorf I'm surprised as a computer guy you didn't pick up on this, but Juicero isn't that stupid of a product. Just like printer manufacturers only really make money selling ink cartridges, this BS company only really makes money on the packets, not the machine. People are generally stupid, but with the internet you can't take people for a ride anymore because all it takes is one moderately intelligent person with a social media account to tell all the dumb ones how things work and the jig is up.

This product would have been a HUGE success in the 90's because the internet wasn't mainstream back then and there would have been plenty of dumbass people that would have bought it who would have never thought to squeeze the juice out of the packets without the assistance of the machine.

There are a LOT of companies still stuck in the 90's because it really was the last best time economically. Things were easy back then. The world was a lot simpler, the economy was incredible, and it didn't take much to make it big. Things aren't like that anymore.

And it's not just companies, but regular people as well. There are millions upon millions of Boomers and older folks stuck in the 90's and before that have never adapted to the scary new world. I feel like American's are generally in these categories:

1. Stuck in the past - they think of the world as it used to be and refuse to change.
2. Nostalgic about the past - they realize that things are really sh*tty now and wish things were different, but they can't do anything about it so they simply try to live on past glory.
3. Detest the past - these people are obsessed with moving forward, no matter what the cost. These people will push forward the most change, but not all of the change they push is good.
4. Live in the now - these people are generally wealthier and pretty well off. They think life is great right now and look forward to the future. These people are few and far between.

This product relies on an absolutely enormous amount of people in category #1. Unfortunately, most people in category #1 also use social media and the internet at least a little, so the only people that would fall for this products are basically in nursing homes right now. I don't see them buying too many $400 juicers and juice packets. This is why this product will fail.

 

[Sky-Knight]

I learned a long time ago... that what sells is greatly disconnected from what's good. Those $300 Dell or HP specials at Walmart... we know they're all junk, but they sell.

Capitalism thrives off encouraging poor decision making. It always has, and it always will. If the social media backlash always worked, Apple would have been out of business 20 times over by now, but there's still an endless march of idiots that want their 5 year out of date flagship for $1000.

And NTFS isn't horrible, it's a little dated but it's far from horrible. It's got a strong reliable track record. But, Microsoft IS replacing it with ReFS, which isn't really all that different from ZFS. But there's a rub, NTFS performs generally better for what we'd call normal computer use. ZFS and ReFS both excel at virtualized workloads, or network storage applications.

 

[britechguy]

I think it all boils down to this:
America has become so anti-innovation – it's economic suicide

I think it boils down to America having become anti-intellectual on a very broad scale. Isaac Asimov sagely observed, decades ago:

The strain of anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that "my ignorance is just as good as your knowledge."

and this spot-on observation (specific to the anti-vax movement, but definitely applicable more broadly) appeared in the New York Times in September:

Science has become just another voice in the room; it has lost its platform. Now, you simply declare your own truth.
~ Dr. Paul A. Offit, in New York Times article, How Anti-Vaccine Sentiment Took Hold in the United States, September 23, 2019

There was a time, and it was for most of my lifetime, when the general public recognized that there exist subject matter experts for areas in which one has no real training, and when you don't know about something the smart thing to do is to defer to those same experts. There was also a lot more emphasis in formal education on critical thinking skills and one of the first things you learn in being a critical thinker is, "Consider the source." A huge swath of the American public thinks a tweet or a Facebook post by some unknown bumpkin holds the same weight as, say, a statement on the human genome by Francis Collins or an observation on the economy by Paul Krugman or Peter Diamond.

So long as the belief expressed by Asimov is actually holding sway in public life we will continue on a path of decline.

(Sky-Knight is also correct about hucksterism being a built-in component of capitalism, hence the need for regulation. And even that will never ensure "the best" outcomes, but attempt to avert the worst.)

 

[britechguy]

ReFS is on shaky ground:
Microsoft to remove full ReFS support from Windows 10 Pro ...
(and that article is from 2017).

 

[Sky-Knight]

Well, I'm not sure that ReFS makes sense on the desktop, as I mentioned before it's designed for bulk file storage, and virtualization.

NTFS on the desktop is functionality and quality wise equivalent to EXT4, and many other filesystems designed for general use. So I have a hard time calling this "bad", other than the use of one of the open standards would make Microsoft systems more compatible. If that's even possible, because even NTFS has been rather open in the last decade, so I'm not sure too many more gains are to be had here.

But in regards to ReFS in the home, it's rather niche... because most people I'd assume would just get a NAS.

 

[Philippe]

We use NTFS because it's New Technology (File System)
I have to make this one lol

 

[Diggs]

And NTFS isn't horrible, it's a little dated

But, but, but it's Microsoft's "New Technology File System". ...chuckles.....

@add - @Philippe beat me to it.

 

[Sky-Knight]

Ahh, yes the NT jokes. Classic!

 

[britechguy]

By 1993, when it was introduced, Microsoft should have known better than to give anything a formal name with "New Technology" as part of it.

If there's any industry where the word new has the shortest possible shelf life, it's computing!

 

[phaZed]

Well, it might not be new anymore, but it is still the newest (For Windows anyways) - not a bad call IMO, it has lasted 27+ years so far. Even for "Windows NT" - We're still using large swaths of NT today.

 

[Alexey]

From my point of view, NTFS still arguably the best I have seen.

 

[Galdorf]

While NTFS is still ok we are moving away from mechanical hard drives now lets talk speed:
https://openbenchmarking.org/result/1608041-LO-LINUX44BT99

Notice NTFS is dead last in ALL tests not to mention no wear leveling native support most devices have this built in which adds to the cost why have there been no real innovation in a better file system that not only allows more performance for gamers but also business applications as well and supports SSD/NVME drives natively.

Even EXT4 is better i have done extensive testing using a PLEX media server and transferring huge amounts of video files NTFS with huge amounts of files the performance degrades very fast VS EXT4 not to mention fragmentation is not an issue where NTFS well if your transferring large amounts of files deleting files and replacing files causes HUGE performance hits on mechanical drives requiring very long defrag times.

FAT32 vs Ext4 vs NTFS

• FAT32 is the older. NTFS is the newer drive format. Ext4 is the newest of these drive formats.
  
  
• FAT32 originally designed in 1977. NTFS introduced in July 1993. And Ext4 stable version released on 21 October 2008.
  
  
• FAT32 is read/write compatible with a majority of recent and recently obsolete operating systems, including DOS, most flavors of Windows (up to and including 8), Mac OS X, and many flavors of UNIX-descended operating systems, including Linux and FreeBSD.
  
  
• NTFS is fully read/write compatible with Windows from Windows NT 3.1 and Windows XP up to and including Windows 8. Mac OS X 10.3 and beyond have NTFS read capabilities, but writing to an NTFS volume requires a third party software utility like Paragon NTFS for Mac.
  
  
• Ext4 is one of the latest and greatest Linux file formats.
  
  
• Ext4 modifies important data structures of the filesystem such as the ones destined to store the file data.
  
  
• The ext4 format allows users to still read the filesystem from other distributions/operating systems without ext4 support.
  
  
• Ext3/4 is by far the best filesystem format, but it's not supported natively by Windows or Macs. A good option is to create a small FAT32 partition and copy or install an application such as Ext2Fsd and format the rest as ext4.
  
  
• ext4 has very large limits on file and partition sizes., allowing you to store files much larger than the 4 GB allowed by FAT32.
  
  
• Use Ext4 when you need a bigger file size and partition limits than FAT32 offers and when you need more compatibility than NTFS offers.
  
  
• NTFS is ideal for internal drives, while Ext4 is generally ideal for flash drives.
  
  
• Ext4 filesystems are complete journaling filesystems and do not need defragmentation utilities to be run on them like FAT32 and NTFS.
  
  
• The ext4 filesystem can support volumes with sizes up to 1 exbibyte (EiB) and files with sizes up to 16 tebibytes (TiB).
  
  
• The maximum possible size for a file on a FAT32 volume is 4 GiB.
  
  
• The design of the FAT32 file system does not include direct built-in support for long filenames.
  
  
• Ext4 is backward-compatible with ext3 and ext2, making it possible to mount ext3 and ext2 as ext4.
  
  
• Ext4 uses a performance technique called allocate-on-flush.
  
  
• Ext4 allows an unlimited number of subdirectories.
  
  
• The ext4 file system does not honor the "secure deletion" file attribute, which is supposed to cause overwriting of files upon deletion.
  
  
• Windows uses hard links to support short (8.3) filenames in NTFS.
  
  
• NTFS is a journaling file system and uses the NTFS Log to record metadata changes to the volume. It is a feature that FAT does not provide and critical for NTFS to ensure that its complex internal data structures will remain consistent in case of system crashes or data moves performed by the defragmentation API, and allow easy rollback of uncommitted changes to these critical data structures when the volume is remounted.
  
  
• When it comes to file checking, EXT4 is quicker because unallocated blocks of data are marked as such and are simply skipped during disk check operations.
  
  
• The Encrypting File System (EFS) provides the core file encryption technology used to store encrypted files on NTFS volumes.
  
  
• FAT is a simple file system that is supported for reading and writes on all major operating systems (which is why it's a good choice for external drives), it has no security and it does not perform well with large files. NTFS makes improvements on FAT with security and in many cases contiguous reads, but it still suffers some similar ailments. Ext is generally a good choice for working with most files, however, small files would benefit more from contiguous allocation.
  

 

[Alexey]

lets talk speed:
https://openbenchmarking.org/result/1608041-LO-LINUX44BT99

They seriously benchmarked Linux driver for NTFS?

 

[Alexey]

Also, no modern filesystem has any kind of software wear levelling. As of now, it is done in hardware. Software support for TRIM is available in all major filesystems; don't know about FAT, and don't know about NTFS driver for Linux, but all native drivers for all modern filesystems support TRIM.

 

[phaZed]

Also, no modern filesystem has any kind of software wear levelling. As of now, it is done in hardware.

This.

Doing a Google search, it looks like only JFFS2(Journalling Flash File System version 2, 2001) and YAFFS(Yet Another Flash File System, 2002) and UDF(Optical disks) have wear-leveling. JFFS2 and YAFFS are both for embedded systems and embedded linux products that would contain "cheap" flash chips of the 1990's and early 2000's (eg. no flash hardware controller, whatsoever).

If wear-leveling is in hardware and not software it's:
1. Faster.
2. Less Overhead; no system bus usage.
3. Dedicated. QC by manufacturer and is software and file-system agnostic

I wouldn't want to "emulate", in software, what custom logic ASIC's and FPGA's can do. So, in reality, there would be no need (It would be worse, actually) for a File-system to implement wear-leveling and would introduce a host of other logical issues.

 

[Sky-Knight]

They seriously benchmarked Linux driver for NTFS?

Yeah that made me laugh... NTFS interoperability with other OS's has been a huge problem for ages. And benchmarks against that driver mean little when using a Microsoft OS directly.

And comparing NTFS to FAT32?!? Are you joking? The latter doesn't journal, and doesn't have ACL support! The comparison to EXT4 is fair however.

One more thing, NTFS has versions too, and it automatically upgrades as the new kernels see the old filesystems. So one cannot say NTFS is "old", each version of Windows since XP has released an updates to the filesystem, particularly in how the ACLs are stored. That's why you can always go forward, but sometimes you cannot go back despite all the OSs involved supposedly using NTFS 3.1.

 

[Galdorf]

My gripe with NTFS and FAT is speed and fragmentation on mechanical hard drives when working with VERY large drives and HUGE amounts of large files being created and deleted it takes forever to defrag when i mean huge i am talking 800k+ fragmentation should not be a thing in 2019 on mechanical drives.

 

[Joep]

Fragmentation of large files isn't that much of a deal unless it's excessive. Yes, defragging a large file takes time because it actually involves copying data from one point to another so it's limited by speed of the drive. It takes more time defragging than accepting a huge file being fragmented into a few pieces, in fact defragging such files doesn't make much sense in such cases. Solution, use a SSD instead?

I agree with others that NTFS, while not the youngest, and maybe when done from scratch again today it would be designed differently, is very mature and rock solid. If technology is good, or even good enough and widely accepted then it tends to stick around. People like stuff that has proven itself over time. Heck, even FAT, originally designed for floppy disks is still around and pretty much standard on SD Cards, and yes doing just fine.

Flaws you see are due to you mixing up stuff partially at least, you attribute slowness to the file system while it is at least a large part is due to the fact that you're using slow mechanical drives.

 

[Alexey]

My gripe with NTFS and FAT is speed and fragmentation on mechanical hard drives when working with VERY large drives and HUGE amounts of large files being created and deleted it takes forever to defrag when i mean huge i am talking 800k+ fragmentation should not be a thing in 2019 on mechanical drives.

What tool are you using to measure fragmentation, and are you talking Windows or Linux NTFS drivers? Specifically, what is the 800k+ number?
Also what is your standard of very large and huge? I found out that upwards of 16TB volumes and upwards of 100-200 GB per file, you want to disable System Restore. With System Restore enabled, some write operations will randomly take minutes to complete. However, my understanding is that the problem is with System Restore.

Another factor with spinning drives is SMR, Shingled Magnetic Recording, which is used on high capacity drives and totally kills random write performance. However, it affects all filesystems in a similar way. And no, there is no standard to force manufacturers to clearly label SMR drives as such.