28

Back in 2008, Linus Torvalds famously said in an interview that "OS X in some ways is actually worse than Windows to program for. Their file system is complete and utter crap, which is scary." I've looked for more details about why he feels this way about the OS X filesystem (HFS+ presumably) but I haven't been able to find anything.

Linus surely doesn't dislike the basic Unix filesystem model, and I doubt he hates HFS+ for being case-insensitive. And despite how provocatively his comment is phrased, I doubt that it's completely without merit. Since the comment was in the context of programming for OS X, I suspect his opinion may have been based on performance, robustness, the operating system interface, or something along those lines. Does anyone know what complaints 2008-era Linus might have had with 2008-era HFS+?

|improve this question
  • 2
    He has been known for having really strong opinions about some things, for example when he gave a talk about git @ google, he spent a good part of the talk trashing the other systems. So I would say that he probably has a reason to believe what he thinks but he is also a very exaggerated person, even though he is a genius. youtube.com/watch?v=4XpnKHJAok8 – El Developer May 18 '12 at 21:27
  • 3
    If you do not get the response to this question here which you were hoping for then you might also consider searching (and possibly also asking) on either Unix & Linux or Super User. (With so many sites available now it is sometimes hard to know which is the place to ask a question. At least IMHO. :) – irrational John May 19 '12 at 0:04
  • I tend to butt heads with HFS+ more than any other filesystem I normally encounter. These days on most systems I don't feel like I usually even notice or care about what filesystem it's using, but HFS+ always comes up with something. Like just today I found I was being screwed by its lack of sub-second resolution for modtimes. There was also the time I found two lines of C code that could cause a deadlock in the filesystem pretty much bringing down the whole machine. That wasn't even fixed as of 10.5. Not sure about more recent versions. – Iguananaut Mar 1 '13 at 16:13
21

A transcript of the “Q&A” session in which Linus made the comment is available, but it seems he wasn't asked to elaborate. I'm not sure whether a more in-depth analysis of his opinion on HFS+ has been written down somewhere else.

For someone else's analysis of the matter, you can take a look at John Siracusa's Mac OS X reviews. In particular the one for Mac OS X Lion which has a section titled “What's wrong with HFS+.” I think the most salient bit is (emphasis mine):

Concurrency, metadata written in the correct byte order, sub-second date precision, support for massive volume sizes, and sparse file support are all common features of Unix file systems. Mac OS X, of course, is built on a Unix foundation. When HFS+ was ported from classic Mac OS to Mac OS X, it needed to be extended to support some minimum set of features that are expected from Unix file systems.

Some of those features were an easy fit, but others were very difficult to add to the file system without breaking backwards compatibility. One particularly scary example is the implementation of hard links on HFS+. To keep track of hard links, HFS+ creates a separate file for each hard link inside a hidden directory at the root level of the volume. Hidden directories are kind of creepy to begin with, but the real scare comes when you remember that Time Machine is implemented using hard links to avoid unnecessary data duplication.

The important point here is that Mac OS X is using a file system which wasn't even designed for a Unix system, it was designed for classic Mac OS and patched to implement the features of Mac OS X 10.0 while maintaining backwards compatibility. Apple has subsequently implemented the additional features that it now has in Mac OS X 10.7 (journaling, metadata, filesystem events ...) using the same patching approach rather than a “design from the ground up” approach. I'm not sure how to explain this non-technically, but you could say that all of these additional features are resting on a classic Mac OS foundation that was never designed to support them. This means the solution isn't as good as it could be. The example that Siracusa goes on to discuss is that the solution Apple had to use for hard links while working within the limitations of HFS+ is too sensitive to hardware failure, which is compounded by the fact that HFS+ was also never designed to concern itself with data integrity. Of course, maintaining compatibility with classic Mac OS was a desirable limitation in Mac OS X 10.0 but it really isn't anymore in Mac OS X 10.7.

|improve this answer
  • 1
    Great link; that covers many important things. Lack of sparse file support is pretty nuts. Linux ext2 did sparse files even with simple block-bitmap based allocation, like HFS+ uses. I think he makes too big of a deal about storing metadata in big-endian, though. The x86 bswap instruction is very fast. It makes the code bigger and uglier, but maintaining on-disk compatibility is a big deal. Linux XFS still stores all metadata big-endian (except native-endian in the journal), due to its origin at SGI on MIPS CPUs. It's not an ideal situation, but XFS isn't held back by it. – Peter Cordes Mar 18 '15 at 7:18
7

Although I´m not an Operating System expert, and I´ve just started using OSX after coming from Windows, I consider myself a PowerUser in Windows, and fairly competent in Linux. Coming from that background, I´ve been surprised that in a fairly modern OS like OSX, the filesystem has quirks such as the way the names of files are "mungled".

I understand that Linus´ issues with HFS+ stem from the same point: from what I've found researching the issue, HFS+ stores the names of files using Unicode, but when a file uses "extended" or NON-ASCII characters (like á, é, í, ó, ú, ñ from Spanish or things like the ü in German), for which Unicode provides 2 ways of encoding the name, OSX silently "normalizes" the encoding at storage time... Not a real issue when the file has been created and consumed in OSX, but when you're sharing information with users of other OSs, the fact that the name of the file changes, makes for all sort of weird behaviours...

Case in point: I've been tracking my work "artifacts" (files, documents, etc) in Subversion for the last 8plus years. When moving to Mac, I got the SVN client for Mac, and after doing a Checkout of my relevant directories, I found that all files that have accents appear to be missing, and a new file with the same name appears as non-versioned. Digging into it, the issues is that the file IN the file-system is apple-encoded, while the data in the repository uses another (perfectly valid and legitimate) Unicode encoding...

This, I think, is a gross "mangling" of my data. Apple DOES understand both formats of the filename encoding (accessing a share in Windows, or using a USB stick from Windows shows the proper file names, etc) but at file creation time, it's decided "it knows better" and just renamed the files...

Again, not something most users will notice - until they make a copy of a file, or rename it, and put it back to where the original one was and end up with two files that are apparently the same!!!)

|improve this answer
  • 1
    This is just one point, and the real issue is that different OSs simply normalize the strings in different ways, and cross-platform apps don't deal with that. Not normalizing the names would probably be worse (you could have two different files with names that normalize to the same string, on OS X). – Blaisorblade Mar 19 '16 at 13:38
4

John Siracusa & Dan Benjamin discuss some disadvantages of HFS+ in Hypercritical #56.

They addresses data corruption in HFS+ and consider some of ZFS's features.

|improve this answer
  • 9
    Is there any way you can provide a summary of their discussion in your answer? The audio stream is (at this point in our current technology) un-searchable and very long. Not to mention it's on another site so it's susceptible to link rot. This would be a much better answer if it contained specific details about their discussion. – Ian C.♦ May 27 '12 at 13:10
  • 1
    The filesystem talk starts 23 minutes in. – neoneye May 27 '12 at 16:36
  • 1
    Most of the information available in the podcast can also be found at an Ars Technica article by John Siracusa (one of the two men in the podcast.) – TML Jul 26 '13 at 9:09

You must log in to answer this question.

protected by nohillside♦ Aug 12 '13 at 15:16

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged .