Former Apple engineer delivers ZFS support to Mac OS X

Don Brady, a former Apple engineer and his company, Ten’s Complement, “are bringing ZFS support to Mac OS X with the company’s latest project, ZEVO Silver Edition,” Jordan Kahn reports for 9to5Mac.

“The Silver Edition is just the first product from Ten’s Complement and includes basic ZFS support for $19.95,” Kahn reports. “Other products will follow including a $40 Gold Edition and a Platinum Edition with support for RAIDZ and other advanced features. A Developer Edition will provide devs with ‘conventional ZFS command line tools,’ GUI options, and customization options.”

Kahn reports, “As MacRumors pointed out, the MacZFS open source project is also continuing development on the original work done by Sun and Apple.”

More info in the full article here.

Related articles:
Apple can do better than Sun’s ZFS – October 26, 2009
Apple discontinues ZFS project, turns attention to own next-gen file system – October 24, 2009
Apple’s Mac OS X Snow Leopard Server’s ZFS goes MIA – June 9, 2009

21 Comments

    1. Yes, it truly is a nice option to have, G4Dualie. But it isn’t for everyone.

      The advantage: ZFS pretty much guarantees against write errors and bit-rot. It is really nice for those who have mission-critical data that must be letter perfect from start to finish. It is especially nice for files that don’t work at all when a bit flips, such as zip archives and Excel files (the types of files that you click on and they say “This doesn’t appear to be a valid…”).

      Disadvantage: ZEVO currently won’t work on your boot drive.

      I’m all prepared to buy it, but only after it has a year-long track record where there are few users complaining that their data went to Heaven and ZEVO works on boot drives.

    1. HFS+ is the current OS X file system (Finder) and was designed by Don Brady and his team at Apple. Brady was a twenty-year Apple engineer who knows more about Finder/Zettabyte File System (ZFS) than anyone else in the business and subsequently left Apple when they scrapped the ZFS project.

      The “plus” in HFS+ was a stop-gap measure designed to upgrade/modernize the file system in the classic Mac OS for use in OS X (Unix) with plans to replace HFS+ with something more reliable and robust but, for whatever reason it didn’t happen. HFS is now twenty-five years old and “plus” has been in service for thirteen years!

      Brady left Apple, determined to implement ZFS in OS X as a commercial product, while Apple continues to support an open-source version. Brady is an expert in HFS and ZFS and is leveraging that knowledge to create a product that hides the complexities of file management and storage.

      For example, a hard-drive formatted by Apple’s Disk Utility using the ZFS feature creates a storage “pool”. That pool is supported by a number of virtual devices that the end user should not be exposed to (like the way sausage is made) because it’s too complex and confusing. But Brady’s team has made the process Mac-like and you’ll never know your using another file system.

      The immediate benefits of the basic ZFS properties will be storage efficiency and data integrity. Also, for those who have witnessed a decline in Apple’s server products have long realized HFS+ for server applications is not scalable like ZFS. While ZEVO is not directed at enterprise and its IT community, it is being offered to professionals and home-users alike. Quoting Brady, he said,

      “There’s a huge chasm between using Xsan over Fibre Channel and a USB drive with Time Machine,” Brady told Ars. “That middle piece is what we’re looking at—users that want the convenience of a device like a Drobo, but with more reliability and [easy verifiability].”

      Silver ZEVO, the entry-level product, will finally provide Mac users with a file system that self-repairs corrupt data on the fly, something Finder has never been able to do. In fact, Finder never concerned itself with data corruption. Whereas ZFS continuously monitors not only a files data but its metadata structures too. For example, data corruption in file system metadata structures can render a directory or an entire disk unreadable. If you’ve ever experienced a blinking question mark during the boot process, then you’ve experienced corruption of the metadata structure.

      There are so many other features available such as, Snapshots, RAIDZ, Copy-on-Write, etc. ZFS is a 128-bit file system, so it can address 1.84 × 1019 times more data than 64-bit systems such as NTFS.

      I think it’s inexcusable that Apple didn’t build the file system in OS X from the ground up and instead insisted Brady update Mac OS’s file system as an interim measure for use in OS X and here we are twelve-years later still using the tired HFS+.

        1. That’s what I’ll be doing with my purchase of this file system, only I won’t be using any server products, just OS X and Z-410.

          That’s the beauty of Brady’s work, he’s hidden all of the complexities of using a modern file system from the end user, making it “Mac-like” from the get-go. Perhaps one reason Apple stopped pursuing ZFS on the desktop was because they thought it was too complex for the typical Apple consumer.

          Confronted with the same limitations, Brady put a lot of thought into creating the same simple interface that many Apple products use to spare the user from dealing with the complexities of a modern operating system.

          For example, ZFS supports Snapshots, which means giving Mac users the ability to return our operating system to a former state (before the chaos occurred) with a few clicks of the mouse, something Windows users have enjoyed for years.

          However, Mac users will realize much more efficiency in storage because of the way ZFS stores updates to existing files on our hard drives. Instead of rewriting the entire file when changes have occurred, ZFS creates a small file on the drive that contains only the changes, preserving the original file. This is what makes Snapshots simple and efficient.

          Another feature I’m looking forward to in ZFS is the use of Sparse File support, something sorely lacking in OS X, but Windows users have been enjoying since the introduction of NTFS.

          HFS+ produces files that contain both data and empty or partially filled data blocks, that represent the actual file and is very inefficient by today’s standards. When Finder copies that file to the hard drive it purposefully fills all the empty blocks with zeros to ensure nothing else can be written to that space on the drive.

          ZFS, using sparse files, writes brief information to the metadata that “describes” the empty space rather than filling the space with zeros, and wasting drive space. So only actual data is written to the ZFS-enabled hard drive. When reading the file from the drive, ZFS refers to the metadata regarding the empty blocks and converts the empty blocks into real blocks filled with zero bytes during runtime and no one knows the conversion is occurring on the fly.

          The immediate benefit is you will notice an increase in potential capacity and speed.

          1. For example, ZFS supports Snapshots, which means giving Mac users the ability to return our operating system to a former state (before the chaos occurred) with a few clicks of the mouse, something Windows users have enjoyed for years.

            Where I meant to go with this is, Brady incorporated the use of ZFS Snapshots with the Time Machine back up system, making the implementation almost seamless for Mac users. Using the Time Machine interface you’ll also have access to Snapshots.

            What makes Snapshots so appealing is, instead of restoring your entire hard drive using Time Machine, just to return a single file to its normal state, you can use Snapshots to restore a file in a similar manner one does with with multiple undos in Photoshop. Think of the History feature in Photoshop, where you can click to switch to incremental states of your file. That is Snapshots in action.

            Like I said, incremental changes made to a file are not incorporated into the existing file and instead these changes are written to a secondary file and are created very quickly, and since all of the data making up the snapshot is already stored on the drive, Copy-on-Write makes for an extremely efficient storage system.

            1. Depends on what you mean. If you’re speaking of previously encrypted backups using HFS+ on a ZFS file system, then no. Those HFS+ backups will have to be upgraded to ZFS encryption.

              ZFS data is encrypted using Advanced Encryption Standard (AES) with key lengths of 128 ,192, and 256 in the CCM and GCM operation modes. ZFS encryption is embedded into the I/O pipeline.

              ZFS uses Oracle Solaris Cryptographic Framework and will take advantage of any hardware acceleration or optimized software implementations of the encryption algorithms automatically.

            2. Quoting you, G4Dualie: “What makes Snapshots so appealing is, instead of restoring your entire hard drive using Time Machine”…

              With Time Machine, you can select just certain files or an entire folder, like your who System Library or specific folder within it, and roll just it back to a certain time frame.

              The value of ZEVO Gold Edition is you don’t have to worry that your files had been rotten-defective for the last year and, you only just now discovered it, and you have to roll the file to the dawn of time (introducing other issues).

          2. “Something Windows users have enjoyed for years”?
            Thanks for the laugh, and thanks so much for putting that near the top of your clueless rant so I knew I didn’t have to keep reading.

      1. Thanks for the explanations, but isn’t the Finder wholly different from the file system?

        I always thought of it along the lines of the file system is a foundation, the Finder being the house sitting on the foundation. Wouldn’t the Finder still be used with the ZFS file system?

        1. At the risk of conveying I know what I’m talking about, I’ll just say Finder is a quirk in an otherwise solid OS and I have no idea how ZFS deals with Finder, but I doubt you can bypass it altogether.

          But therein lies the beauty of OS X, we don’t have to think about such things. Apple engineers have taken most of the guess work out of operating our systems, and weary Finder takes a nap while we click away our work. Hat tip to Don Brady.

          ZFS is Finder gen-two as far as I’m concerned. ZFS has grown stable over the years and has matured, nowhere near as tight as it could be, but it’s ready for primetime. And like a fine wine it will improve in time and my worries will go sublime.

          I have data I’ve been packing for twenty-years, in myriad formats, waiting for a day when I can begin a read/write marathon to consolidate all of my creations. ZFS is designed to work efficiently with hardware-accelerated products, but excels in a Mac with lot’s of RAM, making the transition from Finder-blessed files to ZFS’s file system, speedy.

          Those who have less than 16GB of RAM in their Mac, could use a SSD as RAM, to facilitate the upgrade to ZFS’s file system.

          ZFS has the ability to map to a solid-state drive, using the disk as a spool, as it writes the file to disk. With each write, ZFS treats both the metadata and file data with a coating of integrity. After that, subsequent calls to that file’s data is redirected to a sparse file for writing/recording subsequent changes. It should be noted though, ZFS will let you know if a file is FUBAR, and for all I know, it will present you with Finder to deal with the bad file. Short of Disk Warrior, et. al., the file is thrashed. Let it go.

          I can only imagine using a SSD as a substitute for RAM is not ideal, however it would probably be fast enough to minimize latency, as you upgrade your drives/files with ZFS. Once the upgrade is complete, read/write access will be f-a-s-t, your Mac will chew through audio I/O, putting the sizzle back into your Strat!

          So, in a perfect world, there would be no data corruption between our blind faith in Deus ex machina and the spoken word, so I have to rely on ZFS.

          I’m grateful to all the folks at Ten’s Complement for their diligence and relief.

          🙂

    2. What it means to the user, in broad terms is that you can have as many physical drives as you like and they will all comprise one large volume. Add or take away physical drives and the volume grows or shrinks automatically. That’s why it’s a 128 bit system. It has to be able to address a massive amount of storage space all in one volume. There are other major benefits, but the main thing is that you no longer have to fiddle with separate volumes, partitions, etc. for all your physical storage units.

    3. See above. If you had an operation that had crucial data that would kill your business if it wasn’t perfect, and even relying upon Carbonite backups and Time Machine wasn’t good enough because of the chance those could become mirrors of your defective files, then you would want ZEVO.

      As it currently stands, ZEVO won’t work on a boot drive (usually your built-in drive on which your operating system resides). Accordingly, it is for an external data-storage drive with mission-critical data.

      The real value of ZEVO will be the Gold Edition, which hasn’t yet been released. It will *correct* the bit errors. I assume it will also work on boot drives when it is released (as will the Silver Edition some day, I assume). I personally wouldn’t want to bother with the Silver Edition.

      But for most Mac users, if you aren’t already using Carbonite, don’t have an offsite hard drive that you bring in and back up to periodically, and don’t even have Time Machine, then you will not likely want to be responsible for upgrading and maintaining the atomic-level details of your computer system (which you would be doing with a hard drive file manager and logic volume manager).

  1. The Tenscompliment site is down; maybe they need to either use their own ZFS implementation, pay for a site that is reliable, or they *really* need the money (in which case I wouldn’t trust my data to their filesystem).

Reader Feedback

This site uses Akismet to reduce spam. Learn how your comment data is processed.