Why you can edit a 250GB file on a MacBook Pro with 16GB RAM

“When you fire up an application, the operating system assigns it a virtual address space,” Robin Harris writes for ZDNet. “In Windows 8.1 and later with 64-bit apps, that address space is a hefty 128TB, while macOS offers a ginormous 18 exabytes of addressable space for 64-bit processes.”

“The operating system — Windows, Linux, or macOS — then manages the virtual to physical address translation and swapping in and out of physical RAM of active program segments. Typically the segments (or pages), are 4 or 8KB,” Harris writes. “The CPU provides hardware assist to the OS to keep track of millions or even billions of pages.”

“Naturally, the speed with which pages can be swapped has a huge impact on system performance. That’s why advanced PCIe/NVMe drives — such as those in the latest MacBook Pros — are vital,” Harris writes. “From a cutting edge feature in the ’70s to omnipresent and forgotten in the ’10s, virtual memory is the technology that enables your notebook or desktop to run data sets that are way beyond RAM size. I commonly edit 250GB ProRes video files on my five year old 16GB MacBook Pro — without maxing out RAM usage.”

Read more in the full article here.

MacDailyNews Take: Behold the raw power and capabilities of macOS teams with blazing fast SSDs.

30 Comments

  1. Yes – true. And as a long time UNIX user, I concur – a great feature that’s inherent in the UNIX and derivatives and has been for decades and one that works for many circumstances. BUT sometimes more memory is just the answer – esp. for streaming large samples, video files, working on dozens of tracks in Pro Tools / Logic / etc.

    So – while “16GB is more than anyone will need” (to paraphrase an old saying 😉 ) it’s even currently not true for (ahem) significant categories of Pro users…

    Wearing another hat, I work sometimes with very large data sets being crunched in parallel that are CPU and memory bound and paging is not the answer there since keeping all the data in memory is of paramount importance. 48GB or up to 64GB is not uncommon in such circumstances for comfort.

      1. Next iteration will probably see this option. By now we all no the “whys” of Apple’s decision. 16GB serves the purpose of 90% of MB Pro users. Hopefully by WWDC we’ll see a new batch of MB Pros that will address the needs of the remaining 10%.

        1. Yup. Once Intel gets off their ass and has an efficient ddr4 controller, we will see 32gb as an option. I think everyone is missing the forest through the trees of phil’s comments about power consumption. If they used 32 in this machine, that would require Meroe power sure, not a ton, and probably not by itself enough to inhibit battery performance. But couple that with excess heat, and the fans have to spin faster and more often, causing more energy drainage. But an efficient memory controller fixes that, which is what they’re waiting for.

          1. I wish people would stop blaming this on Intel.

            Apple could have shipped this variant of the MBP more than five months ago (and maybe more than that) in the old case and had enough space for significantly more battery and not had to resort to the low power RAM for long battery life. Then when the Kaby Lake chips ship next quarter Apple could have come out with a 32/64 GB version that used low power RAM and had long battery life too.

            It was *APPLE’s* decision to not ship something late this past spring or early this summer. It was *APPLE’s* decision to go for thinner and sleeker at all costs in this iteration. It had absolutely nothing to do with Intel or the chips that Intel was shipping.

            1. Precisely!

              The proof in how bad Apple muffed the new MacBook Pro is right here:

              https://browser.primatelabs.com/mac-benchmarks
              Notice how the new 2016 MBPs don’t actually crunch numbers any faster than the ones shipped 2-3 years ago!!!!!!!!!!!!!!!!

              There is an uncomfortable trend on MDN to blame everyone else when Apple makes boneheaded decisions. Intel is not the problem. Apple’s obsession with making products thin is the problem. The MacBook Pro is not a pro computer in this day and age, it’s clear Apple sacrificed battery and processing power in order to give Ive a thinner device. So unlike many Macs of years past, the MBP is an executive laptop designed for office work and giving presentations, that’s it.

              Pros have been asking for more power and more user configuration/updateability in laptop and desktop configurations and NOT thinner machines for at least 3 years, ever since that turd of an overpriced trashcan was released to resoundingly bad sales.

              By refusing to listen, Apple is digging its own grave. The efficiency and capability of the Mac OS has not improved since Snow Leopard, meanwhile the competition has improved. Apple can’t claim to be competitive using inferior hardware. Apple’s current strategy seems to aim only for undemanding well-heeled fashionistas while the older generations of educators, photographers, video producers, small businesses, power users of all kinds are being forced to look at other options or be overcharged for underperforming Macs today.

              Something stinks in Cupertino, and I think it’s the rotten leadership.

    1. Fair enough Derek, but it is also true that those types of massive datasets have not historically been handled on laptops. By necessity, portable computers represent a compromise between size, weight, performance, battery life, etc. The degree of compromise has gradually declined with SSDs and improvements in CPU performance/watt, and current laptops are more powerful than “supercomputers” not so many years ago.

      Apple could certainly build an MBP with more RAM, and I have no doubt that it will in the future. But I also believe that the number of people actively complaining about the 16GB RAM limit in the new MBPs *far* exceeds the number of people who actually need it and can fully utilize it.

      1. So are you saying I should revert back to the days of mainframe work — tell customers and associated teams I cannot answer their questions on the road, but rather that I need to take down their questions and I’ll get back to them in a few days? Why can’t I expect to move beyond that stage?

        Yes, a laptop is no substitute for a true workstation or top 500 machine. (And that brings up the long standing question of, “Where is Apple’s current Mac Pro workstation?”) However, I should be able to do reasonable work on the road. My hopes were dashed when the current version of the MBP was announced. Now I’m hoping for something about this time next year.

        Yes, the capabilities and implementations of laptops is a compromise between what’s available in a true workstation and what’s in a smartphone. For my work Apple has erred too far to the sleek and thin side of things.

      2. Even if I do not work with those massive data sets often on Macbooks – I do sometimes – I *do* work with potentially large audio samples and large track counts in PT/Logic etc. as I said. For those, the more memory the better.

  2. so i have 123 frames of 8K 3D, each frame with multiple layers.. and some of these frames take up 8-10GB…

    i need to play it back at 60FPS..

    is the new macbook pro gonna do that for me?

    uh. no. no its not.

    my 128GB 2400Mhz Ram Desktop workstation is though.

    1. Actually, it will. With the way OS X swaps files onto the ssd, and with nvme drives being so fast, creating a 64gb swap file isn’t an issue at all. I’ve been very surprised with how well memory optimization works with the new 15″ since it’s been running. It’s faster at several editing tasks than our Mac pro’s and more than 3x times faster than my 2012 at exporting 4K footage. We also have no issues importing 6k raw red imagery onto them, and the new graphics card (although not the best for anything beyond 1080p gaming) is incredibly good at parallel computing and graphics rendering. Before making these types of comments, I suggest you actually use the product. It’s quite honestly the fastest laptop I’ve ever used, and that includes some of those beastly gaming machines. The sum of the parts, like an iOS device, is greater than any one specification, that new ssd makes such a difference you’ll be shocked. Besides, your 8k footage isn’t going to take up 10GB a frame while editing, and when played back it’ll be compressed to run well within the graphical buffer of the gpu… 4K footage typically consumes ⅛ to ¼ of the full buffer, with 6k at ⅓ and 8k at ½ to ⅔ of the 4GB buffer in the Radeon 460 pro inside this machine, even while editing. That’s partially due to the efficiencies of OS X, which if you’re a Windows user, you do not understand. If you need a quad core 4.0 GHz processor, 64gb of ram, and an 8-12gb video card to render and edit 4K and 6k footage on windows in premiere (which is inefficient and terrible), cut all of those specs in half at least and you can get the same or better results in Final Cut Pro, or davinci resolve, or even premiere on the macintosh. If an iPad Pro can edit two simultaneous 4K streams (which it can), and even the lowly MacBook can easily handle 4K editing…. then it’s the OS stupid. And the integration that we all prize so highly.

        1. Uh huh, fuck off, you don’t even understand what we’re talking about anonymous troll. Getting ready for the homophobic crap to start soon. Oh and in case you bring it up, we are moving to Canada march 9th.

          1. Nice response /s, but does not help your case.

            Yes, a single Digital Cinema 8K frame (as different from Super Hi Vision, the proposed follow on to UHDTV) even at the realistic max of 36 bits per pixel and multiple layered fields per frame for 3D is still less than 1 GB per frame uncompressed including meta data overhead.

            However, stating that the new MBP is 3x faster than your 2012 Mac Pro at editing 4K (I assume your talking about UHDTV and not true Digital Cinema 4K) just hurts your argument as it’s not realistic — unless you have an extremely underpowered 2012 Mac Pro. (You could buy a 2012 Mac Pro with just a single 4 core CPU, 6 GB RAM and really crappy hard drives. But no video editor worth their salt would have been caught dead with that configuration.) Besides, for less than 2/3 the price of a fully tricked out current MBP you can take a decent 2012 and upgrade it to a machine that will eat the current maxed out Mac Pro for lunch and as for more.

            Oh and some big companies (e.g., Macdonald Detwiler) are moving to the U.S. from Canada next year. So what’s your point?

            1. Fair enough I left that out. This is an ongoing discussion, and I have referenced my 2012 Mbp several times in the course of the discussion, as far as the system being faster at several tasks than our Mac pros, it’s true. Exporting footage happens to be one of them, but it’s not that much better. When I say my 2012 I’m referring to my mbp. My bad. Secondly, the homophobic remark I made is what that person typically says to me in their comments and my reference to moving to Canada is also in reference to the ongoing discussion of my leaving the country with the results of the election.

            2. My current machine, to reiterate is a 2012 mbp with the following specs: (non Retina) 2.6 i7, 650m gt, 2x512GB ssd’s in raid 0 (optical drive removed). That gives throughout of 1047 MB/s read, and 944MB/s write. (The drives are 2 sandisk x400 Sata 3 ssd’s) My new mbp has speeds of 3230MB/s read, and 2000-2100 MB/s write (it’s never very consistent) or 3x faster. This is the 2TB drive, I do not know the speeds of the smaller versions. As far as exporting content, computationally the new system is 2,500 points higher on the processor for multi core and 1200 faster on single core, the graphics processor is significantly faster scoring 56,000 instead of 11,000 which is where the exporting really comes into its own. With background rendering on export times for short projects is 3x faster, I’m sorry to be the bearer of bad news, but it’s the truth. Go ahead and use one and you’ll be blown away with it. The negative I’ve found so far is the lack of a magnetic charger.

  3. As they say, “You can draw any trend line you want through a single data point.”

    Yes, as in this single example, it is possible to act on huge data files with very limited RAM IF AND ONLY IF the application is only working on a small subset of that data set at any given time. In the example given, even though a data file might be 250 GB, the application is only working on a couple GB of that data at any given moment. Plus, there is little need to rework parts of the data set that have previously been worked — there’s no highly recursive manipulation of that data set.

    Now take an example that I often implement: large data files that need to be worked as many sizeable subsets spread across that large data file and have that work necessarily be recursive, i.e., any subset needs to be reworked multiple times as adjacent and near neighbor subsets of the full data set change. Implementing just that type of analysis on a 16 GB MBP on a much smaller overall data set than 250 GB can get that RAM and SSD thrashing in short order. AND, thrashing a SSD severely shortens its life as SSD have a finite number of reliable writes.

    My data point is just as valid as the author’s. Should Apple make a MBP that can have a maximum of 128 GB or more (ideal for some of my uses) just for people like me? Probably not. People like me are a subset of users. Conversely, should *anyone* (including Apple) support the author’s position that you shouldn’t need more than 16 GB of RAM even for huge data sets? Probably not. People

  4. We can’t argue against ourselves and against our own benefit because Apple is involved. Or can we… Well, we shouldn’t.

    Simply put:
    1-. If the MBP is fast with 16 GB, and is able to handle most user needs it will be even faster, more reliable and capable to satisfy even more users with 32 GB.

    2-. Memory modules these days are cheap, 16 GB are at around $150, and 32 GB are at around $300. Being at a very reasonable price makes no sense to deny the option.

    3.- Also memory RAM modules are the best doing what they do and they do it for life, almost always.

    4.- The discussion is not about what can the MBP do with 16 GB, it is what it will be able to handle with 32 GB. Even bigger files, more applications running fast at the same time, more synergy.

    5.- 32 GB is not rare, unique or extreme in a mobile workstation. It is very desirable and benefits users from many areas.

    In the end why sacrifice an expensive, soldered, no user replaceable part on an expensive machine as it is the SSD in the MBP doing something a cheaper, more reliable and designed to perform the same task should do? Obviously it is not a design gold but a faulty choice/decision/priority from Apple.

  5. Forget virtual memory, just because a file is big doesn’t mean an editor has to slurp it all into memory. In fact I’d probably bet almost certainly that it wouldn’t in the case of video files. You could probably totally disable virtual memory on your Mac and it would still work.. up to a point.. in editing a 250GB file.

  6. That’a a very nice and tantric explanation BUT still is not an excuse to sell such a handycapped but insanely expensive notebook!

    End-customers are paying A-LOT-MORE for LESS!!!!

  7. And what about if I just want to install more f*** RAM just because I am bored and want to do that??????

    WHY APPLE JUST DON’T LET BORED PEOPLE LIKE ME TO DO SO WITH THE NOTEBOOK I PAID FOR??????

    WHAT IF I ENJOY DRAINING THE BATTERY JUST FOR THE JOY OF ADDING ALL THE RAM I WANT???

    Kind of tech-comunism!!!!

  8. The problem is we needed the macbook pro to be the workstation as Apple is not building a Mac pro that we need.
    The extra ram on the laptop would have got us through for a while but if like me you own 2013 macbook pro retina 1tb and 16gb quad core. Then this laptop is an anal exploration is self obsession rather than power.
    If Apple release a proper desktop computer for power users than i doubt there would be so much backlash but this lap top is for (pro users) supposed to be the best work station we are not getting in the form of a new Mac Pro.
    we are all really waiting for a no holds barred and totally serious Mac Pro.
    Please Apple have a Mac Pro for the first quarter of 2017 that will re engage all the power users who are the back bone of all the things Apple claims to champion in creativity.

    Apple should be able to see that we (the creators and artists ) feel stifled by your fashion statements for thin and the narrow minded corporate nickel and dime focus of a few products for profit.
    without the creators of music and visuals (power users) your hardware is nothing but a dead weigh of consumer tech.
    The creators and developers are the oxygen that allows you to create the fire.

  9. More nonsense. 250GB would be a 60+ minute HD ProRes HQ movie at 60fps. I call shenanigans that anybody is “commonly” editing with this as a piece of source media.

    The premise being fabricated, the analysis is similarly suspect.

    MDN, it’s fine to cherry-pick your articles and even to ignore sections of an article you DO cite that are critical to Apple’s products (see the hack-a-thon fiasco), but at least please avoid linking columns that don’t pass the smell test.

Reader Feedback

This site uses Akismet to reduce spam. Learn how your comment data is processed.