“Apple has just issued a software fix through its Beta Software Program to address an issue that arose when the company’s new MacBook Pro laptops were being tested in Consumer Reports‘ labs,” Consumer Reports reports. “Although those computers performed well in tests of display quality, performance, and other factors, we found the battery life to be so variable on the models tested that we could not recommend them to consumers. In our tests of three different MacBook Pro models, we saw battery life results as long as 19.5 hours and as short as 3.75 hours.”
“As a result, these laptops were the first MacBooks not to receive Recommended ratings from Consumer Reports, and the only ones in our ratings of 140 laptops to demonstrate this degree of inconsistency in battery life,” Consumer Reports reports. “We have now downloaded the software fix and are rerunning our battery tests with the fix in place on the same computers previously tested. If the battery life results are consistently high, the ratings score for MacBook Pros would rise, and those laptops will then receive Consumer Reports’ Recommended rating given their performance in all our other evaluations.”
“We communicated our original test results to Apple prior to publication on Dec. 22 and afterward sent multiple rounds of diagnostic data, at the company’s request, to help its engineers understand the battery issues we saw in our testing,” Consumer Reports reports. “After investigating the issue, Apple says that the variable battery performance we experienced is a result of a software bug in its Safari web browser that was triggered by our test conditions.”
“‘We appreciate the opportunity to work with Consumer Reports over the holidays to understand their battery test results,’ Apple said in a statement. ‘We learned that when testing battery life on Mac notebooks, Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache… We have also fixed the bug uncovered in this test,'” Consumer Reports reports. “Apple says the beta fix will be a part of a broader Software Update available to all MacBook Pro users in a few weeks.”
“In our tests, we want the computer to load each web page as if it were new content from the internet, rather than resurrecting the data from its local drive. This allows us to collect consistent results across the testing of many laptops, and it also puts batteries through a tougher workout,” Consumer Reports reports. “According to Apple, this last part of our testing is what triggered a bug in the company’s Safari browser. Indeed, when we turned the caching function back on as part of the research we did after publishing our initial findings, the three MacBooks we’d originally tested had consistently high battery life results.”
Read more in the full article here.
MacDailyNews Take: What did we write about this kerfuffle last month? Oh, yeah:
It’s the software, stupid (Safari, specifically and/or CR‘s testing software). Not Apple’s hardware. — MacDailyNews, December 23, 2016
We look forward to finding out exactly how Consumer Remorse f’ed up their testing this time. — MacDailyNews, December 29, 2016
The problem was Consumer Reports‘ testing methodology. Consumer Reports‘ attempt to blame an obscure bug is a copout.
So, good, an obscure bug found within Safari’s “Develop” tools has been found and fixed. We bet CR will now soon recommend Apple’s MacBook Pro to their geriatric readership. Whoopie shit!
Apple’s statement to CR makes it plain: “This is not a setting used by customers and does not reflect real-world usage. [Consumer Reports’] use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab. After we asked Consumer Reports to run the same test using normal user settings, they told us their MacBook Pro systems consistently delivered the expected battery life.”
SEE ALSO:
Consumer Reports stands by its weird MacBook Pro battery test results – December 29, 2016
Consumer Reports says do not buy Apple’s new MacBook Pro, citing erratic battery life – December 23, 2016
Consumer Reports evaluates iTunes Store movie streaming, confusion ensues – August 13, 2012
Is Consumer Reports having its revenge against Apple? – July 10, 2012
How Apple and facts killed Consumer Reports – March 29, 2012
Consumer Reports was no iPhone killer and they’re no iPad killer, either – March 28, 2012
Tests prove Apple’s new iPad heat levels comparable to Android tablets – March 26, 2012
Expert: iPad heat claims overblown, not a real issue – March 22, 2012
What’s the deal with Consumer Reports and Apple? – March 21, 2012
Consumer Reports’ bombshell: New iPad runs hotter than predecessor but ‘not especially uncomfortable’ – March 20, 2012
FUD Alert: Consumer Reports to ‘investigate’ reports of iPad and ‘excess heat’ – March 20, 2012
Consumer Reports hops off free PR gravy train, officially recommends Apple iPhone 4S – November 8, 2011
Why does anyone believe Consumer Reports? – April 6, 2011
Consumer Reports on iPad 2: We didn’t notice any significant speed improvement – March 15, 2011
Consumer Reports was wrong on Verizon iPhone 4; so-called ‘death grip’ fixed by Apple – March 2, 2011
Consumer Reports: Verizon iPhone 4 has antenna ‘problem’; not recommended – February 25, 2011
Consumer Reports continues laughable vendetta against iPhone 4 – January 14, 2011
Android sweeps Consumer Reports’ rankings as iPhone 4 is omitted – November 17, 2010
All of Consumer Reports’ ‘recommended’ smartphones suffer attenuation when held – July 19, 2010
Consumer Reports: Apple’s free Bumper case does not earn iPhone 4 our recommendation – July 16, 2010
Consumer Reports: Apple’s Bumper case fixes iPhone 4 signal-loss issue – July 15, 2010
Consumer Reports continues harping on iPhone 4 attenuation issue – July 14, 2010
Electromagnetic engineer: Consumer Reports’ iPhone 4 study flawed – July 13, 2010
The Consumer Reports – Apple iPhone 4 fiasco – July 13, 2010
Consumer Reports: Oh yeah, almost forgot, Apple iPhone 4 is also the best smartphone on the market – July 12, 2010
Consumer Reports: We cannot recommend Apple iPhone 4 – July 12, 2010
Consumer Reports does their readership a disservice, says viruses target Apple Macs – December 13, 2005
Consumer Reports: Apple’s new iPod screens scratch-prone like iPod nanos – October 28, 2005
Consumer Reports dubiously finds 20-percent of Mac users ‘detected’ virus in last two years -UPDATED – August 10, 2005
Doesn’t really matter now…..consumers are already brainwashed that they have battery issues along with many others. Running great here though with mine!
What goodness could come out of this:
Maybe Apple will relent and fatten up their laptops with a lot more battery space, IOW improved battery life.
At least we know things aren’t is idiotically dire as Consumer Reports’ bungled report suggested.
Put the skinny little battery in the ‘Air” type models. Put some larger batteries in there for the Pro’s to use. There are times that thinner is not better, this is one.
“We have now downloaded the software fix and are rerunning our battery tests”
Don’t forget your duck tape Consumer Reports, you know, for the interviews.
Good to know..👍.. spread the news …..
These morons turn off a battery saving feature and then complain about battery life….
Do they disable the batteries on a Prius to test gas mileage?
Funny you should mention that since that actually IS a practice to test gas mileage w/o battery.
This reminds me of something……oh yeah, that guy who drove the Tesla model S around and around till it ran out of juice then proclaimed it didn’t make it to the destination, in the background a picture of the Model S on a tow truck. Tesla checked the “black box” and proved what he had done. The reporter was fired.Apple should make CR pay a similar price.
So CR dioes a test wherein they meddle with the software settings, so they are really not testing the product one buys. They are testing a hacked (in a way) product. The product is a compbination of software and hardware, not one or the other. So once again CR proves they do not know what they are doing when they design the tests for the latest technologies. This was not an Apple bug, it was a CR screw up that Apple has now presummably disabled. Beware of fools reporting test results.
CR has a POOR sense of professionalism.
If only they’d taken a remotely scientific approach to testing the MacBook Pro. Yet again, they’ve PIED THEMSELVES.
Not certain how this is a fix. For most of us, erasing the cache upon closing the browser should be a standard feature. In fact it is a security problem if you do not.
Unless they are talking about going back to a web site during the same sitting and treating it as new, I’m not certain how this fixes anything.
Suggest you read the entire article for a start.
The iPhone Doc didn’t miss anything in the article. There are many reasons that a user would turn off page caching. I take issue with the many asshats who disparage CR for running a consistent test. The test is more valid and repeatable than anything anyone else has proposed. Moreover, CR unearthed an Apple bug that indicates once again how shoddy Apple quality control has become. Thank you CR!
If CR does turn off browser caching as a normal testing practice to test systems with various browsers it should also be stated otherwise you’d be ‘favoring’ MacBook Pro battery results over other laptops they tested in the same way and providing non-comparable data. Not to mention also giving Safari a false ‘good’ rating when it comes to conserving battery resources.
They shouldn’t be using hidden features. If they want all pages to reload then they should set up their own server to generate new content and URLs and/or set up some kind of proxy that sets the flag to disable caching. What would these clowns do if the hidden feature disappeared? What do they do on other platforms?
So, they are running tests with settings 99% of the population won’t use?
Now i know I can’t trust consumer reports
You are wrong in your assessment. You should read the true story at ARS Technica:
“To test battery life, Consumer Reports sets laptop screens to a specific brightness level and then loads a series of webpages in the laptop’s default browser (Safari in this case) in a loop until the battery dies. Apple suggests that disabling browser caching for a test like this doesn’t reflect real-world use, but it does make sense for a synthetic test—users will continually read new pages rather than visiting the same static pages over and over again, so Consumer Reports wants to make sure that its test is actually downloading data over the network rather than simply reading cached data from the disk.”
In order to have a comparable test, you can’t pretend every web page can be served up from a disc cache. You have to assume that the user will be downloading new content as he uses a browser. Therefore, the test is accurate, comparable, and valid.
CR uncovered Apple’s dumb battery design choices, so don’t blame the messenger.
A better way to configure the test would be to HAVE a large (but consistent) variety of content to test against, instead of loading the same page(s) over and over.
They were lazy, and their artificial test didn’t reflect real-world usage.
Even if you could have a large consistent set of content to test against, in real life you will run into many dynamic websites that caching even when turned on will not work on. As long as the test set is put into a loop, similar discrepancies would crop up. If you think about it, turning off caching by itself is not the only variable that could cause the huge range in results. Assuming the test was on ‘real’ websites vs an in-house server, the connection itself could affect results by timeouts or delays in downloading content.