“Blaze Software performed a huge study on browser performance, loading 1,000 different Web sites in the Android and iPhone browsers and taking 45,000 measurements. It says that the results conclusively prove that the Android browser is 52% faster than the Safari browser in the iPhone,” Eric Zeman reports for InformationWeek.
“In order to perform the study, Blaze created its own custom applications — one for the iPhone and one for Android — to measure page load times on the smartphones. The apps ran on the actual devices, loaded a page on demand, and measured how long it took,” Zeman reports. “And therein is the problem with this study — it didn’t use the actual Web browsers. It used the browser’s rendering engines, but through this custom application. That significantly undercuts the results.”
Zeman reports, “As Gartner’s Michael Gartenberg explained to me via Twitter, ‘For one thing, Mobile Safari was not used, so the new JavaScript engine wasn’t being measured. Safari is how users browse the Web. More importantly, things like 52% faster are misleading when dealing with times of a second or less. In real world that’s margin of error.'”
Read more in the full article here.
Jim Dalrymple reports for The Loop, “A Canadian software company, Blaze Software, released a report today of what it calls the ‘largest ever research study of smart phone browser performance.’ The problem is, the study is flawed.”
MacDailyNews Take: Bad day for our Canadian friends: First the NYT makes you their guinea pigs and now this mess.
Dalrymple continues, “According to Blaze’s own documentation the ‘measurement itself was done using the custom apps which use the platform’s embedded browser. This means WebView (based on Chrome) for Android, and UIWebView (based on Safari) for iPhone.’ The problem with using UIWebView is that, even though it’s based on Safari, it didn’t receive any of the updates that Safari did in iOS 4.3. Using an embedded browser is not the same as using the official browser.”
“Apple’s Safari Web browser included the Nitro JavaScript engine that Apple said runs JavaScript up to twice as fast as its predecessor,” Dalrymple reports. “Since UIWebView didn’t include any of those enhancements, it’s kind of disingenuous to say that Android beat Safari.”
MacDailyNews Take: No “kind of” is necessary, Jim.
Read more in the full article here.
MacDailyNews Take: So, why didn’t they use the real browsers? Well, perhaps because when the real browsers are used, it can’t be claimed that Android phones browse “faster” than iPhone?
For some reason, this MacDailyNews opinion piece from over four years ago popped immediately to mind:
If you thought that iPod and iTunes was subject to FUD, you ain’t seen nothin’ yet, my friends. You’ll have to look to the Mac to find a threat of such magnitude that inspired such a FUD campaign. The reason for such a campaign against iPhone? Money. Lots and lots of money and the fear of losing a good portion of it to Apple.
You can call me conspiratorial. You can call me a crackpot. Call me whatever you like, but I will be proven right soon enough…
The other phone makers, the other mobile device makers, and the other makers of so-called “smartphone” software understand the massive threat Apple’s iPhone poses. They have no recourse but to start up the FUD campaign, desperately hoping to slow Apple’s assault on the market. There is so much money at stake that things will get very nasty, very quickly.
The chits will be called in and the articles will get written. It’s already started.
So, keep this in mind whenever you read about Apple’s iPhone and you see an article slanted against the iPhone: the real Fear, Uncertainty, and Doubt is being felt by all of the companies that Apple just humiliated yesterday. They are very scared and rightfully so. – SteveJack, MacDailyNews opinion section, January 10, 2007
[Thanks to MacDailyNews Readers “John A.” and “Manny S.” for the heads up.]
Many unethical people out there!
Any bets that these Canadian guys are connected to RIM?
My first thought was “who benefits” from this “comprehensive”. Blaze Software? Only if someone paid them to set up a test that would deliver a predetermined outcome. I like the RIM theory. Make it look like it’s an Android plant to point the conspiracy theorists like us elsewhere. The main point to take Apple’s iPhone down a peg. RIM benefits.
Ever wonder why Apple passed google long time ago? because apple does not have to pay “reviewers” or “ANALyst” to speak very good about their products.
From what I see in this “Study”, google does.
Rounding Error
Bogus (not flawed) study. They also disable flash on Android to skew the results further in favor of the junk droid knockoffs.
I think they also forgot to mention that they were using the Android phones on WiFi and the iPhone was a 2G using the Edge network. And it was still less than a second off. 🙂
That’s ok. Fandroids need all the “proof” they can muster. It wouldn’t matter to me if Chrome on Android was still faster, but at least do a true test of browsers, but web rendering in a 3rd party app.
I think time to move out Canada. Not all Canadian are nice people, there lots of Canadian Company’s who will mislead the public, look at our federal government.
Thank God is St. Patty’s day, go wash down this week with a few cold beers and start anew next, May 25 ipad2 yeehaw.
Sounds like you started partying early. The iPad 2 is being released in Canada on MARCH 25th.
… that’s bad, or the folks who designed it? Given the number of discrepancies and diluted efficiencies, I’d have to conclude it’s the people who designed it – designed it with this result in mind. But, I could be wrong. Possibly. It’s just that, I’m really used to being right!
I see on a lot of these kinds of “comparison tests”, they wait till the web page is fully loaded to mark the time. I notice that the iPhone/iPad sites load enough to be readable/usable when the task bar is still not complete, but others are not always usable until fully loaded. I think that is an important distinction they miss. Either way, my iPhone 4 and iPad 1 are very fast at loading load pages, so this is a useless comparison between the the two platforms.
Old Maxim:
Figures don’t lie, but liars can figure.
Just like junk science biostitutes, I’m sure there are peeps out there who can rig up a study that will show a TRS-80 can run PAC-Man better than a modern computer. Edges up methodology, but still doable with enough caveats.
Edges should be Effed. Damned auto correct.
I thought they used the actual browsers. I need to read this again.
oh crap. not this again. the speed wars.
The underlying assertion of the so called “test” is more dishonest than the bogus test itself. They essentially aspire to give the reader reason to believe actual user differences are negligible, so pick whatever you find most visually appealing.
It’s a lie:
Just ask all the disappointed Xoom users who have been slapped in the face with a tablet that crashes over and over. (yeah, that’s a stat that got panned over too, I’ll bet. I’d lay odds that despite the handicap, the iOS devices performed without crashing or locking up and doubt the Android ones could make the same boast.)
I saw the headline on Google News about the speed test and thought to myself “I’m gonna wait to hear from MDN”. Sure enough, it was much ado about nothing.
——RM
http://free.avg.com/us-en/homepage
this is sad!!!!!! go down the page and there’s an add for android anti-virus security!!!!!!!!!! epic FAIL
Who cares ! I love my iPhone regardless of its speed or what others say. I wouldn’t trade it for anything else (may be a new iPhone 5!). Blaze is a small Ottawa company with 10 people. cheap way of getting free publicity, ie. trash a mac product.
This looks to me like a typical engineering mistake. Engineers always look for a way to automate a test and remove human error. In this case they ended up using a different javascript engine and you also miss out on the advantages/disadvantages of a better UI.
Can someone explain why everyone assumes mobile Safari uses a different Javascript engine than an embedded UIWebView? Supporting two Javascript engines in iOS would be pretty ridiculous.
That being said, not using the actual browser they’re purporting to test is still stupid.
Perhaps the difference is that the iPhone4 has 72% more pixels to calculate and render to the display than either the Galaxy S or Google Nexus phones have to handle. That probably takes a tad bit longer to do than the more graphically crippled Android phones have to do… Just a thought… from one of us who realizes that all things on these phones are NOT EQUAL and thinks these things through to their logical conclusions.