Some questioning Power Mac G5 benchmarking

“Analysts and others are raising questions over Apple Computer’s claim that its new Power Mac G5 is the world’s fastest personal computer,” CNET reports. “In his keynote speech Monday at the company’s annual developer conference, Apple CEO Steve Jobs introduced the aluminum-encased Power Mac and showed a variety of industry benchmarks that placed the machine ahead of 3GHz Dell machines using Pentium 4 and Xeon processors. Jobs also showed the Power Mac, built around an IBM chip dubbed the G5, outperforming the Dell on various applications.”

CNET reports, “Apple has won praise for the new Macs as a substantial improvement over prior machines, some have criticized the choices made in putting together the rival Dell machine for use in tests developed by the Standard Performance Evaluation Corporation (SPEC). Apple’s tests, which were conducted by third-party testing firm Veritest, used the same GCC compiler for both machines, with the Dell boxes running the Linux operating system. Critics charge that much higher benchmarks can be achieved using the Windows OS and an Intel-optimized compiler, rather than GCC.”

“‘It wasn’t really a fair test,’ said Gartner analyst Martin Reynolds, who said that the Dell machines are capable of producing scores 30 percent to 40 percent higher than those produced under Apple’s methodology. ‘The reason this happened is Apple had a third party go out and test a Dell under less than optimal conditions,'” according to CNET.

“In response, an Apple representative said it wanted to compare hardware performance, so it made sense to use the same compiler on the Mac and the Dell. The SPEC benchmark tests measure the performance of the hardware and the compiler,” reports CNET.

“‘We set out to conduct and report as fair a comparison as we could,’ Greg Joswiak, Apple’s VP of hardware marketing, said in an interview on Tuesday. ‘SPEC measures a combination of hardware and compiler. The only way to do a hardware-to-hardware comparison is to normalize the compiler,'” CNET reports.

Full article here.

17 Comments

  1. i know, the second we have something that might give microscheisse competition (oh dear, the world is caving in on micro$oft’!), everybody and their grandma running dos 3.0 has to attack us. it’s a window’s centric world – we’re just gonna have to tough it out until they eat their own words.

  2. What amazes me is how miliant the PC side gets over any inovation that little insignificant Apple does….Their comments and claims against Apple more or less make it pretty obvious that Apple is definitely onto something hot once again.

  3. Every system mfr. picks and chooses benchmarks to portray what they need to sell. “Analysts” could “question” any of them with good reason. Unlike some, though, Apple used an independent testing company and revealed full details of the tests.

    More importantly, Apple backed up the benchmarks with a WIDE range of REAL-WORLD app tests. Which came out even BETTER for the G5 than the purely academic SPEC tests did. Better as in over TWICE as fast as a dual Xeon. (You can’t even GET a dual Pentium 4.)

    Wintel companies (AMD?) will come out with even faster chips some day. Then Apple/IBM will come out with THEIR faster chips again. Faster G5s, and then G6s (980s). It’s a cycle, and all that really matters is that the best OS in the world will be in the top range of speed for the forseeable future. Motorola no longer holds the platform back. Let the G4 be the Mac Celoron.

  4. Lets the G5 gets better reviews once reviewers get their hands on one. There is nothing like real world testing to reveal what is fastest. I suspect they will be very impressed.

  5. I think some people are missing the point. Apple compared the G5 to CRIPPLED Pentium 4’s. Disabling hypertransport and SSE2 does not constitute real world benchmarks the way people use their machines. On top of that, they used the standard benchmarks from those machines to compare it to the G5 and it was trounced.

    Either way, the article was definitely a good read and showed that the Wintel crew is still far ahead and AMD was the first company with a 64bit desktop chip, NOT Apple.

    Flame all you want, but this is the truth.

  6. To address Chomper. You are wrong. Apple IS the one with the 64 bit chip for the DESKTOP. AMD is NOT (they dont make desktops with a Opteron chip only workstations. A workstation is NOT a desktop. Go to the website!). Furthermore the Power4 by IBM processor has been out for a while WAY before AMD’s 64 bit chip which is what the 970 chip is based off of.

  7. Of course they’re going to use GCC on both boxes, and therefore, Mac OS and Linux. If they’d used the Intel optimized compiler, it’d be like trying to figure out which computer is faster by running one with Photoshop and the other with CorelDraw. It’s not going to tell you anything unless they’re both running the same software. It doesn’t take a brain surgeon to figure that out. Perhaps the Intel optimized compiler does work faster than GCC on Linux, but since it’s not available for the Mac, how exactly would they test them? That particular analysis wasn’t designed as a real world test, it was a scientific hardware test. In the real world tests they did, the G5 was still better. Why is anyone surprised by this? Look at what they did to that box! The G5 out-performs the P4 chip to chip without anything else being changed. We knew this from the day IBM announced it. Now with all of the other new architecture in the box, why would you have any reason to doubt it. PC lovers just can’t stand it that Apple now has the lead. It’s understandable I guess. Faster chip was all they had. Now that’s gone too.

  8. I think we’re talking semantics here, if an Optetron workstation is cheaper to build than an Apple Desktop and is more powerful, then who cares what it’s called? My point is that it seems Apple was fudging a lot of things during the keynote.

    In the end, it comes down to price right now, and the performance is obviously not as much in the stratosphere as we like to think.

  9. What continues to blow me away is the amount of energy expended on boards like this, press booths, executive committees, and the average consumers mind, about these insipid numbers. But in a lowest common denominator culture I suppose that’s to be expected. It’s not the box, it’s not the numbers … it’s the apps and the experience of using same that count. I hate MSoft as much as anyone but we all know Steve would be just as Borg-like given the opportunity. Get over the numbers. Get over the fanboy zealotry, and get real. I use Mac’s, I prefer Mac’s, I advocate Mac’s but I do it by being real about it … not by spewing numbers.

  10. True, it is all about the app’s. I found a comparable BTO Opteron Workstation for $100 more than the $1999 Apple model.

    But again…doing video…nothing compares with Final Cut Express OR Pro in the PC world. How fast the render is, is negligible between the two platforms. It is what I find which has a better workflow to get the job done. That is the Final Cut applications. It just feels right compared with the PC app’s that I use at work.

  11. Wasn’t Sun Sparc 64bit CPU the first in workstations 10 years ago?

    Will Steve Jobs be releasing more comparisons VS. Intel Extreme chips and AMD 64’s?

    Will he again use a dodgy custom malloc library?

    Why are Spec results available for the comparison machines that are 40% higher than those Apple paid Spec to run?

    Why was Apples G5 advertising campaign outlawed by fair trading agencies around the world?

    Why don’t you guys argue the facts instead of proposing paranoid conspiracy theories?

    Its a great CPU and OS BUT the overhyping by Apple has reached ILLEGAL status in some countries.

    Apple has dragged the chain for years with hardware development, now when they catch up they deceive all those who stuck with them over those years. It sucks, and if you wanna roll over for them then good luck to you!

    Lazza.

  12. Oh, and for the record, the much Mac-hyped ‘tell all’ SpecFP2000 scores (higher is greater):

    DUAL G5 2Ghz cpu: 800
    P4 Extreme 3.2Ghz cpu: 1583
    Athlon 64FX – 51: 1371

    Lazza.

Reader Feedback

This site uses Akismet to reduce spam. Learn how your comment data is processed.