Gunnar Van Vliet, self-described Mac user and music lover, wanted to know how good AAC was in comparison to MP3, and finally to see if it could come close to standard CD. So, Van Vliet encoded the same track in iTunes using 96, 128, 160 and 192 kbps AAC and MP3, and one AIFF for reference. The track is from the Kansas City Soundtrack – “I Surrender Dear.” It’s a very well recorded live in the studio jazz piece and it’s a track that Van Vliet says he knows very well. It features a solo saxophone and trumpet which are clearly localized in the mix and very closely resemble the real instruments.
The usual caveats of testing apply. This is an unscientific test and it’s not double blind, but Van Vliet thinks that this test can be of use if you’re trying to compare something to a known reference. Van Vliet’s system consists of Yamaha RX-595 receiver, CDX-490 cd player, and Energy Veritas 2.1 speakers. He describes the system as “well balanced and revealing.” Van Vliet told RecordStoreReview.com, “I’ve compared it to many other systems over the years and it never disappoints.”
Van Vliet’s test results:
128 kbps MP3 – Flat, compressed sound/dynamics. Rolled treble (quite bad). One dimensional, plodding bass.
– Tonal Accuracy – 5/10
– Imaging/Soundstage – 5/10
– Naturality – 4/10
– Musicality – 4/10
– Total – 18/40
128 kbps AAC – Rolled treble, but not too bad. Light bass especially in transients/impact. Compressed dynamics. Surprisingly musical.
– Tonal Accuracy – 7/10
– Imaging/Soundstage – 6/10
– Naturality – 6/10
– Musicality – 8/10
– Total – 27/40
AIFF: Beautiful sparkle to piano keys. Generally filled with much more life and atmosphere on a tactile level. Far more musically involving. This is the reference piece, so it naturally gets a perfect 40/40 score.
On the whole, there weren’t any surprises. Van Vliet’s observations echo what most people have said about AAC vs. MP3. AAC is higher quality at the same bit rate, so you can use a smaller file to achieve the same quality as MP3 which is a good thing for portable and computer users. Ultimately, both formats still sound pretty bad in their practical ranges compared to CD.
Also, Van Vliet didn’t test 256 or 320 kbps because it’s impractical for most users to use these encodings. The Apple Music Store uses 128 kbps, and Van Vliet suggests that if you have room for 320 kbps and you care about sound that much you’ll probably use AIFF or just play the CDs themselves.
Full article and results using 96, 128, 160 and 192 kbps AAC and MP3 here.
The utter lack of imaging is what makes all these compression schemes painful to listen to on a high-end system. For background music, or for listening to on the go in noisy environments, though, they’re just fine.
I’m not a music expert, but I am a kick-butt designer, art director and creative director.
If you want a clean crisp digital image then you need as little compression as possible — or no compression — and a lot of data, which means big files.
MP3 and AAC are about being small and being “close enough” — much like a JPG.
A 72 dpi RGB Jpg with a small K-size is good enough for the web page (MP3 and AAC)– but for print you want a 300dpi or more CMYK image which will be a few MBs on up to a few hunder MBs (AIFF).
The listening tests are likely flawed. You need to listen to all material at the *exact* same level, else the ears will distinguish timbre differently at even the slightest variences in volume. Unless Van had three different CD players piped through his receiver (with MP3/M4P burned to disc), he performed his listening tests with different DA converters (computer output VS. high-end CD player output) and at different volume levels.
Moreover, you need to make sure that the Quicktime/iTunes volume is not maxed out, introducing slight distortions. (lower the volume in the app and adjust the overall system volume)
I initially thought I heard major differences between formats, but when volume levels were adjusted accordingly, things evened out. (though I suspect different music tracks will still encode better or worse than others…) The best way to compare is to set up several Quicktime windows you can TAB-key back and forth with. This set up makes it much easier to compare codecs (and volume levels when setting up) You should also use high quality headphones for optimal listening tests.
HOGWASH.
Im so tired of people posting their subjective opinions about the sound quality of anything audio.
There is simply no need to do this. I did a paper on this subject when I was in college. Anyone who’s interested in doing a real comparison, needs to look at the digital file. Not what is comming out the end of your stereo as measured by the human ear.
There are wave form analysis tools available for the Mac that should make this comparison easy for anyone interested.
You can then define the loss of the copy in actual percentages compaired to the original. Also, do not be fooled into thinking that the sounds outside the range of human hearing are unimportant. While you cant hear these sounds directly, they do have an impact on the sound waves you *can* hear.
So the gauntlet is thrown… Someone do this – im going to get a beer.
Oh and on a side note. The guy who did the “testing” here sounds like an “audiophile”. While working on my paper, I found that the “audiophiles” who were convinced they could tell a CD from an LP or Cassette (this was before MP3s) actually had the worst hearing of my sample pool and were the least likely to accurately tell the difference between sources. My guess is because they have a habit of listening to very loud music and have damaged their hearing. Im not saying that this is the case here, but its something to think about.
Dave, doing wave analysis is not the point. Most listeners do not care about what the sound wave looks like.
The human ear is not perfect. Music compression takes advantage of this. In fact, the whole point is to remove the sound data where the human ear is least sensitive.
Sound tests are the only way to go, because they are the only thing that counts.
If the listener/reviewer is aware of the specifics of the source, the ‘test’ is necessarily flawed. You’d have to have a listening room into which music was piped from a control room, with an engineer sending in the sounds from random sources for this to have any validity.
We’re talking about differences at the level of nuance here, and subjective judgements can easily be swayed hugely by simple knowledge of the technical details of the source.
I am a pro audio engineer and I hate these horrible audiophile people. I started my career years ago repairing expensive hifi for these sorts of people and I was always amazed about the amount of crap they talk.
In the old days they used to talk the same way about cassettes I recall. Stupid really as they WERE awful and not worth bothering with.
However, he’s right about the AAC and Mp3. Hi-end roll-off, subsonic filtering are applied to reduce the load on the algorithm – he detected that. MP3 has bad stereo imaging and lumpy bass, pre-echo whereas AAC copes with these things much better by including two auto-switching encoding methods (for quiet and loud basically). The result is a much better sound which he noticed too.
AAC’s are much more ‘fluent’ (to use an audiophile word) in musical presentation that MP3.
I suppose I agree then – what a wanker!
> Unless Van had three different CD players piped through his receiver
> (with MP3/M4P burned to disc), he performed his listening tests with
> different DA converters (computer output VS. high-end CD player
> output) and at different volume levels.
Wrong. He encoded the CD into AIFF and played it from his computer through the exact same D/A convertors that he played the MP3 and AAC files. He then encoded the same CD into MP3 and AAC. Any theoretical volume changes would be due to the MP3 and AAC encoding process, and so they are a valid part of the critique. And in any event, the differences in 128 Mbps AAC are frighteningly obvious- it sounds thin. Why don’t you listen yourself?
> I suspect different music tracks will still encode better or worse than
> others
That is true, different music tracks WILL encode better than others- highly compressed pop music styles will not show the problems as obviously as well recorded acoustic tracks with a lot of air around the instruments. This is mostly because the source material is just not that great sounding. But 128Mbps is just too little data for good sounding AAC in any music style.
> You should also use high quality headphones for optimal listening
> tests
Most music is not mixed to be played on headphones, it is mixed to be played on speakers. You may hear some nuances from good headphones that you wouldn’t otherwise hear (the speakers are extremely close to your eardrums after all), but it makes for artificial listening tests.
It’s only a worthwhile exercise if he does these listening experiments “blind”, that is, he doesn’t know what he’s listening to when he’s listening to it. But the fact that the AIFF gets an “automatic” 40/40 suggests that this was not the case.
I know it HAS occurred to some of you but is lost on most but . . . If you really want to listen on a high-end system it’s obvious you want the uncompressed source. Digital files are intended for playback on computers and digital players like iPods, and these AAC files sounds quite good on them. That MSNBC wanker who converted the AAC files to CD and compared to the original CD and complained, well what was he expecting? And these audiophile wankers are the same folks who complain about CDs sounding crappy, they want to stick to those big vinyl disks (whatever THEY were called :-~ )
Ahh, but it doesn’t matter what the sound waves look like or anything like that. Who cares. When I’m listening to a song, all that is important is how it sounds to my ear. Why should I care if one type of song file is somehow “better” if I can’t detect the difference. Although using listening tests appears to be not as technically perfect as other tests, it measures exactly what you should measure – how the song file sounds to us. I do agree, however, that the tests should have been done with the listeners being blind to which file they were hearing.
So–the music store is going to get Cubed. Great.
Steve
As a TRUE audiophile, I find that even CDs are nowhere near good enough quality for me… I insist that the original musicians come and play the music live for me. Sometimes it gets a little crowded with the New York Philharmonic in my living room. Not to mention having them along when I go jogging.
SHN offers a lossless compression alternative for those who would like to cut down a bit on the size of those AIFF files. There are even realtime player plugins available – links and more discussion on SHN encoding at http://www.etree.org/macos.html
I think there will be a higher bit-rate option from iTunes in the future. I’m guessing that the “good enough” quality is to resolve fears of the digital copy being “TOO GOOD”. If converting to AIFF and then to another compressed codec sounded AS GOOD AS a cd, then I don’t think Apple could have gotten iTunes of the ground.
The other issue is that the Itunes web purchase is pretty simple, and I’m sure they didn’t want to have “needless options”. Most people will find 128k GOOD ENOUGH. But I would prefer the higher option–but that just ain’t gonna happen until the labels are comfortable with iTunes. Once they see that people are willing to pay for convenience and quality, I would think that they might relax the quality restriction.
But no guarantee, because the Labels been pretty idiotic and petty so far, so we’ll see.
Digitally recorded music sounds SO artificial. If you want really good sound, it’s got to be live! The sampling and compression involved recording to CD cuts out so much of the original signal, there’s just no comparison. Real audiophiles only listen to live music. Accept no compromises!
There already is higher bitrates available. You can make up to 320 kbps AAC files right now on iTunes 4.
Preferences – Importing – Setting: Custom (then choose anything up to 320 kbps)
Disregard, I just read the rest and realized you were talking about music from the iTunes Music store.
Ever notice how when you go into a Circuit City to buy a TV that it’s easy to see certain differences, and finally you narrow it down between two and there is $100 difference between them. Always go for the cheaper route — you’re not making the comparison once you get the set home because it’s the only one you’re looking at.
I defy all of these testers to accurately identify which is which AAC or original if given hours between the listening/identification of one and the listening/identification of another.
all over the web now, self-appointed aac/mp3 comparitors
are pontificating on this as if it’s never been done before.
really now! are dolby/lucent/sony/fraunhofer just a bunch
of pikers who don’t know audio? try the following link
http://www.tnt.uni-hannover.de/project/mpeg/audio/public/w2006.html
(in turn pointing to the complete w2006.pdf report) for
multi-golden-ears tests. by now, you’d think a few world-class
laboratories are hip not only to the vagaries of “pre-echo”,
but actually know how to use princen/bradley methods
to deal with it. 128 kbps .mp4 stereo audio is mighty fine,
so get used to it!
SO I guess some people like AAC and some people prefer AIFFs (myself included in the latter camp). Why do people expect compressed audio played on high end systems to sound good? Why do people expect AIFFs played on crummy little computer speakers to sound good? Must there always be a “mine is better than yours” discussion? Who cares?
All of you satisfied with, and defending the qulity of 128 bit AAC file, should but ’em. Me, I can hear the diff and until Apple offers a higher quality file other then the one I paid a buck for just to try, they can keep their music. I have no need for it. And all of you who want to tell me I can’t hear the differance, you are all dumber then my dog. I can “hear” the differance and I would rather “own” a quality file for a buck.
Michael’s analogy is appropriate. Instead of findind fault with formats such as mp3 and AAC, people should dwell on what each format does RIGHT! These compression schemes do a pretty credible job of providing a reasonable sounding copy of the original audio. They were never meant to provide a perfect copy. How spoiled we have become! For all of the audible faults in the mp3 and AAC formats, I still prefer both to analog cassettes with all of their high frequency compression problems, dropouts, alignment problems, Dolby mistracking, etc, etc, etc. Let’s give credit where credit is due. Both mp3 and AAC do a remakable job for what they are… lossy compression schemes.
Why doesn’t someone try the straightforward approach? Compare the Fourier transformed outputs from mp3, aac, and aiff and see what is lost in the frequency spectrum.
The first poster suggests that looking at a fourier transform will resolve the quality issue.
This is nonsense.
If a particular frequency component is preserved, that does not mean we can actually hear it, so it does not necessarily affect the perceived quality. The encoders target perceptual effects such as masking, by which a particular frequency component may or may not be salient depending on what else is happening.
Try to think more carefully about what you’re saying.
Why not just look at the waveforms and see what’s missing?
This is really no better or worse than a Fourier transform-
in both cases you may see missing details, but you cannot conclude whether how much this affects the perceived sound quality. The Fourier transform is an information preserving linear transform, of which there are an infinite number in face; none of them match what human perception says.