First results from project to measure codec quality with computer

vbimport

#1

I just posted the article First results from project to measure codec quality with computer.

Forum member Turing has made an interesting post in the Club CD Freaks Nero SDK Forum. Turing has been working on a project to measure the quality of an audio codec with a computer. He…

Read the full article here:  [http://www.cdfreaks.com/news/10327-First-results-from-project-to-measure-codec-quality-with-computer.html](http://www.cdfreaks.com/news/10327-First-results-from-project-to-measure-codec-quality-with-computer.html)

Feel free to add your comments below. 

Please note that the reactions from the complete site will be synched below.

#2

This is pointless. Why? Well, to start with, a strict comparison with the original wav file is invalid because of a little thing called psychoacoustics. If the comparison is purely based on what information is missing versus what is PERCEIVED by a listener as missing (a purely subjective measurement that is unique to each person), then it’s not telling you anything. Different people hear things differently. Th eonly thing it can tell you is if a given codec is grossly inept. It can’t tell you a damn thing about audio quality. To draw a parallel, this is about as useless as blind ABX testing, another “test” certain types like to drag up as valid but which has no place in the real world that the majority of us live in. You don’t do it in the showroom and you certainly don’t do it in your living room, so what’s the point? Just another example of academics trying to go about quantifying something in ass-backwards fashion.


#3

This is actually adressed in the paper. I hope the author of the study will come by and maybe explain a little more.


#4

Can’t say I agree it’s pointless, there has been many such tests with graphs showing how well the encoders (codec’s) cope converting a wav to mp3 and back, then showing the resulting comparisons from low to high frequency against the original. In the past, Fraunhoffer was shown as poor with a falloff above 16Khx, errors at certain frequncies, and then there were really poor codecs such as Xing (?). Things have moved on - Lame really is good enough for anyone these days. If the decoded audio varies greatly from the original file, then it’s a bad codec. They also compared the same mp3’s in blind tests - on the whole, the codec’s that gave great matches to the original wav’s scored well, while those that differed greatly and had poor top end received low scores.


#5

Graphs and mathematical formulas don’t tell you anything about perceived sound quality of lossy codecs. To draw a parallel, this is about as useless as blind ABX testing, another “test” certain types like to drag up as valid but which has no place in the real world that the majority of us live in. That statement was absolutely pointless … double blind ABX testing (if applied correctly) is the [b]only[b] feasible way one can definitely assess a codec’s quality at a given setting for his personal hearing abilities and listening equipment.


#6

“That statement was absolutely pointless … double blind ABX testing (if applied correctly) is the [b]only[b] feasible way one can definitely assess a codec’s quality at a given setting for his personal hearing abilities and listening equipment.” ROFLMAO. You’ve been hanging out with the pinheads on Hydrogen Audio for FAR too long instead of actuall going out and experiencing REAL audio. In short: Prove It. If you do somehow manage to fabricate real-world empirical proof for that religious claptrap, you’ll be the first to do so and I’ll be the first to say “award the man a Kewpie Doll!”. The simple fact is that you won’t be able to, never mind the pile lofty rhetoric and psuedo-science that will undoubtedly be employed in the attempt to “justify” that line of “reasoning” (it always is). Listen. Carefully. For specific things. Over and over again. If a difference is there and audible on your equipment, you’ll hear it. No double blind BS required. Period.
[edited by Roj on 24.05.2005 12:45]


#7

Be careful… a big placebo effect might creep up behind you and bite you in the ass


#8

More likely psuedo-science will dazzle me with it’s BS before that happens… ABX is relatively new. Ears and listening have been around for millenia. It works, don’t mess with it.
[edited by Roj on 24.05.2005 16:50]


#9

Yeah, right. Are you saying older means better? Come on… You seem to have a very distorted perception of science in general and ABX testing in particular. “Ears” and “listening” is exactly what you use when making an ABX test, nothing else. The only difference is that you try to differentiate what you THINK you hear (placebo?) from what you actually DO hear. It’s as simple as that.


#10

No. I’m saying “tried and true” works. As it has for a VERY long time. And is understandable by all. And easily practiced by all. As it has been for again a VERY long time. And since all I am in fact doing is listening in either case, I don’t need the psuedo-scientific-pop-psycolological bullcrap to somehow “legitimize” it. Do you do ABX testing in the showroom when you go to audition audio gear? Do oyu do ABX testing when you listen to a piece of music and want to pay particular attention to the subtle nuances of it? Didn’t think so. So, if you want Holy Water to anoint the Blessed ABX and believe that somehow the process will “make it right for the world”, I’d suggest you see a priest. Me? I’ll be actually exercising a truly novel concept called “listening to the music”, the same way everyone else can. And should.
[edited by Roj on 24.05.2005 17:31]


#11

You are contradicting yourself. Just listening is not “tried and true”, it is very unreliable, as your listening experiences will be influenced by your mood etc. If this is enough “proof” for you that you actually DO hear a difference between codecs or equipment, go ahead and trust it, but don’t spread false information about the validity of ABX testing, which has been a tremendous help e.g. for the development of audio codecs. Without this evil science, your MP3 or MPC files wouldn’t sound as good as they do. And whoever said that ABX testing should replace the enjoyment of actually listening to the music you like? No one. The only application of ABX is to make decisions about what audio codec or equipment sounds better - it is just a scientific way to improve audio technology and has never, ever been meant as a replacement for enjoying music!
[edited by sTisTi on 24.05.2005 17:37]
[edited by sTisTi on 24.05.2005 17:39]


#12

I am congradicting myself? “But listening is unreliable”. “But listening is all you’re doing with ABX”. Make up your mind, sirra. You have an indefensible position. On the one hand you say that people shouldn’t listen but in order yto use ABX, they have to. I say drop the religion and just listen. Listen to a portion of a piece of music encoded with codec A. Repeat as required to learn its nuances. Then listen again to the same piece encoded with codec B. Are the nuances the same? Binary answer: yes or no. If no, then dig deeper and listen some more and quantify. Describe. It’s so simple a six year old could do it. But the great psuedo-gods of claptrap can’t see this. It’s too simple for them to grasp. Never mind that musical instruments were tuned this way long before they were even thought of, let alone born. No, they have to somehow create a process for it that satisfies THEIR minds, never mind the ears of the listener. The other way is too simple and doesn’t require holy water. I prefer the KISS principle. Unfortunately there are indeed those for whom KISS is entirely too difficult to grasp… Are you one of them? In any event, I’ve wasted enough time on this foolishness. If you choose to bow to the god of unnecessarily complicated processes, that is indeed your right. Feel free to do so. However, I leave you with this inescapable truth: The proof inevitably lies in the ear of the listener and the less complicated that process, the more valid its results.
[edited by Roj on 24.05.2005 17:54]


#13

“The proof inevitably lies in the ear of the listener and the less complicated that process, the more valid its results.” LOL, that sounds like an true advertisement for ABX testing:p Anyway, enough of this discussion as you ignore my arguments anyway and/or seem to have no idea what ABX testing is all about and prefer your faith-based methodology no-matter-what. And, please do yourself a favour and read up on “placebo effect”:wink:


#14

That would do him some good, provided he could actually understand what he tries to talk about. Sorry about the slight trolling. I’ll go and play with Ignore List. Over & out.