Music vs Sine Waves

MEASUREMENTS & AUDIOPHILES: One of the goals behind this blog is to explore some of the more popular audiophile beliefs. Which ones are true, partly true, or completely false? When it comes to measuring audio gear there are many different beliefs but I often run into variations of these three (photo: Leon Wilson):

• Measurements use test signals, not music, so they’re of limited use
• Measurements fail to account for real world usage and loads
• Measurements matter little as you can only trust your ears.

MUSIC vs TEST SIGNALS: Intuitively music is far more complex than test signals. So it’s not surprising many believe such measurements cannot accurately convey the performance of audio gear. But there’s lots of well documented research demonstrating the right measurements using test signals can help predict the sound quality of a lot of audio gear. Some things to consider (photo: Inha Leex Hale):

• Sine Waves - Sine waves are not some abstract signal created in a lab. They’re the primary building block of all sounds we hear. Analogies would be a single color of light or a pure chemical element from the periodic table. All the colors we perceive are combinations of individual wavelengths of light. And everything we experience in the physical world is made up of elements from the periodic table. And, in much the same way, music is just a collection of sine waves. A perfect sine wave is a single pure tone and has no distortion of its own. It's the most pure component of sound.
• Steinways & Yamahas - The note "A" above "Middle C" on a piano strongly resembles a 440 hz sine wave. It's a relatively pure tone at a single frequency. The wood structure, nature of the strings, hammers, etc. all slightly alter that 440 hz sine wave. A Stieinway grand might have a slightly faster attack, a longer decay and a different set of distortion products than a Yamaha grand. These subtle properties are well enough understood it’s possible to simulate the sound of different pianos using software (photo: Mrs Logic).
• Amplifier Distortion – Just as the sound of a particular grand piano can be simulated by understanding its distortion, the same can be done with amplifiers. In the mid 80’s Bob Carver challenged high-end audio magazines and ended up duplicating the sound of a seriously expensive Conrad Johnson tube amp using one of his inexpensive mainstream solid state amps. He did this by simply measuring the tube amp using test signals. The golden-eared editors had a very difficult time telling which amplifier was which despite the massive price difference. There are many other examples that demonstrate the power of proper measurements. If Bob Carver can “describe” and essentially equal the sound of a high-end audiophile amp using measurements that says a lot.
• YACA (Yet Another Car Analogy) – Much as music is more complex than test signals, public roads are more complex than a closed circuit on a racetrack. But when someone wants to figure out if Car A outperforms Car B they take them to a track where they can be evaluated under identical controlled conditions. There are too many uncontrolled variables in real world driving such as other traffic. Few dispute a racetrack is the best overall way to evaluate acceleration, braking and handling limits of cars. A test bench for audio gear offers the same for audio gear. A fair and valid performance comparison is only possible under rigidly controlled conditions—not casual listening or driving.
• Correlation – Lots of studies compare measurements made with sine waves to perceived distortion when listening to music. And, for decades, the research has supported a strong correlation between the measurements and what we hear. It’s not black and white when you’re dealing with human perceptions, but virtually all of the research has pointed towards various measurement thresholds that help define what people can perceive under various conditions.
• Sufficiently Transparent – When there are audible differences between audio gear it’s sometimes difficult to definitively pick a clear winner as that’s subjective and personal preferences will bias the result. This is especially true with things like headphones, speakers, and phono cartridges. But with electronics like amplifiers, DACs, pre-amps, etc. measurements can help a lot. It’s been shown once all the right measurement thresholds are met, the equipment in question becomes essentially “transparent” in the signal chain—i.e. it doesn’t alter the audio signal in an audible way. For example, Meyer & Moran demonstrated you can insert an A/D and D/A operating at 16 bits and 44 Khz into a high resolution SACD signal path and even skilled listeners could not tell when the extra hardware was present. The A/D and D/A were sufficiently transparent and did not alter the sound enough for anyone to detect.
• Myth Busted – Sine waves are not some artificial signal with no basis in reality. They’re  a well proven method to reveal distortion in audio gear and they’re a building block of all sound we hear including music. And audio engineers have more tricks up their sleeves than just sine waves.

REAL WORLD USAGE & LOADS: Some claim measurements fail to account for real world conditions. But that’s all in how the tests are done. Some tests account for a much wider variety of conditions than typical listening tests:

• Test Loads – Proper tests are done using proper loads. And it’s not uncommon to run at least some tests with reactive loads or even real loads. It’s not difficult to model a loudspeaker or headphone driver on a test bench. And it’s easy to compare the results with various kinds of loads including real ones. Hence there’s a pretty good understanding and body of evidence as to how simulated loads affect the performance of audio sources.
• Complex Test Signals – Not all tests are just a simple sine wave. Some involve sweeps or other techniques covering the entire audio spectrum, using multiple tones, impulse signals, chirps, pink noise, square waves, etc. And measuring noise doesn’t require any test signal.
• Worst Case Testing – It’s not that difficult to come up with worst case operating conditions that represent real world usage. I did just that in developing the criteria for the O2 headphone amp. Once you establish the worst case criteria, tests can be run to verify the performance under those conditions. If the gear measures well enough to be transparent even under worst case test conditions, it’s a safe bet it will also be transparent under realistic conditions in the real world.
• Audio Differencing – It’s possible to test many kinds of audio electronics using a method known as analog or digital audio differencing. These tests can be done using real music and real loads (i.e. headphones). In essence, the input and output are matched in level and subtracted from each other. This method was originally put forward by Baxandall and Hafler as a method for evaluating power amplifiers under real world conditions. Differencing captures nearly all forms of distortion and can be quantified objectively by measuring its level and spectral properties. It can also be quantified subjectively by listening to the nature of the difference signal and how unpleasant it is.
• Myth Busted – If anything it’s more common to fully challenge audio gear on a test bench than in listening tests. Just like testing a car on a track, it’s generally best to find the ultimate limits on a test bench rather than in real world usage.

TRUSTING YOUR EARS: Audiophiles put a lot of trust in their ears and subjective impressions certainly matter. But there's a problem. Human senses, including hearing, are greatly influenced by other factors (photo: Travis Isaacs).

• Primal Brain - A primal and involuntary part of our brain constantly filters our senses to avoid sensory overload. For example, your brain automatically filters out other conversations at a noisy party so you can better hear the person you’re talking to. Check out how the brain involuntarily filters what you hear with this brief BBC video demonstrating the fascinating McGurk Effect
  • BBC McGurk Effect on YouTube
• Seeing & Hearing – In the video above, if you close your eyes your hearing is accurate but with your eyes open, your brain deceives you. It turns out when listening to audio gear you have go one step further and remove the knowledge of what gear you’re listening to. Otherwise much the same thing happens—the brain tries to help out and serves up an altered version of what you’re listening to. The objective geeks call this “sighted listening bias” and it’s been well proven in many studies.
• Bed Sheets & Hearing – The simple act of throwing a bed sheet over an equipment rack can make allegedly obvious differences in sound quality disappear. The listener’s abilities, room, music, and the hardware remain unchanged, yet just removing the knowledge of which equipment is playing removes the previously audible differences. This has been proven again and again, even in the homes of audiophiles, by listening tests such as this one: Matrix Audio Test
• Conditions Are Different – Just because a headphone amp sounds good with 300 ohm Sennheisers doesn’t mean it will sound equally as good with 25 ohm Denon headphones. And gear that might sound great with classical music may fall on its face with the next guy’s hip hop. So listening tests are often only valid for a particular set of audio preferences, conditions, type of music, volume, etc. And all those things differ widely from one person to the next. I’m not saying these tests are useless, but they’re highly subjective and very difficult to compare between people and conditions.
• Ears Are Different – If Michael Fremer at Stereophile listens to a piece of gear and declares it best-in-class what does that really mean? It’s much like a wine critic saying the same thing about a particular wine. But the next wine critic will often choose a different wine as he had different tastes. The same is true in audio and it’s a fundamental problem with subjective listening. Everyone’s tastes, preferences, priorities, hearing acuity, listening skills, etc. are different. One man’s “detailed” is another man’s “excessively bright”. So it’s difficult to trust someone else’s ears. And, there are not many stores where you can walk in and audition high-end headphones to hear them with your own ears before purchasing. Measurements greatly supplement these subjective reviews and provide a much better means of comparison.
• No Contest – With other controversial topics, say global warming, there’s typically conflicting research. But, in this case, there’s lots of research supporting the problems with sighted listening and essentially nothing credible opposing any of it. If there’s some problem with blind listening tests, why hasn’t the well funded high-end audio industry managed a single study supporting the supposed advantages of their products and/or sighted listening?
• Myth Busted – The points above explain why it’s difficult to trust your own ears and trusting someone else’s ears is much like trusting a wine critic. Hence the classic “you can only trust your ears” belief is highly suspect. Objective measurements, however, generally can be trusted as they’re immune to all the issues above.

OTHER RESOURCES: If you’re still not convinced the three popular beliefs are more myth than reality, or if you want more information, the following links are worth a look:

• Audio Myths Workshop Video – This is a fascinating video covering everything from human psychology to assorted listening trials. One of the presenters is Ethan Winer who has also made available audio files to allow your own comparisons.
• Science and Subjectivism in Audio – Douglas Self shares his views on this debate in an older but still poignant article. He’s the engineer behind some high-end gear such as the current flagship Cambridge Audio products. He’s published some of the most highly regarded books on audio design in the world.
• Subjective vs Objective Debate – This is my own article covering more of the philosophical differences between the “trust your ears” and “measurements are best” camps. The comments are also enlightening.
• Testing Methods – I share some of my thoughts on how I test audio gear and why. Tests should be done in standard ways to allow fair comparisons between gear and they should be verifiable by others.

BOTTOM LINE: If you want to know the ultimate performance limits of a car you take it to a test track or race course. And if you want to know the ultimate performance of audio gear, you put it on a test bench and use an audio analyzer to make appropriate measurements. Subjective impressions are still important in both cases. For example, the numbers tell you nothing about how easy the controls are to use. But when it comes to determining if the BMW or Mercedes is the higher performance car, measurements offer the best answer. The same is true when comparing audio gear.