Measuring Audio Product Performance

Why do audio products sound different, and how can we examine the differences? It is crucially important to compare audio products under controlled conditions to quantify their performance. Here’s how.

WORDS BY DAVID MACKINNON

Over the trio of decades that I’ve been in the mobile enhancement industry, I’ve seen, touched and heard thousands upon thousands of different products from hundreds of companies. I lived through the neon speaker wire and tweed-finished installs of the late ’80s, the shiny fiberglass and neon of the ’90s and the layered look of the late 2000s. In that time, I’ve had the honor to audition, judge (through IASCA) and formally review more products than I can remember. I consider myself fortunate and downright lucky.

When I was reviewing a lot of gear, my favorite listening sessions were with amplifiers. I’d amass a half-dozen amps from different companies and head to my friend Brad’s house to listen to them

back-to-back with his Gallo Acoustics speakers and Conrad-Johnson CD Preamp. Unbeknownst to me at the time, listening to multiple amps in a single listening session afforded me a unique opportunity to not only experience each product but also to directly compare the performance of all the products in the test group. The test reports stood on their own, but the listening experience was incredibly valuable in learning how to audition audio products. Thank you to Sound Wave Customs of Virginia Beach, Va. for providing build photos for this article.

Head to Head Battles

About five years ago, I was talking to a friend about a line of supposedly premium amplifiers and mentioned I hadn’t formally auditioned one of their high-end solutions. He offered to let me borrow one of his and sent it over a few days later.

I took the amp home and wired it into my reference system and sat down for a listen. It was very nice to listen to. The sound was warm, smooth and inviting. After about a dozen tracks, I started to pack everything up when I realized I should immediately listen to the same tracks on my reference amp. So, that’s exactly what I did. The differences were stunning. My reference amp was what I would consider transparent. Percussion was violent and detailed. Vocals were natural and crisp. Most importantly, the sound didn’t seem to be warm. The music sounded more like a live performance and less like listening to it on a stereo system. This experience sparked a fervor to figure out why audio products sound different.

Measurements Versus Listening

I’ve heard dozens of people state that measurements can’t tell you what something sounds like. For example, If you look at an RTA measurement of a car audio system, you don’t know if it images well. In fact, you can’t even tell if both the left- and right-side speakers are playing. If someone provides a directivity plot (that shows frequency response relative to the microphone direction) you can get a sense of where the sound is coming from. There are impulse response graphs that will give you an idea of how different frequencies arrive at the listening position and measurements that characterize the reflections of a listening environment.

In the case of amplifiers, speakers, source units and processors, I think the reality is that most companies and publications don’t offer enough measurements to let educated enthusiasts form a valid opinion.

Let’s talk about a few amplifier measurements that can provide a unique insight into the design and performance of a product. We are all used to seeing frequency response graphs. They show you the output of a device relative to a flat-response test signal. The objective is to quantify the upper and lower frequency response limits of the device and ensure that the performance is smooth and flat through the region where you’ll be using the device.

If everything in the graph is smooth, then you’d expect that the amp will sound pretty good. The problem with a response graph is that it doesn’t show you how different frequencies interact with one another.

The frequency response of one channel of a high-quality Class D amplifier.

How Come Nobody Talks About Distortion?

When I was coming up through the ranks and learning my craft, I used to think that distortion was something that only happened when you pushed an amplifier to the point of clipping. If you were looking at a sine wave, the tops and

bottoms of the waveform would be flat due to the rail voltage limitations of the amp. We have all heard distortion caused by clipping. It makes your music sound garbled and messy.

The reality about distortion is that it happens even at very low audio levels as well as when you attempt to get more power out of your amp than it was intended to produce.

Before we discuss when and why distortion happens, we need to talk about what distortion is. Distortion is caused by the non-linear behavior of components in a circuit. Even something as simple as a resistor adds a microscopic change to the signal that passes through it. Think about all those electronics bouncing around randomly inside the device. Yes, it’s small, but it exists.

Passing a signal through a transistor or MOSFET to amplify a voltage involves a lot more electrons and presents the chance for different responses to different amplitudes and frequencies. Again, the changes in a well-designed circuit may be minute, but they still exist. With that said, each part of your amplifier, from the differential input op-amps on the input stage, through the crossovers, the driver stage and the output stage each offer the opportunity to muck up the audio signal.

Distortion is the addition of information that was not present in the original signal.

There are two key kinds of distortion: Harmonic and Intermodulation. Harmonic distortion is the creation of multiples of a frequency. Say you feed a 50Hz tone into an amp. You may get a very small amount of 100Hz, 200Hz, 400Hz and 800Hz output. Harmonics in higher frequencies are likely buried beneath the noise floor (background noise) of the amp.

A theoretical representation of how Harmonic Distortion works. Intermodulation distortion is created by the interaction of two signals. It’s the mixing of multiple tones to produce sums and differences. The most popular test for Intermodulation Distortion (IMD) is to play two tones, typically at 19kHz and 20kHz at equal levels, and measure the spectral output of the device. The first thing to look at from a perspective of IMD is the creation of a signal at 1kHz—the difference between 19kHz and 20kHz. Likewise, there also will be new signal content at 18, 17, 16 and maybe 15kHz on the low side of the stimulus signals, and 21, 22, 23 and maybe 24kHz on the high side. The presence and audibility of these side-band signals depends on the amount of IMD the amplifier design produces.

It’s Not Warm, It’s Distortion!

The examples above demonstrate that what we get out of a device may not accurately represent what we put into it. For Harmonic Distortion, low-frequency content creates higher-frequency harmonics. For Intermodulation Distortion, high-frequency content can create significant lower-frequency information. So, how does this translate into our listening experience?

Think about what someone is talking about when they describe a device as warm. In most cases, they are referring to the presence of more information in the mid-bass region. If we are listening to a band with a vocalist, guitarist and drummer, there will be a lot of musical energy through the entire audio spectrum. Each instrument or word comprises its own harmonic content. This is what makes one person sound different from another or an upright bass sound different than a guitar, even when singing or playing the same note.

If your amp produces harmonic distortion, then it adds more high-frequency content to that signal. In extremely

A theoretical example of Intermodulation Distortion in a moderate-quality solid-state amplifier.

An extremely high-quality Class AB home audio amplifier reproducing a 50Hz sine wave.

A high-quality home audio Class D amplifier reproducing a 50Hz sine wave.

This Class AB amplifier produces a 1kHz tone that is about 80dB below the 19 and 20kHz test signals. This 1kHz would be inaudible. The side-band distortion is extremely low as well, peaking at -120 dB relative to the -6dB test tones.

generalized terms, tube amps add moderate amounts of even-ordered harmonics while solid-state amps add small amounts of odd-ordered harmonics. Again, that’s a gross generalization and the reality depends on the design.

Here are comparisons of three significantly different amplifiers and their behavior when fed with a 50Hz test tone.

You can see that the 50Hz test signal produces significant additional output in the 100 to 200 Hz region. In the case of the vacuum tube amp, even though the level is more than 40 dB below the test signal, it would change the overall tonal balance of the instrument or voice. For the solid-state Class AB and Class D amps, the harmonics are 110dB and 85dB below the original signal, and would likely be inaudible.

If you read high-end home audio reviews, then you’ve perhaps encountered a comparison discussion of vacuum tube amps versus solid-state amps. The writers will always say the tube amps produce even-ordered harmonic distortion and solid-state amps produce less-pleasing odd-order distortion. Debate after debate has raged on the Internet that this only matters when the amp is driven hard. The reality is quite the opposite. The distortion exists in some amount at all drive levels.

Now let’s look at how these three amplifiers fair in terms of their Intermodulation Distortion performance.

Not All Amplifiers are Created Equal

These two simple tests tell us a lot about the distortion characteristics of these amplifiers, and subsequently how and why they sound different. Please, don’t take this comparison of technologies to demonstrate that all Class AB amplifiers sound better than all Class D amps and vacuum tube amps. I’d take a well-designed Class D over a poorly or even moderate-quality Class AB amp any day, and one of the vacuum tube amps I have reviewed remains one of the most enjoyable listening experiences I have ever encountered.

What I want you to take away from this is the understanding that there are good, mediocre and bad products in every category in our industry. I’ve tested source units that suffer from significant IMD and THD and I’ve listened to them compared to units that measure favorably. The difference IS audible. Signal processors have the same characteristics. Some are very transparent while other seem to blur the sound. Guess what? Speakers do it, too! I had a great Facebook Messenger conversation with a store owner a few weeks back, and we were talking about different speakers. He asked what I thought

This Class D amp doesn’t fare as well. You can see the side-band and 1kHz difference are at about -77 dB. The levels are very low, but you can see how much distortion is created.

Our vacuum tube amp noise floor becomes very visible in this test. The sidebands and difference distortions are only down 50dB from the test signals.

Fundamental, second-order and third-order distortion measurement of the JBL 4367 home loudspeaker. Source: JBL 4367 White Paper.

about a specific brand and I told him that in every case when hearing them, I thought the midbass region sounded bloated, over-emphasized and unnatural. He replied that he felt that the emphasis gave the systems a warm sound.

Speakers are notorious for even and odd-ordered distortion, especially at high drive levels. Problems with changes in suspension compliance, magnetic field strength and inductance can wreak havoc on the output of any kind of speaker.

Think about what happens if the suspension of a speaker is more compliant as the cone moves rearwards as opposed to forwards. What if the magnetic field strength changes relative to position, and is stronger moving backward as opposed to forward?

Most speakers suffer from significant changes in voice coil inductance based on cone position. This change in inductance changes the high-frequency response

The second image is with the mic in the same position, but with the volume turned up a little bit. This increase in output level (by about 8dB) dramatically emphasized the distortion at 2200 Hz (which I can hear as buzz) and showed an issue around 170Hz.

of the driver and produces significant Intermodulation Distortion.

Will either of these (very common) scenarios result in linear cone motion at all drive levels and in both directions? The answer is a resounding NO.

I have yet to see a mobile audio company publish distortion measurements of their speakers. Perhaps it’s because they don’t know these measurements exist. Perhaps it’s to prevent consumers from directly comparing the performance of their solutions with other brands. I stumbled across one such distortion measurement graph a few years ago on the JBL Synthesis home and pro audio website. I was window-shopping for things I can’t afford and reading some of their information sheets and white papers. Being ever-so moderately geeky (ahem), I was interested in the distortion-reducing designs used in the woofer and tweeter of their 4367 speakers—a design that comes highly recommended by a few people whose opinions I respect. Seeing a graph in their white paper that included second- and third-order harmonic distortion measurements blew my mind.

As we mentioned earlier, if you don’t have something to compare a graph to, it can be difficult to determine if there is a problem. So, I’ve attached included two distortion graphs of the little sound system I use on my desk.

The first graph shows that the 30-yearold subwoofer under my desk causes a lot of rattles and that relates to the midrange driver causing noise at 2200 Hz.

The second image is with the mic in the same position, but with the volume turned up a little bit. This increase in output level (by about 8dB) dramatically emphasized the distortion at 2200 Hz (which I can hear as buzz) and showed an issue around 170Hz.

Listening to Learn and Learning to Listen

I am fortunate to have made a career in the mobile electronics industry, and to have auditioned products from so many companies. What I hope you take away from this article is that it’s crucial to compare products under controlled conditions to quantify their performance.

If you are interested in a great audio-related book, check out Floyd Toole’s Sound Reproduction – Loudspeakers and Rooms. There’s a lot of great information in there and some interesting words of wisdom. I’ll leave you with a specific quote from the book: “Only when you compare one thing with another do you start to realize what differences there may be. And if you do a simple A vs. B comparison, and both A and B both share the same problem, you won’t hear that problem.”