DAC Listening Challenge Results

A BIG THANK YOU! I appreciate all the responses to both listening tests. I know listening for subtle differences requires some effort and I want to thank everyone for taking the time and sharing their results.

NOTE TO HEAD-FI MEMBERS (revised 3/28):  Unfortunately, before I posted this article, the administrators at Head-Fi.org started censoring links to this blog. So if you had some trouble finding the results here, I’m sorry. The links were apparently not a problem when I was posting at Head-Fi on several other products and topics. The censorship only started after a paying Head-Fi sponsors (NuForce) publically complained about my review of their product. Head-Fi has gone so far as even deleting posts from other Head-Fi members that reference this blog.

BACKGROUND: NuForce responded to their uDAC-2 measuring poorly by saying it was designed to sound good even if several measurements are notably bad. So I came up with the best way I knew how to judge the NuForce purely on sound quality. I also thought it would be interesting to compare the $29 Behringer UCA202 with a high-end product like the Benchmark DAC1 Pre. The DACs were recorded playing real music under as realistic and similar conditions as possible. Anyone could download the recordings and compare them without knowing which was which—sort of like a “brown bag” wine tasting. Would people like the $3 wine better than the $30 wine? I thought it would be fun to find out!

TEST METHODS: Not having conducted a public web-based listening test before, this was something of a learning experience for me. For the first round, to make the comparison as fair as possible, I used the line outputs of all three DACs . The second round of tests used the headphone outputs driving real headphones and included a modified version of the Behringer UCA202. The modifications were made using a few dollars worth of parts and improve the headphone output of the Behringer. The files were given names of US Presidents in the first test and common trees in the second. The original listening tests, and all the details, can be found here:

• NuForce uDAC-2 Listening Test
• DAC Listening Sequel

A NOTE ABOUT “MASKING” (revised 3/17): Some have questioned the validity of the tests because, as NuForce put it, the output of the DACs have been “re-digitized”.  And some also argue the equipment used to play back the files may not be high enough quality. The concern is these could mask the differences between the DACs. In reality this should not be much of an issue using playback hardware with reasonable fidelity. Here’s why:

• These trials are only about the differences between the files, not the absolute accuracy. It’s like shopping online for a shirt. You might look at 3 different blue shirts at the same online retailer. Even if your computer display can’t convey every color and detail perfectly, you can still easily tell most of the differences between the shirts if they’re good pictures, all taken the same way, and your have at least a reasonably decent display. For example if your computer distorts the exact shade of blue, you should still be able to tell which of the 3 shirts is the darkest blue. The same is true in comparing the sound files. Even if your headphones might exaggerate the bass, you can still tell differences between the bass in the various files. This “difference effect” has been well documented in research.
  
• As to “re-digitized”, the Benchmark ADC1 used to convert the output of the DACs to a CD quality file is already better than most studio gear used to record the music we all listen to. Studio A/D converters typically cost about $100 – $400 per channel of conversion. The ADC1 is a reference-grade A/D with a $900 per channel price tag. It generally has better specifications than nearly all studio gear. If the less expensive studio gear is good to enough to capture the subtle difference between say a high-end Steinway and a high-end Yamaha grand piano, the ADC1 should also capture audible differences between DACs.
  
• Most recordings have already been through many more steps and kinds of digital processing yet are still very revealing of subtle differences. So adding one more relatively “pure” step isn’t going to make much difference. The recordings used in these tests are unusually pure and transparent.

PARTICIPANTS: This was a very informal survey. A total of 20 listeners picked their favorite (and sometimes least favorite) tracks. Many ranked them top to bottom. Some only participated in one of the tests and/or one of the songs. Some used ABX and some listened conventionally. More participated in the first (line out) trial than the second (headphone out) trial.

SCORING: Not everyone participated in every trial, some devices were offered in more trials than others, and different listeners provided different sorts of “votes”. Someone who’s better with statistical analysis than I am is reviewing the raw data. But, for now, to summarize the results here’s my attempt at a rough analysis:

• Top choices scored 2 points
  
• Second (“Runner Up”) choices scored 1 point
  
• Least favorite (worst) scored –1 point (several indicated only their least favorite and not a favorite)
  
• The results are summarized by category (line output vs headphone output)

RESULTS (corrected 2 minor errors 3/16): Here are the total points scored for each downloaded file ranked from most favored (highest score) to least favored using the above scoring method:

Taft - Benchmark Line Out Brick House	9
Harrison - NuForce Line Out Brick House	8
Juniper - Benchmark CX300 Brick House	7
Wilson - Behringer Line Out Brick House	6
Jefferson - Reference CD Just Dance	4
Lincoln - Benchmark Line Out Just Dance	3
Oak - Behringer CX300 Brick House	2
Spruce - Mod UCA202 CX300 Brick House	2
Acorn - Benchmark CX300 Tis of Thee	2
Maple - Mod UCA202 CX300 Tis of Thee	3
Monroe - Reference CD Brick House (!)	1
Hawthorn - NuForce CX300 Tis of Thee	1
Fig - Behringer CX300 Tis of Thee	1
Jackson - NuForce Line Out Just Dance	0
Cypress - NuForce CX300 Brick House	0
Adams - Behringer Line Out Just Dance	-2
Pine - NuForce UE SF5 Tis of Thee	-4
Total Benchmark Line Out	12
Total Benchmark Headphone Out	9
Total NuForce Line Out	8
Total Behringer Line Out	4
Total Modified Behringer Headphone Out	5
Total Behringer Headphone Out	3
Total NuForce Headphone Out CX300	1
Total NuForce Headphone Out Ultimate Ears SF5	-4

COMMENTS ON RESULTS (edited 3/16): Even with the small sample size a few things were fairly clear:

• The NuForce headphone output with the Ultimate Ears SuperFi 5 Pro headphones was an obvious “fail”. Several expressed a clear dislike of “Pine” and nobody favored it. I think this is due to the large frequency response variations caused by the relatively high output impedance of the NuForce uDAC-2 with these headphones. Several commented the high frequencies were rolled off or “dull”. This consistent with what the measurements would predict.
• The reference CD tracks didn’t score as well as expected. For “Monroe” it’s likely because the reference track stood out as being different. Early in the forums it was labeled “by far the worst”. And that public comment likely tainted others into also hearing it as the “worst”. Six people rated it as the worst of that group of tracks. I personally think this is a good example of subjective bias--the original piece of music was deemed the “worst”! The Just Dance reference faired much better with a positive score of 4 likely because it wasn’t labeled publically as being bad. And much fewer even bothered to vote on Just Dance so the score of 4 was close to “perfect”.
• In terms of total votes, the Benchmark stood out as a clear favorite. It was included mainly as a “reference” and not necessarily as fair competition to the other two, much cheaper, DACs. But, some might have expected all the DACs to sound roughly similar via the line outputs based on the measurements. The could be many reasons why this didn’t happen. The most likely is the same “peer bias” that caused a reference track to be rated poorly (see above). But it’s also possible the cheaper DACs do have audible problems.
• The NuForce did much better via the line outs than the headphone outputs. Given the output impedance problem and higher distortion of the headphone output, this is what the measurements would predict.
• Any channel balance error was corrected in all tests. So this removed the audible imbalance of the NuForce helping it score better.
• NuForce’s claim of “better sound” doesn’t seem to be true when using the headphones. It faired poorly even with the more common and “impedance friendly” Sennheiser CX300’s.

Trying to draw other conclusions is a bit more difficult, but some things worth noting:

• Nearly half the people couldn’t hear any differences at all.
• The Just Dance track had a lot of inherent distortion that made hearing differences difficult for most. So those results are likely less valid (and there are fewer of them).
• A number of votes were somewhat randomly distributed between all the middle scoring test files. This may suggest more guessing rather than clear preferences.
• The line outputs of  the $29 Behringer didn’t do as well as its measurements would suggest. The NuForce and Benchmark were preferred. This could be “peer bias”.
• The differences among the individual files were more obvious via the headphone outputs rather than the line outputs. This is to be expected due to the impedance interaction.
• For those who argue tests like this mask differences, it’s interesting to note the test may not have masked any subtle advantages of the high-end Benchmark. This is despite the fact that few listened to the files on similarly high-end gear.

BOTTOM LINE: I think the most obvious thing is roughly half the participants couldn’t tell any difference and another group of the “middle scoring” results is almost random with no clear preferences. Comparing the headphone outputs, however, the differences seemed more obvious. And they were very obvious when using balanced armature headphones (the SuperFi’s) on the NuForce. It’s clear the frequency response variations created by the NuForce’s relatively high output impedance creates audible problems.

It’s apparent subjective opinions of differences are easily swayed by public comments. The reference track from the CD of Brick House was strongly disliked by six listeners. They heard something different and were easily swayed into thinking different was worse based on previous public comments. It just took one person to say something negative and several others followed. This is exactly the sort of subjective bias that affects the majority of online subjective reviews in forums. As further proof, others using ABX (which is blind) voted the same track their favorite.

There’s also significant evidence the NuForce uDAC-2 does not sound better—at least using 2 different types of headphones—as NuForce claims it does. But it seemed to do a respectable job via the line outputs—at least with the channel balance error removed.

Overall I think this has been an interesting experiment. I learned a lot about how to run (and not run!) a listening test. And there were some fairly clear results—some expected and some not. If there’s sufficient interest, I may try to build on what I’ve learned here and conduct future listening tests?

COMMENTS WELCOME: Please feel free to add comments to the end of this article on the results—especially if you participated. I’d also like to know how many are interested in future listening tests? They’re a fair amount of work to put together and only really valid if you get a reasonable number of votes. So it’s something I only want to do if there’s enough interest. Please feel free to make suggestions, etc?

---

TECH SECTION:

IMPROVING THE RESULTS: I realize this wasn’t the best run study. It was more an informal experiment than anything. I think a larger scale listening test, that had more uniformity, would be needed to verify some of the closer results. If I do this again, I’ll research better methods, and I also welcome input from others with experience in this area?

DIFFERENT LOADS: It’s been suggested it would be also useful to include high impedance headphones and, except for the complexity that adds to the mix, I agree that would be interesting. I chose low impedance headphones as they’re, by far, the most popular—especially for use with a portable entry level DAC.

PASSWORD: An encrypted 7-Zip file was included with the file descriptions to prevent me from cheating or changing anything after people’s votes. The password is:

CY&YUMN5cZ9x2X8BhNj2t

PINE EXPLAINED: The graph below, in blue, shows the frequency response of the NuForce uDAC-2 using the Ultimate Ears headphones used in the trial:

The 4+ dB of response variation seen above is caused by the relatively high output impedance of the uDAC-2 interacting with the SuperFi headphones (typical of balanced armature designs from many manufactures). For more on this see Headphone & Amp Impedance.

OTHER DETAILS: The other details of how the test was run can be found in the original articles:

NuForce uDAC-2 Listening Test

DAC Listening Sequel

Behringer DAC Modifications

INTRO: The little Behringer UCA202 DAC, for under $29, did surprisingly well in my review.  In all areas but one, the UCA202 is a respectable performer. The photo shows the much larger output capacitors (big black objects above the volume control) laying on their sides that are part of the modification.

PRO-SOUND DESIGN: Behringer likely designed the UCA202’s headphone jack for use with higher impedance studio headphones. While it can drive such phones fairly well, it’s not very compatible with typical consumer 16 – 32 ohm headphones.

WEAK HEADPHONE OUTPUT: The 50 ohm output impedance of the UCA202 limits the volume you can get with low impedance headphones and also causes substantial frequency response variations with certain phones due to how the impedance interaction (see the article on headphone/amp impedance).

The graph (click for larger) shows the frequency response of the original UCA202 with no load in yellow and, without changing the levels or volume, what happens when you plug in a pair of Ultimate Ears SuperFi 5 Pro headphones. The balanced armature design of the SuperFi’s create wide impedance swings that cause a total of 14 dB of frequency response deviation (+/- 7dB). The broad midrange boost and high frequency cut is very audible and undesirable.

THE CURE: I wondered if it’s possible to lower the output impedance and raise the output level? If so, the Behringer would be a bargain USB Headphone DAC.

SOME RESEARCH: Unlike similar “disposable” consumer electronics that are often glued or snapped together in a way you have to destroy them to open them up, the UCA202 comes right apart after removing two easily accessible screws. My first observation was the general purpose op amp used to drive the headphones can’t drive lower impedances very well. And there are 47 ohm resistors in series with its outputs (required by the op amp). These further raise the output impedance and limit the output power. I came up with a modification that requires replacing the op amp and four to six other parts.

THE EXPERIMENT: I warmed up the soldering iron and went after the UCA202’s PC board. I replaced the headphone op amp with a much higher quality pin compatible IC. The new amp can much better drive lower impedances at low distortion and generate much more output power from the same power supply voltage. A few other component changes were also required to optimize the new design. The photo shows the old IC removed and the other component locations circled.

THE RESULTS: The table below summarizes the measurements between the original UCA202 and the modified version. The main goals were met with more than 12 times more output power into 16 ohms and the output impedance dropped from 47 ohms to a much more reasonable 2.5 ohms. Using Ultimate Ear’s headphones, the frequency response improved from +/- 7 dB to a much more neutral +/- 1 dB. The other measurements were similar enough to be the same (better results are in bold):

 

 

 

 

 

 

 

LISTENING TEST: In the Listening Test the modified version scored a slightly better than the unmodified version’s headphone outputs. But I used headphones in the test with a relatively constant impedance so the frequency response changes were minimal. The listening level was also (barely) within the unmodified versions power capability. Changing the headphones, or raising the volume, would have yielded a much more obvious advantage for the modified version.

NOTE TO BEHRINGER: Behringer could, if they wanted, mass produce a similarly improved version of the UCA202 and the retail price would only need to be increased by less than $10. The changes I made would raise their wholesale component costs by less than $2. Unlike the current model, the new $35 – $39 USB DAC would meet the headphone needs of both professionals and consumers. It would suit a much wider range of headphone types and play significantly louder with any headphones with more accurate frequency response.

NOTE TO OTHER USB DAC MANUFACTURES: This experiment, along with the original UCA202 Review, show what can be done with an inexpensive USB headphone DAC. It’s my opinion if any company is going to charge several times more, and can’t at least equal this level of performance, they’re probably not offering a good value for their customers.

BOTTOM LINE: I was pleased with the above results. The main purpose of this rather academic experiment was to prove: It’s entirely possibly to design, manufacture, and sell through retailers, a high quality headphone DAC for $39 or less. The bad news is, as explained in the Tech Section below, it’s not practical for most to perform the modifications. And I’m not holding my breath Behringer will offer an improved version given their target market. And if you paid someone else to do it, the total cost (including the UCA202, labor, and parts) would probably spend around $100. And, at that price, there are likely some better options that don’t require modifications (i.e. the $99 FiiO E7, $99 Firestone Fireye2, etc.). As always, I welcome comments and feedback!

---

TECHNICAL SECTION (all the details):

WHAT DIDN’T CHANGE: Because only the headphone amp was changed, the performance of the DAC chip itself, and the line outputs should remain the same. I did some quick checks to verify the line output performance, DAC linearity, jitter, etc. were similar to my earlier review. They were.

DIFFERENT TEST LOAD USED: The original UCA202 did so poorly with my usual 15 ohm load, I used a 150 ohm load for most of the headphone output tests. But this makes comparing the UCA202’s headphone performance to other devices I’ve tested more difficult. So, for these tests (unless otherwise noted) I used my standard 15 ohm test. Note this is a much more challenging load, so in some ways, one can expect degraded performance from the previous 150 ohm measurements.

MAXIMUM OUTPUT: The original UCA202 had a hard time with my 15 ohm test load. It only managed 179 mV RMS at 1.3% THD+N which is 2 mW into 16 ohms. Here’s the plot:

And here’s the modified version doing more than twelve times better better at 635 mV which is over 25 mW into 16 ohms:

And because the new amp IC can swing closer to the supply rails, even with 150 ohms, it has an advantage. Here’s the original max output at 150 ohms of 668 mV or about 3 mW with the volume set as high as possible for < 1% THD+N:

Here’s the modified version into the same 150 ohm load with nearly 4 times as much power (11.3 mW) this is with volume control all the way up at 0 dBFS:

THD+N AT REFERENCE LEVEL: At the reference level of 400 mV RMS here’s the original driving the relatively easy 150 ohm load:

And here’s the modified version, same level, but into a much more challenging 15 ohm load. The distortion is virtually identical to the above measurement and likely determined more by the DAC chip and implementation than the new headphone amp (anything below 0.01% is widely considered inaudible anyway):

OUTPUT IMPEDANCE: Besides output level, the other big problem with the original UCA202 is the approximately 50 ohm output impedance. Here’s the modified version with no load referenced to the same 400 mV level used above. Knowing the no load (source) voltage is 467mV and the voltage with the 15 ohm load is 400 mV, the output impedance calculates out to 2.5 ohms which is a huge improvement and very close to my ideal of 2 ohms or less. I’m not sure yet why it’s not closer to 1 ohm but I suspect it’s at least partly the output capacitors I used (more on those later):

NOISE:  The A Weighted output noise previously measured –92.0 dBA while the modified version measures a nearly identical –91.4 dBA. A difference of 0.6 dB is not generally considered audible. The 47 ohm series resistor in the old design was attenuating the output noise with a 150 ohm load by about 2 dB. Considering the new amp doesn’t have such an attenuator, it’s noise performance is actually better. The new amp IC has better noise specs than the original so this is what you would expect. And the remaining noise appears to be from the DAC itself. So, for all practical purposes, the noise performance is the same:

SWEPT THD+N: The original UCA202 did well on this test and the modified version does even better despite driving a much more challenging load (15 ohms vs 150 ohms). The original is shown in red, the modified version in blue, and the NuForce uDAC-2 in green (note the vertical scale is expanded to a max of 0.1% compared to the NuForce uDAC-2 Review):

CHANNEL BALANCE: Channel balance is very load dependent. There are multiple reasons for this. More challenging loads cause the power supply (which is shared between both channels) to have more “load ripple” which is essentially the audio signal being superimposed on the power supply. Another reason is higher loads create higher currents and higher currents create more electromagnetic radiation which “cross couples” between the channels. This happens both inside IC’s shared between both channels (like the headphone amp IC is in the UCA202) and even between tracks and passive components on the circuit board. So I wasn’t surprised the UCA202 did much worse at 15 ohms compared to 150 ohms. If you compare the 150 ohm performance with the modified amp, however, you’ll see the result is unchanged. So the red line below isn’t a result of the modification, but mainly the much more challenging load. It’s not a great measurement at about –38 dB. But, especially for headphone use where a lot of people add far more “crossfeed” on purpose, it’s likely not a significant problem:

FREQUENCY RESPONSE: Because the UCA202 lacks a split power supply, or bridged amplifier design, it uses AC coupled amplifiers and outputs. So in both versions, the headphone amp requires an output capacitor in each channel to block DC from reaching the headphones. Into 50+ ohm loads a relatively small capacitor works fine. The original UCA202 uses 100 uF output caps. With a 15 ohm load, however, the –3 dB low frequency roll off changes dramatically. So I increased the size of the output capacitors to help compensate for this. The values I used create a low frequency roll of only about –0.2 dB at 20 hz even with this worst case load. There’s also a high frequency rise that can likely be easily fixed. The new amp IC is “faster” than the old one, and hence requires different feedback compensation. I don’t have the right small value surface mount capacitors on hand to tweak the feedback loop. So I left the original Behringer compensation and it’s causing about a 0.4 dB rise at 10 Khz as shown (the original UCA202 is shown in yellow):

And here’s a composite plot showing the frequency response of both versions with a variety of loads taken at the same relative level. The most interesting thing to compare is the blue plot of the original UCA202 driving Ultimate Ears SuperFi 5 Pro headphones versus the red plot showing the new version driving the same headphones. You can also see the massive drop in output and low frequency roll off with a 15 ohm resistive load in orange versus the modified version in green:

OTHER TWEAKS: As mentioned above, the feedback loop could use some attention. Optimizing the feedback should remove the slight 0.4 dB rise at the highest frequencies and might improve the sound. And I suspect using higher quality output capacitors might lower the output impedance still further. The problem here is finding better parts that will fit inside the original case. All my modifications fit completely inside (although I did have to cut a plastic post off the inside of the case). Better audio-grade output capacitors (and possibly adding quality bypass caps) may also improve the sound quality and further lower the output impedance. It might also be possible to improve the channel separation with some tweaks to the power supply lines feeding the new amp IC.

THE BAD NEWS: This is mostly an “academic experiment” as few have the tools or skills for the surface mount re-work these modifications require. As can be seen in the photo, surface mount components are incredibly small and not made for human soldering (machines do it all when the boards are made). The photo shows three surface mount resistors in their original paper “tape” next to a regular pencil. The entire parts are smaller than the tips on most soldering irons.

THE DETAILS (added 3/25): I’ve received several messages and emails wanting to know the details of the modification. My plan was to research the issues mentioned above and then publish the final component values. However, for those of you who are just curious, here are the details. I want to stress that anyone who doesn’t have enough surface mount soldering/re-work experience could easily damage their UCA202 beyond repair attempting these modifications. And I also have not explored the high frequency “peak”, tried to improve the channel separation, or explored the somewhat higher than expected output impedance. So for anyone attempting this modification, please be aware it is presented “as is”, has not been fully developed, and you may damage your UCA202!

I chose an Analog Devices AD8656 op amp. It’s designed for high quality audio use from a 5 volt power supply with very low distortion and noise. And, equally important, it has +/- 75 mA of peak current capability to better drive low impedance loads directly along with a “rail-to-rail” output increasing the power available for higher impedance headphones. For a high-end op amp it’s also relatively inexpensive in single piece quantities at $2.75 (I wanted to keep the total parts cost < $5). If cost is not an issue, I would consider the $5.22 AD8397 as it has +/- 310 mA of peak current capability and is used in many respected commercial products as the headphone amp although it’s not optimized for a single 5 volt power supply.

I replaced the series 47 ohm Behringer output resistors (circled in the photo earlier) with 1 ohm 0603 SMT resistors shown next to the pencil in the photo. And I replaced the 100 uF Behringer output capacitors (pads circled) with 2200 uF 6.3v capacitors you can see at the start of the article (laying on their sides). I had to cut one of the internal plastic posts from the case (also visible in the phone) to make room for the larger caps.

FINAL WORDS: All things considered, for a fairly simple “hack”, I was rather pleased with how this turned out. The modified Behringer UCA202 has enough power to satisfy most anyone (significantly more power than a Sansa Clip+ for example). And the output impedance drops from a very poor 50 ohms to only 2.5 ohms. It retains its previously low distortion, similar noise levels, and most other characteristics. The very slight ( +/- 0.3 dB total) frequency response variations are inaudible but I’d feel better about this modification with the feedback compensation adjusted for the new amp IC. But I have to order some parts to do that. If enough people are interested, I can take this project further?

FiiO E5 Headphone Amp

INTRO (updated 4/7): The little FiiO E5 typically sells for under $20 and is so small you barely notice it. It’s “Apple-Shuffle-size” small. And, despite the ultra slim metal package, it has a Li-Ion rechargeable battery tucked inside, volume control, and selectable bass boost EQ. They even wisely made it rechargeable from a standard mini-USB cable so you probably have a cable laying around already if you lose the one that comes with it.

FOOD INGREDIENTS: Ever watched any of those “chef shows” on TV? They give a team of chefs a simple assortment of ingredients and challenge them to make something amazing. The best chefs take common everyday items and create really tasty mouth watering entrees. That’s what FiiO has done here. This is in sharp contrast to certain “boutique” audiophile companies that take high-end exotic ingredients and combine them in ways that don’t taste very good. The difference, in my opinion, is engineering talent and resources.

To put this another way, a good French chef can do more with a few eggs, cheese, butter and a piece of bread than a bad chef can do with $500 worth of exotic ingredients. What good is including exotic ingredients if the entrée tastes awful? FiiO seems like they have a French chef on staff able to transform inexpensive parts into something that’s relatively impressive.

CLIP IT ON: Like the Sansa Clip+ (see my review) it has a built in clip (sexy metal even!) to help with cable management. So you can have your player in your pocket, and the E5 clipped to your shirt, or whatever might work.

OTHER FEATURES: For the price, the E5 impressed me as being relatively well designed. Here’s a link to the features & specs.

OPERATION: It has a small power button, EQ switch, and a volume up/down “rocker” switch. The volume steps are impressively fine with no big jumps and you can hold down the rocker to steadily increase or decrease the volume easily. You can also use it while it’s charging or just leave it plugged into USB power. On my PC I didn’t hear any extra noise using it on the wimpy headphone output while charging it from a USB port on the same PC. There’s a 3.5mm input and a 3.5mm output jack, the mini-USB charging port, and that’s about it. It comes with all the necessary cables. The LED indicates charge status when it’s re-charging. There are plenty of volume steps so you don’t find yourself wanting for finer resolution.

BASS EQ: The bass EQ subjectively is fairly pleasant, relatively subtle, and particularly welcome with headphones that have flat bass response (i.e. those from Etymotic, RE-1’s, etc.). With bass shy headphones it adds a bit of of “heft” without creating any boom. The curve is shown in the Tech Section but it’s about a 3 dB boost centered at 80 hz.

HISS & NOISE: For such an inexpensive amp, it’s impressively quiet. It doesn’t have much noise of it’s own. Even with my ultra sensitive Ultimate Ears SuperFi 5 Pro’s, the hiss at full volume with no input is barely noticeable. It will, like any amp, boost whatever noise your source device has. See my article on headphone amps.

SUBJECTIVE SOUND (updated 3/21): The E5 sounded impressively good for its size and price. It’s clean, clear, plays plenty loud, and has no issue with low impedance, high impedance or balanced armature headphones. I had no complaints with the sound.

QUALITY CONTROL? My E5 sample has a slightly audible 3.8 dB channel imbalance but only with the bass EQ off. And it’s relatively constant regardless of the E5’s volume setting. Many other E5 owners have confirmed this is not normal. So apparently I got a slightly defective one.

MEASUREMENT SUMMARY: The good news is the maximum output level, output impedance, distortion, and most everything else was amazing for a such a small inexpensive amplifier. The frequency response was –3 dB at 20 hz which isn’t likely to be a problem but isn’t as flat as say the Sansa Clip+ either. And the noise was a few dB higher than I would have liked to see but still respectably low at around –88 dB. The distortion also climbs at very low frequencies but this isn’t likely to be audible as human hearing is not very sensitive to deep bass distortion.

BOTTOM LINE: This a respectable little amp for the size and price. It’s surprisingly well designed, well made and it performs much better than I expected. To put it in perspective, this little $20 amp easily outperforms the amp in the $130 NuForce uDAC-2 in several respects. It has a much lower output impedance, lower overall distortion across most of the audio spectrum and higher output into low impedances. The lower output impedance, in particular, allows it to deliver much better performance with IEMs that use a balanced armature, multi-driver, or similar technology (i.e. Shure, Etymotic, Ultimate Ears, Westone, etc.).

---

TECH SECTION (optional reading for more details):

THD & REFERENCE LEVEL: With approximately a 200 mV RMS input (which just about any source should be able to provide), I adjusted the E5’s volume control for 400 mV of output (a gain of “2X”) as this should approximate real world use. With a 15 ohm resistive load, here’s the result:

The distortion of 0.01% is quite good. By comparison, for example, the NuForce uDAC-2 measures about 0.05% on this same test with the same load. What’s really not so good is the 3.9 dB of channel imbalance. This isn’t a “tracking” problem as it’s essentially the same at any volume setting. It’s just one channel has nearly 4 dB more gain than the other but, interestingly, if you turn the bass EQ switch on, it goes away (the channels are balanced). To measure the output impedance, here’s the same set up with no load. The voltage in Channel A went from 401 mV to 420 mV and doing the math, that’s a very impressive 0.7 ohms—that’s even better than I measured with the Sansa Clip+ and I thought it was impressive:

MAXIMUM GAIN & OUTPUT: Here’s the E5 with its volume at maximum. I raised the input voltage until it was close, but not yet clipping. Here, even with 1.27 volts of output into 15 ohms—over 100 mW of output and the THD is still under 0.02%! This is very impressive. And 100 mW is a very impressive output level. This is about twice what the NuForce uDAC-2 could manage into the same load. Note also, this is referenced to the input voltage. So the maximum gain of the E5 is 8.5 dB in one channel and 4.6 dB in the other. I didn’t bother testing into a higher impedance load, because with the 0.7 output impedance it makes little difference. It always starts clipping around 1.3 volts RMS.

INPUT CLIPPING THRESHOLD: To measure how much signal you can drive the E5 with, I gradually raised the input voltage until clipping was detected with the E5’s volume turned down. The answer is a lot! it didn’t start to have a problem until around 760 mV RMS:

THD SWEEP: Here’s the THD swept versus frequency for both channels. The lower traces (red and yellow) are with no load. The upper traces (blue and gray) are with a 15 ohm load. The distortion rises dramatically at lower frequencies and barely at high frequencies. It hits around 0.5% at around 25 hz. Fortunately, humans are rather insensitive to low frequency distortion. Some studies have found even 5% THD below 100 hz is hard to detect. That’s a good thing because most transducers (speaker or headphones) tend to have rather large amounts of low frequency THD at higher volumes. So the midrange and high frequency performance here is excellent, the deepest bass is a bit disappointing but likely not an audible problem:

NOISE: Here’s the “worst case” noise with the E5’s volume at maximum and no input. It’s 1 – 3 dB better and lower volume settings. This is acceptable noise and similar to many portable players and the Behringer UCA202. The NuForce uDAC-2 is significantly quieter.

IMD: Here’s the SMPTE IMD at 400 mV out into 15 ohms. It’s very clean:

FREQUENCY RESPONSE: Here’s the frequency response with various loads. Note the divisions are only 0.5 dB. The –3 dB point is 20 hz. Or, put another way, the E5 is +/- 1.5 dB from 20 hz to 20 Khz which many would argue is “flat enough”. I would have liked to see a bit less low frequency roll off, but this isn’t bad at all as it’s only down 1.5 dB at 30 hz. Interestingly it doesn’t seem to be caused by an output capacitor because the roll off doesn’t change with loading:

 

LOW FREQUENCY EQ OPTION (added 4/7): Below in red is the response (starting at 20 hz instead of 10 hz as above) into 150 ohms. In blue is what happens with the Bass EQ switch turned on. it’s a fairly gentle broad rise that, at its peak, is only 3 dB higher than the reference level. The peak is broadly centered around about 80 hz. Note this nicely compensates for the low frequency roll off as the response is still a bit above 0 dB even at 20 hz. As far as bass EQ goes, this is much more subtle than most. And the channel balance problem goes away with the EQ turned on.

CHANNEL SEPARATION: Here’s the channel separation and it does very well for something so tiny with the two channel jammed close together—a mostly constant 46 dB. Note this is driving a difficult 15 ohm load. The separation improves into higher impedance loads:

1 KHZ SQUARE WAVE PERFORMANCE: The E5 also does really well here. Unlike when testing DACs with square waves, there’s no ringing from the low pass filters all DACs have. So here it shows the flat frequency response (flat top of the waveform) and stability of the amplifier (lack of much ringing):

10 KHZ SQUARE WAVE PERFORMANCE (added 4/7): The dScope above makes a great 24 bit high resolution oscilloscope but only up to 96 Khz. This is fine for testing digital sources (players, DACs, etc.) but to really “stress” an analog amplifier you can use a 10 KHZ signal with a very fast rise time (from a 25 Mhz Tektronix AFG3022 Waveform Generator). The signal is measured with a 100 Mhz digital oscilloscope to evaluate the “speed” and other behavior of the amplifier. The red waveform on the bottom is the input to the amplifier and the blue trace on top is the output. The rounded corners represent the high frequency roll off and the slope of the rising edge represents the rise time and slew rate. This is decent, but not high-end, performance:

Here’s a ‘zoomed’ view of the rise time. Rise time is usually specified from the 10% to the 90% point. I didn’t use the on-screen cursors here but the time from –0.4 volts to + 0.4 volts is about 4.2 uS. The slew rate is the “flat” sloped part of the line and is about 0.4 v/uS. That sounds like a really bad number but, in practice, an amplifier only needs about 0.2 v/uS per volt of RMS output to reproduce a 20 Khz waveform perfectly. From a 44 Khz sampled digital source (i.e. CD quality audio) that’s the fastest slew rate you need. Even a 20 Khz perfect digital square wave comes out of a 44 Khz DAC as a 20 khz sine wave. So the 0.4 v/uS of the FiiO E5 means it can reproduce 20 Khz up to about 2 V RMS without slew limiting. And, because it clips around 1.4 V RMS, there’s a decent safety margin here.

TECH SUMMARY: The performance is impressive in most areas given the size and price. The extremely low 0.7 ohm output impedance and 100 mW into 16 ohm power level at low distortion are the stand outs on the good side. The bass EQ is subtle and overcomes the slight LF roll off nicely.

Benchmark DAC1 Pre

WHY THE DAC1? Some here have asked why I would own a product like the Benchmark DAC1 Pre when it seems I’m relatively pleased with even a $29 Behringer DAC? That’s a fair question and here are some of the reasons:

• For professional reasons, I needed a DAC that’s more accurate than other equipment I work with. It needed to have as complete of performance specifications as possible. I also needed balanced outputs, multiple digital inputs, and USB connectivity that did not require proprietary drivers. I needed a reference-grade DAC. That list of requirements narrows the choices considerably.
• I partly chose the Benchmark because it’s been very well reviewed subjectively by the audiophile media for its “soundstage”, “micro-dynamics”, “lack of artifacts”, “pace”, “black background” and general “musicality”. Here’s an Absolute Sound Review by Robert Greene. Greene said “It is as if the electronics had simply vanished”.  It has won product of the year awards and numerous other accolades.
• The DAC1 has also been praised for its excellent objective measurements. Several have hooked it up to an Audio Precision analyzer and been rather stunned by the results. Various versions of the DAC1 have won the title of “best DAC ever measured” by some reviewers. It’s relatively rare to find one piece of gear that pleases both the highly subjective esoteric audiophile reviewers, and also the hardcore objective engineers of the world. Often gear that measures really well gets branded with adjectives like “sterile” or “clinical”. But the Benchmark is rare in that it largely pleases both groups.
• Benchmark has some very rational views on jitter (they’ve written an interesting paper on it). They’ve come up with a scheme that works very well in the real world (ASRC). You can argue about how much jitter is audible. But it’s hard to argue with how little visible jitter artifacts there are on the DAC1’s outputs no matter what you feed it—my tests and those of others back this up.
• It’s a 24/192 DAC that properly supports 24/96 via USB with no special drivers. A lot of products don’t properly handle 24/96 on a Windows PC without proprietary drivers. It’s becoming more common, but when the DAC1 was introduced, it was hard to find. Many USB DACs use the rather jitter-prone, old and low-end TI PCM29xx or PCM27xx chips for their 16/48 USB interface. I needed something better.
• I respect Benchmark Media as a company. They have a very solid design philosophy and they take their time developing and refining a small number of products. This is in sharp contrast to some similarly small high-end companies that churn out new models every few months trying to follow the latest fad or trend. Benchmark’s roots are in professional gear and it shows in their quality, detailed specs and lack of gimmicks. They’ve done an artful job of bridging professional requirements and performance with the often irrational desires of high-end home audiophiles.
• Finally, it’s not a huge deal, but I like that Benchmark products are made in the USA. They have total control over the products from the original design to shipping them out the door. You can call Benchmark on the phone and the person who answers can probably can tell you what size screws hold the case together (I’m only exaggerating a little). They also have a 5 year no questions asked warranty. They’re a small company and very focused on  doing only a few things really well. They haven’t sold out to some heartless share-price-driven public corporation like Harman International or some Chinese company eager to leverage an established brand name for maximum profit. I hope they never do.

MEASUREMENTS:

The DAC1, in some ways, exceeds even the high-end abilities of my Prism Sound dScope Series III measurement system. I have published a few DAC1 measurements, where applicable here and there, but a full review would be challenging. Regardless, the DAC1’s performance, in every sense, is well past the point of diminishing returns. It’s one of those rare products you never have to worry about not being up to the task. You can just about arc weld with the high current zero ohm headphone output and the line outputs have vanishingly low distortion of any kind.

SUBJECTIVE SOUND:

Some have asked if the Benchmark sounds better than DAC X, Y or Z. And, in some cases it does. In other cases it’s a much harder call. It matters what the source is, what it’s driving, etc. The whole idea is it’s genuinely a true reference-grade piece of gear. It’s the standard by which other gear can be judged.

I think it would be difficult to find a more accurate sounding DAC at any price. Some DAC’s might have an intentionally soft high end that some prefer. Another might have a tube output stage with “euphonic” (pleasant to some ears) distortion. So there are other DACs that sound different. But I would be surprised if anyone can find a DAC that’s more audibly true to the original recording than the Benchmark. As Robert Greene at Absolute Sound put it, it “vanishes” which is exactly what I want a DAC to do. I don’t want a piece of gear that colors the sound or uses other gimmicks to try and be different.

LISTENING TEST (added 3/16): I conducted a couple of listening tests mainly to evaluate the Behringer UCA202, a modified UCA202, and the Nuforce uDAC-2. But I included the Benchmark DAC1 Pre as a reference. I wasn’t sure what to expect, but the results interestingly favored the Benchmark overall even when the listeners didn’t know which was which. This was hardly a conclusive test, and I plan to conduct better blind tests in the future, but the result did confirm the DAC1 seems to sound better.

COMPETITION (added 3/28): The closest competition to the DAC1 Pre is probably the Grace Designs m903 but it’s even more expensive and not as thoroughly specified or tested. The Lavry DA11 is also well regarded but has a somewhat different feature set and is less popular.

BOTTOM LINE:

It’s partly about needing a “better than the rest” reference product for professional reasons. And it’s partly about wanting a DAC I don’t have to worry about being the “weak link” in any setup I might use it in. The Benchmark fills both rolls nicely.

DAC Listening Sequel

INTRO: Thanks to everyone for their comments on feedback on the first round—both public and private—on the recently published NuForce uDAC-2 Listening Test. That challenge compared the line outputs of three different USB DACs and some have suggested comparing the headphone outputs might be even more interesting. The one thing keeping me from doing so was wanting to modify the Behringer UCA202 to have a better headphone output first. I’ve finally got that done, so here’s round two!

THE SEQUEL: This time around we have real DACs, playing real music, through real headphones—all captured as accurately as possible by a Benchmark ADC1. While sequels are often not as good as the original, hopefully it’s the opposite here. Just like before, I’ve done all I can to make sure this is as fair (and verifiable) of a test as possible.

YOU DON’T NEED HIGH-END GEAR: You can listen to the test files below on anything you like. Because the listening test is mainly about the differences between the recordings, not the absolute sound of any one recording, you don't need near-perfect gear to pick a favorite.

It's somewhat like looking at 3 different shirts at an online retailer. You can tell easily how they're different from each other by looking at the pictures of each shirt using your PC. You don't need a perfect computer display to tell that one might be a slightly darker shade of blue than another one. Most any computer display is good enough to show the differences. But you can't be sure of the exact shade of blue until you have the real shirt in front of you.

It's the same idea here. Your headphones and playback hardware don't have to be perfect to reveal the differences between the test files. But the more revealing your gear is, the more subtle differences might become more obvious.

OUT WITH JUST DANCE: Several mentioned they had a hard time hearing any differences with the previous Lady Gaga track. It was chosen because the high average level would hopefully reveal the clipping flaw I found in my review of the NuForce uDAC-2. But, in hindsight, it likely has too much distortion of its own.

ENTER ANNIE: I’ve replaced Ms. Gaga with a bit more refined (and obscure) female vocal track: Annie de Franco’s “Tis of Thee” from her Up Up Up Up Up Up release. It’s much less compressed but her voice still hits some fairly loud peaks and there’s some sibilance on the loud bits. Sibilance can be a revealing test as it tends to sound significantly more harsh if there’s much added distortion (sibilant tracks are often used to evaluate the upper midrange and high frequency performance of headphones and speakers). Tis of Thee is an otherwise fairly well recorded song with some interesting ambiance and low bass extension. The excerpt is about 19 seconds long (the legal copyright limit is 20 seconds).

BRICK HOUSE RETURNS: Lots of you seemed to like Sara K.’s Brick House so here it is again. This also allows comparing the headphone outputs to the Line Outputs of the previous trial (well, sort of, see the Tech Section).

KEEPING REAL SCORE THIS TIME! With the last round I was just looking for general comments. But it’s tough to summarize them in any objective way (although I’ll still try). So this time, I would like to be a bit more objective with the results. So if you want to comment, please include your favorite and a “runner up” (second most favorite) from one or both of the tracks. I’ll count the top choices as 2 points, and the runner up as 1 point when I add up the scores. So you can have up to four votes total (two for each song).

A “NEW” USB DAC (updated 3/16): This test includes the same three DACs including the $29 Behringer UCA202 from the previous listening test. But it also includes a modified version of the same UCA202 DAC. The modification significantly improves the headphone amplifier section for more output and a lower output impedance using just a few dollars worth of parts.

HEADPHONES USED: I could have picked one of my more “difficult” headphones, but I decided to use the very common Sennheiser CX300 16 ohm headphones for all but one file. They have a 16 ohm impedance and a relatively flat impedance curve (see the Tech Section below). For one of the DACs, I also used Ultimate Ears SuperFi 5 Pro headphones which have a much wider impedance range. I wanted to see if that track stood out in the blind tests.

CHEATING IS MORE DIFFICULT: Some enterprising folks “cheated” with the previous tracks by analyzing them in some clever ways that don’t involve listening. While you can still likely get some hints if you want to cheat, there are likely fewer clues here and some may not lead you to the right answers. I was more careful in how I prepared the files this time. Besides, the real fun is seeing how your ears do, right?

NO ORIGINAL TRACKS (yet): (revised 3/8/11) This test is mainly about how the DACs sound not about how accurate they are. Some manufactures, like NuForce, at least partly "design by ear" rather than by measurement and think their gear sounds better even if it measures worse. If the criteria is “the closest DAC to the reference is the winner”, companies like NuForce are more likely to lose. But if the criteria is “which one sounds the most pleasing” it should be a fair contest.  The Benchmark DAC1 is already included as a reference here. And after enough votes, I’ll release the original (as they came off the CD) reference tracks if anyone wants to go back and compare each DAC recording to the original or try making their own recording.

THE COMPARISON:

• NuForce uDAC-2 with Sennheiser CX300 headphones
• Behringer UCA202 (unmodified) with Sennheiser CX300 headphones
• Behringer UCA202 (modified) with Sennheiser CX300 headphones
• Benchmark DAC1 Pre with Sennheiser CX300 headphones
• “Mystery DAC” with Ultimate Ears SuperFi headphones (only for “Tis of Thee”)

INSTRUCTIONS: See the previous NuForce uDAC-2 Listening Test for more detailed instructions and a link to an ABX video (not by me), but the basics are:

• Download either or both sets of tracks in MP3 (faster & more compatible) or FLAC (lossless) format
• Compare them carefully--ideally with Foobar and ABX to make sure any differences are real and not imagined or based on the comments and votes from others (see the Previous Review for more on blind listening)
• Pick the DAC that has the sound you most would like to own and listen to, then pick your “runner up” (second most favorite) sounding track for each song, and either post a comment here, to an applicable forum thread, or message me privately.
• If you think you have a good idea about which DACs were used to record some of the tracks, please keep it to yourself or message me privately until others have had a chance to do their blind listening and pick the favorites.

THE FILES (updated 3/16): The listening test has been completed, and to save bandwidth, the files have been removed.

RESULTS (updated 3/16): The results have been posted! If there’s enough interest, I may conduct future listening tests as I’ve learned a lot from doing this one.

---

TECH SECTION (optional reading):

NEW AUDIO TRACK: Here’s what Tis of Thee looks like in Audacity:

HEADPHONES USED: The impedance of the Sennheiser CX300 headphones mostly stays between 16 and 17 ohms so they’re a fairly easy load. The trace in gold is the impedance, while white is the phase angle in degrees:

The Ultimate Ears SuperFi 5 Pro’s have a much wider impedance swing from about 8 ohms to over 80 ohms (they’re rated at 21 ohms):

LEVELS: A “comfortable” listening level was chosen for each of the two headphones. This was rounded to 400 mV RMS with the Sennheiser’s, and 200 mV RMS with the SuperFi’s, playing a 0 dBFS 1 Khz tone with the headphones attached as the load. The same level was set for each DAC tested using the volume control on each of the DACs

HEADPHONE ISOLATION: The headphone earpieces were carefully isolated from the outside world and each other, and the room kept as quiet as possible during recording to eliminate stray sounds picked up by the CX300’s (or crosstalk between them).

CABLING: A high quality 3.5mm “Y” cable was used to split the output of each DAC to feed both the headphones and the ADC1. The same cabling was used for all the DACs.

GROUND LOOP NOISE: (updated 3/8/11) A ground loop was formed because there are two USB connected devices, and one of them (the ADC1) is also grounded via its power cord. When the ADC1 is used with balanced connections, this is not an issue, but there’s (almost) no such thing as balanced headphone outputs so I had to use unbalanced connections. The input gain of the ADC1 has to be set 9 – 15 dB higher in this test than it was in the previous line output test because the headphone outputs are much lower than the line outputs. This raises the noise by 9 – 15 dB making previously inaudible ground noise audible—both the NuForce and the Behringer had audible “USB noise”. Ideally you want everything in a system grounded at a single point. Once you have 2 or more grounds, the potential for loops exist. This is why HRT, for example, isolates the ground and USB connection for their Streamer II DAC.

To eliminate the ground loop, and all the associated noise, I used a very high quality Jensen audio isolation transformer designed for the purpose to isolate the DACs from the ADC1. The transformer may have some slight “sound” of its own but it measures and sounds very good by itself and should be relatively transparent. But, technically, the set up is not identical if you’re comparing these headphone output files to the line output files of the previous Listening Test. This would have been true anyway in terms of the input gain on the ADC1 being 9+ dB higher changing the noise floor even without the transformer. But having the transformer in the signal chain may very slightly alter the sound in other ways.

ENCRYPTED FILE: The encrypted was revised to add the new names. It has the same password as the old one which will be revealed later. Those who downloaded the older one can save it if you want to verify my honesty as to what’s what.

EVERYTHING ELSE: The rest should be the same as the last Listening Test.