FiiO D5 DAC

INTRO: This is another article in a series of inexpensive USB DAC reviews including the Turtle Beach Micro II and Syba C-Media CM119. Like both of those DACs, it uses yet a different version of an all-in-one C-Media chip (the CM108). So how does FiiO’s least expensive USB DAC measure up against it’s peers?

FiiO D5 USB DECODER: The D5 is a bit different than the other two C-Media DACs. It doesn’t plug directly into the USP port but instead has a short detachable USB cable using a mini USB plug allowing use of other cables. And, true to their tradition, FiiO manages to include several extra features not found elsewhere at this low price. The build quality is impressive using a full metal enclosure instead of plastic. FiiO also includes volume up/down buttons and mute buttons for the output and microphone. There’s an LED that shows it’s been acknowledged by the operating system and another for the Mic Mute. Plus you get two microphone jacks instead of one, a headphone out, a separate line out, and a coaxial digital output. When you consider the D5 has the same $25 street price as the Turtle Beach Micro II, FiiO gives you a lot more features.

FiiO D5 VOLUME CONTROL: The volume control isn’t a “local” volume control in the analog domain after the DAC (as in say the NuForce uDAC-2). Instead it’s really just a remote control for your PC’s volume control. Pressing the up and down buttons on the D5 changes the volume settings for the D5 in Windows (I didn’t test OS X or Linux). So it does not help preserve full 16 bit resolution at lower volume settings. Using the controls on the D5 is no different than using your PC’s volume control which causes a reduction of bit resolution at anything but full volume. Still the volume buttons could be useful, especially if you use a long USB cable and the D5 is located away from the PC or allowing volume adjustment without turning the screen on or logging onto a “locked” PC.

WINDOWS INSTALLATION: The D5 installed smoothly in both XP and Windows 7 without needing any drivers. Windows reported it, interestingly, as a “C-Media USB Headphone Set”. The only sample rates and bit depths available are 16/44 and 16/48 as shown to the right in Windows 7 (click for larger).

SUBJECTIVE SOUND QUALITY: There was moderate hiss and noise with my Ultimate Ears IEMs and it was noticeably worse than the Turtle Beach Micro II and about the same as the Syba CM119. The sound quality seemed similar to the Micro II with its odd dynamic range control turned off but the D5 has more output and seemed to sound a bit better driving 16 ohm headphones than the Syba CM119. A blind test would be required to know for sure, but I’m fairly sure the D5, like the CM119 and Micro II, has some audible flaws.

MEASUREMENT SUMMARY: The overall results were not very impressive and generally similar to the Turtle beach Micro II and Syba CM119 which use two other chips from C-Media. This doesn’t speak well for C-Media based products when three different DACs, using three different C-Media chips, all have significant problems. There are some differences between the three. The D5 has the highest line and headphone output voltage and the lowest output impedance but the Micro II and CM119 have better frequency response into a line level load. The CM119 does especially poorly with lower impedance headphones. Here are the results compared to the $12 CM119, $25 Turtle Beach Micro II, and the $30 Behringer UCA202. The scores are “A” (excellent) through “F” (unacceptable):

Measurement	FiiO D5	CM119	UCA202	TB Micro II
Freq. Resp. 10K	+/- 1.5 dB C	+/- 1.0 dB B	+/- 0.1 dB A	+/- 1.0 dB B
Freq. Resp. 33 ohms	+/- 1.5 dB C	+/- 6 dB F	N/A	+/- 1.8 dB C
HP Output Imp	0.72 Ohms A	5.9 Ohms C	47 Ohms F	0.95 ohms A
Max Output 10K	1.5 Vrms B	0.95 Vrms C	1.12 Vrms B	1.34 Vrms B
Max Output 33 Ohms	1.4 Vrms B	0.68 Vrms C	N/A	1.26 V B
Max Power 32 Ohms	61 mW B	14 mW C	N/A	50 mW B
THD+N 0 dBFS 10K	0.24% C	0.035% B	0.008% A	0.14% C
THD+N 100hz 10K	0.08% C	0.035% B	0.007% A	0.025% B
THD+N 1Khz 10K	0.08% C	0.035% B	0.007% A	0.02% B
THD+N 1Khz 33ohms	0.08% C	0.095% C	N/A	0.12% D
THD+N 10Khz 10K	0.04% B	0.090% C	0.009% A	0.11% C
IMD CCIF 10K/33	011% D	0.028% D	0.005% A	0.028% D
IMD SMPTE 10K	0.80% D	0.012% B	0.002% A	0.02% B
Noise A-Wtd dBu	-90.0 C	-89.0 C	-88.8 C	-93.8 B
-90 dBFS Linearity	0.7 dB A	0.9 dB A	3.8 dB C	0.8 dB A
USB Jitter Jtest	Poor D	Poor D	Very Good B	Poor D

FIRST CLASS:

• It’s cheap
• Nice build quality
• Volume & Mute controls
• Coaxial digital output

ECONOMY:

• Likely worse than many built-in (motherboard) sound outputs
• Significant low frequency roll off from headphone and line outputs
• High overall distortion, especially at low and high frequencies
• Relatively noisy
• Relatively high jitter

BOTTOM LINE: The extra features, especially the Mic Mute button, make the D5 especially well suited for things like Skype and web chat. But, otherwise, it’s external features and impressive build quality are only “skin deep”. What’s inside is yet another poorly performing C-Media chip. Of the three C-Media based USB DACs I’ve tested this one has the most features and highest output but is slightly less laptop friendly for portable use. Each of the C-Media DACs I’ve tested has a few unique relatively strengths and weaknesses. See the Measurement Summary above and Tech Section below for more details. As a secondary audio source for non-critical applications, or perhaps as a primary source for voice chat, Skype, etc, the D5 is a reasonable choice. For high quality audio, however, the Behringer UCA202’s line outputs perform far better for about the same price. And, as documented in my next two reviews, there are some better choices for portable USB “thumb DACs” for only slightly more money.

 

---

TECH SECTION

---

TECH STUFF: Based on information from the web, the D5 appears to use the C-Media CM108. Like the CM119 and CM102, used in the Syba and Micro-II respectively, it’s a fully integrated single chip that does everything necessary for a USB headphone DAC with a few extras thrown in. The volume control is simply another USB endpoint to allow remote control of the PC operating system volume and mute control. This is natively supported in Windows XP, Vista and Win 7 with no special drivers.

HEADPHONE TESTS: As with the other recent DAC tests, I used a 33 ohm load to represent typical portable headphones in the 16 – 80 ohm range. I also spot checked a few results into other impedances from 15 to 600 ohms. These tests were made from the headphone jack.

LINE OUT TESTS: I used 10K which is the typical input impedance of many headphone amps, such as the O2, and powered speakers. Performance into 22k or 50K loads will be very similar. I ran a few tests at 100K. I used the line out jack for these tests.

FREQUENCY RESPONSE: The other two C-Media based DAC’s had relatively flat low frequency response into 10K but not the D5. From either output it rolls off the bass so it’s down about 3 dB at 20 hz. This implies an undersized coupling capacitor that’s “upstream” of the line out buffer and headphone amp. Some might argue this amount of roll off is inaudible as the ear is less sensitive at very low frequencies, but it’s still disappointing. It means significant phase shift much higher in the audio spectrum and with headphones and recordings that go down to 20hz, you just might hear the loss of deep bass. The high frequency performance suffered the same poor filtering, ripple, and early roll off as the other two C-media DACs. Again, this will create significant phase shift in the audio range. Besides the line output roll off, the other big difference is the 33 ohm headphone load didn’t change the low frequency roll off. So apparently it’s a direct coupled headphone output. The second graph shows the Micro II for comparison. Note the Micro II is flat to below 10 hz into 10K and even does a bit better into 33 ohms:

 

THD+N vs OUTPUT: The D5 has slightly higher maximum levels than the Micro II but somewhat more distortion into 10K. The headphone jack into 33 ohms managed a relatively impressive (for this class of product) 1.4 Vrms. Into 15 ohms that dropped to 1 Vrms before hitting 1 % THD and into higher impedances it would hit 1.5Vrms. Power levels into 16, 32, 80, 150, 300 and 600 ohms are 63 mW, 61 mW, 28 mW, 7.0 mW and 3.7 mW respectively. While the headphone jack has more output than average, the distortion from both jacks is disappointingly high—generally around or well above the 0.05% goal. The second graph shows the Micro II for comparison (the trace colors are swapped and horizontal scales different):

 

100 hz 0 dBFS THD+N & OUTPUT IMPEDANCE: The first graph shows the line output can mange the same 1.5 Vrms as the headphone output into high impedance loads. At 0 dBFS the distortion was 0.24% which isn’t great but I’ve seen worse. At least the worst of it is the 2nd harmonic. In the second graph the D5 headphone output is over 1% THD at 1.04 Vrms. Removing the load increases the output to 1.09 Vrms for an output impedance of 0.72 ohms. This is fairly impressive and is another sign there are no output capacitors. The third graph shows the Micro II for comparison:

 

THD+N vs FREQUENCY: The D5 did especially poorly on this test at 0 dBu (775 mV). There’s little difference between the line out jack into 10K and the headphone output into 33 ohms. Both are around 0.06% to 0.10% over most of the audio spectrum and much worse below 100 hz and above 14 khz. This is not what I would call a “clean” DAC. As can be seen by the second graph of the Micro II, it’s significantly worse than even similar C-Media peers. The third graph shows the $30 Behringer UCA202 which has about twenty times less distortion (0.007%) from its line output jacks. An extra $5 buys much better performance despite the UCA202 being a much older design:

 

SMPTE IMD 33 OHMS: Even at only around 400 mV the dScope’s calculated SMPTE value of nearly 0.8% was pretty awful. Some of the IMD products clustered next to the 7 Khz signal are well above the desired –80 dB, as is the THD from the 60hz signal. The rest of the spectrum isn’t so bad. While the calculated value is higher, the spectrum is very similar to the Micro-II:

 

CCIF IMD MICRO II 44 Khz 33 Ohms: The D5 wasn’t quite as awful as the Micro II on this test but it was still fairly similar with lots of distortion products above –80 dB. This is rather poor performance:

 

CCIF IMD MICRO II 44 Khz 10K: Into 10K the CCIF IMD is only slightly better than into 33 ohms above. This is similarly bad to the CM119 and Micro II. The 48 Khz sampling rate was also similar to the Micro II. The second graph shows the Behringer UCA202 for comparison and more how it should look:

 

NOISE & LINEARITY: The weighted noise of the D5 was about 5 dB worse than the Micro II and very similar to the Syba CM119. This is marginal noise performance but is still acceptable for some applications and similar to the UCA202. The linearity was very similar to the Micro II and CM119 with less than 1 dB of error. The second graph shows the Micro II for comparison:

 

JITTER: Here’s the spectrum from the dScope’s J-Test for jitter. The result shows a lot of low frequency jitter (“spread” in the 11025 hz signal) almost identical to the Syba CM119. It’s much worse than average even compared to the $30 UCA202. At least the frequency accuracy (clock accuracy) is very good as shown by the frequency reading on the left. The second graph shows the UCA202 for comparison. Note how the “spread” is confined to below –118 dB on the UCA202 but reaches up beyond –70 dB with the D5:

 

TECH SUMMARY: Having reviewed the Turtle Beach Micro II, Syba CM119, and now the FiiO D5, it’s fairly obvious three different C-Media USB DAC chips all suffer some of the same major weaknesses. They include:

• Poor High Frequency and IMD Distortion
• Poor Jitter Performance
• Poor High Frequency Response
• Poor High Frequency Filtering (frequency response ripple in the audio band)
• Excessive Phase Shift In The Audio Band
• Excessive Low Frequency Roll Off Via Headphone Outputs (and line output for the D5)

To be honest, I would not choose any of these DACs for myself except for voice chat or a similar non-critical application. And for that use, the FiiO D5 is probably the winner because of its extra controls and features. Otherwise all three C-Media based DACs offer relatively poor audio performance. It’s likely most computers have better sound hardware built in—especially if it’s by Realtek, Intel, or Creative. The relatively poor performance of these three C-Media DACs also makes me wonder about C-Media in general. Some high end DACs, like the Schiit Audio Bifrost, use C-Media chips and I have to wonder if they too have excessive jitter, etc? Check back soon for two more DAC reviews that offer better performance.

UCA202 DAC Take 2

INTRO: The Behringer UCA202 was the first USB DAC I reviewed on this blog back in February. It sells for around $30 and has been around for many years. I’ve revised several of my tests and I was curious to see how it compares, apples-to-apples, with more modern inexpensive USB DACs like the Turtle Beach Micro II. I only tested the line outputs in this review as the original review established the headphone output has some serious problems—most notably a very high output impedance.

BEHRINGER UCA202: The UCA202 is larger than most newer inexpensive USB DACs (many of which are about the size of a USB thumb drive). The UCA202 is about the size of a deck of cards and has a fairly long (1 meter) attached USB cable. Like the Micro II it has an optical digital TOSLINK output. Unlike most newer inexpensive USB DACs, it also has a volume control but it only affects the headphone output. It has RCA line outputs and inputs.

WINDOWS INSTALLATION: The UCA202 installed smoothly in both XP and Windows 7 without needing any drivers. Windows reported it as a “USB Audio CODEC”. Three sample rates are available 32, 44 and 48 Khz all at 16 bits as shown to the right in Windows 7.

SUBJECTIVE SOUND QUALITY: Running the UCA202’s line outputs into my 02 headphone amp the sound quality was good with no obvious problems. There was some hiss audible at high gain settings on the amp but at realistic volume/gain settings it was reasonably quiet. Someday I plan to do an ABX blind comparison between the UCA202 and other DACs using the O2 amp.

MEASUREMENT SUMMARY: The overall results are substantially better than the Micro II and mostly quite good for a $29 DAC. The UCA202’s weak areas are noise and low level linearity (and the poor headphone output). To save space and leave room for more columns in future reviews, I’ve replaced my previous “Excellent”, “Very Good”, etc. ratings with a letter grade from A to F where A is excellent and F is Fail (unacceptable).

Measurement	UCA202	TB Micro II	FiiO E7
Freq. Resp. 10K	+/- 0.1 dB A	+/- 1.0 dB B	+/- 0.5 dB A
Freq. Resp. 33 ohms	N/A	+/- 1.8 dB C	+/- 0.5 dB A
HP Output Imp	47 Ohms D	0.95 ohms A	0.13 ohms A
Max Output 10K	1.12 Vrms B	1.34 Vrms B	1.87 Vrms B
Max Output 33 Ohms	N/A	1.26 V 52 mW B	1.4 V 59 mW B
THD+N 0 dBFS 10K	0.008% A	0.14% C	0.05% B
THD+N 100hz 10K	0.007% A	0.025% B	0.007% A
THD+N 1Khz 10K	0.007% A	0.02% B	0.007% A
THD+N 1Khz 33ohms	N/A	0.12% D	0.015% B
THD+N 10Khz 10K	0.009% A	0.11% C	0.007% A
IMD CCIF 10K	0.005% A	0.028% D	0.027% C
IMD SMPTE 10K	0.002% A	0.02% B	0.008% B
Noise A-Wtd	-88.8 dBu C	-93.8 dBu C	-98.7 dBu B
-90 dBFS Linearity	3.8 dB C	0.8 dB A	0.0 dB A
USB Jitter Jtest	Very Good B	Poor D	Very Good B

FIRST CLASS:

• Low distortion even at high frequencies.
• Flat frequency response
• Relatively low jitter
• RCA Line Outputs
• RCA Line Inputs can be used for recording

ECONOMY:

• Poor headphone output
• Just over 1 volt RMS maximum output (typical of USB powered DACs)
• Higher than average noise
• Marginal low level linearity
• Flimsy plastic construction

BOTTOM LINE: So far the UCA202 is still the best low cost USB DAC I’ve tested if you don’t use it to drive headphones. I’ll be testing other low cost DACs in the coming weeks to see how they compare. The UCA202 isn’t as small as some newer USB DACs but it’s still relatively portable.

 

---

TECH SECTION

---

TECH INFO: The UCA202 uses the ubiquitous TI PCM2902 integrated USB DAC chip. It’s an old design, but as you’ll see, it easily outperforms the much newer C-Media chip in the similarly priced Turtle Beach Micro II when using the line output. In this case, newer isn't better.

LINE OUT ONLY: I only tested the line outputs as the previous UCA202 review established the 47 ohm headphone output impedance renders it a poor choice for most headphones. Unlike other headphone DACs I’ve tested, you won’t see headphone loads being used in the tests below. All tests were run with a 10K load unless otherwise specified. Please see the original UCA202 review for tests of the headphone output.

FREQUENCY RESPONSE: The frequency response with a 10K load (such as a headphone amp) via the line outputs was substantially better than the TB Micro II. It’s within 0.4 dB from 10 hz to 20 Khz. There are no significant issues here:

 

THD+N vs OUTPUT: This test starts at 10 mV where noise dominates the measurement. Despite having higher noise than the Micro II, the UCA202 distortion drops much quicker implying it has much less quantization distortion at low levels. Because the UCA202 is likely to be used at line levels with an external amp the distortion will be around 0.01% or even less at typical levels. It has slightly less maximum output (about 1.1 Vrms) compared to the Micro II but much lower overall distortion. The Redbook standard for digital home gear is 2 V rms. But USB powered DACs usually produce around 1 – 1.5 Vrms:

 

THD+N 100 hz 0 dBFS: With the PC volume at maximum, and a 0 dBFS input, the UCA202 produces about 1.1 Vrms at very low distortion. This is excellent performance with all harmonics below the magic –80 dB threshold. Way out of band you can see a spike at the 44 Khz sampling frequency which is fairly normal—especially in low cost DACs. The odd bump in the noise floor above 20 Khz is related to intentional noise shaping in the PCM2902 DAC. It’s an intentional design technique to lower noise within the audio band:

 

THD+N vs FREQUENCY: Here’s the THD+Noise plotted from 20 hz to 20 Khz into 10K (blue) at 775 mV (0 dBu). The Micro II is shown, for comparison, in yellow. The input is –3 dBFS to prevent any digital overload of the DAC. The UCA202 does much better here staying below the ideal 0.01% threshold until about 11 Khz. This is very good performance for a wideband test:

 

SMPTE IMD: This an excellent result with essentially no IMD products and everything well below –80 dB. In comparison, the Micro II had 170 times more distortion on this test:

 

CCIF IMD 44 Khz: The UCA202 does extremely well here for an inexpensive DAC with more than 50 times less distortion than the Micro II at the same level into the same 10K load. More important, all distortion products in the audio band are well below –80 dB. The two spikes around 25 Khz are a little alarming but only your dog might hear them. See the Micro II review CCIF section for both it’s very poor performance on this test and the DAC1’s reference plot. This is excellent for any reasonably priced DAC running at 16/44:

 

CCIF IMD 48 Khz: The UCA202 also does well, although slightly worse, at 48 Khz:

 

NOISE & LINEARITY: The UCA202 is only average for noise and worse than average for linearity. A –90 dBFS signal is reproduced at –93.8 dB for an error of 3.8 dB. There’s less than 1 dB of error at –80 dBFS so the problem is limited to extremely low levels. Noise referenced to my old 400 mV dBr reference would be 83.1 dB and against full output it’s 91.8 dB both A-Weighted. The good news is the noise will be upstream of the volume control if an external amp like the O2 is used and the PC volume is set to maximum. That means you’ll get around 90 dB of real world S/N ratio which is sufficiently quiet. But if you plan to use the PC’s volume control, the noise and/or linearity might be an issue in some circumstances. The spikes at 2, 3, 4 and 5 Khz are likely quantization distortion of the 1 khz low level signal and are relatively typical. I don’t know what’s responsible for the spike at 30 hz but it’s very inaudible.

 

JITTER: Here’s the spectrum from the dScope’s J-Test for jitter. The two things to look for are the number and level of symmetrical sidebands and the “spread” at the base of the 11025 hz signal. Compare the result below to the Micro II’s Jitter to see how much better the UCA202 does. This is a very respectable result for an inexpensive USB DAC and the frequency accuracy (clock accuracy) is very good as shown by the frequency reading on the left:

 

TECH COMMENTS: Used to drive an amp, powered speakers, or other source with a line input, the TI PCM2902 based UCA202 blows away the Turtle Beach Micro II based on the C-Media CM102. The only weak areas are the noise and linearity below –80 dBFS. But neither of those is likely to be an issue if you leave the PC volume all the way up and use the volume control on the amp/speakers.

DAC Listening Challenge Results

A BIG THANK YOU! I appreciate all the responses to both listening tests. I know listening for subtle differences requires some effort and I want to thank everyone for taking the time and sharing their results.

NOTE TO HEAD-FI MEMBERS (revised 3/28):  Unfortunately, before I posted this article, the administrators at Head-Fi.org started censoring links to this blog. So if you had some trouble finding the results here, I’m sorry. The links were apparently not a problem when I was posting at Head-Fi on several other products and topics. The censorship only started after a paying Head-Fi sponsors (NuForce) publically complained about my review of their product. Head-Fi has gone so far as even deleting posts from other Head-Fi members that reference this blog.

BACKGROUND: NuForce responded to their uDAC-2 measuring poorly by saying it was designed to sound good even if several measurements are notably bad. So I came up with the best way I knew how to judge the NuForce purely on sound quality. I also thought it would be interesting to compare the $29 Behringer UCA202 with a high-end product like the Benchmark DAC1 Pre. The DACs were recorded playing real music under as realistic and similar conditions as possible. Anyone could download the recordings and compare them without knowing which was which—sort of like a “brown bag” wine tasting. Would people like the $3 wine better than the $30 wine? I thought it would be fun to find out!

TEST METHODS: Not having conducted a public web-based listening test before, this was something of a learning experience for me. For the first round, to make the comparison as fair as possible, I used the line outputs of all three DACs . The second round of tests used the headphone outputs driving real headphones and included a modified version of the Behringer UCA202. The modifications were made using a few dollars worth of parts and improve the headphone output of the Behringer. The files were given names of US Presidents in the first test and common trees in the second. The original listening tests, and all the details, can be found here:

• NuForce uDAC-2 Listening Test
• DAC Listening Sequel

A NOTE ABOUT “MASKING” (revised 3/17): Some have questioned the validity of the tests because, as NuForce put it, the output of the DACs have been “re-digitized”.  And some also argue the equipment used to play back the files may not be high enough quality. The concern is these could mask the differences between the DACs. In reality this should not be much of an issue using playback hardware with reasonable fidelity. Here’s why:

• These trials are only about the differences between the files, not the absolute accuracy. It’s like shopping online for a shirt. You might look at 3 different blue shirts at the same online retailer. Even if your computer display can’t convey every color and detail perfectly, you can still easily tell most of the differences between the shirts if they’re good pictures, all taken the same way, and your have at least a reasonably decent display. For example if your computer distorts the exact shade of blue, you should still be able to tell which of the 3 shirts is the darkest blue. The same is true in comparing the sound files. Even if your headphones might exaggerate the bass, you can still tell differences between the bass in the various files. This “difference effect” has been well documented in research.
  
• As to “re-digitized”, the Benchmark ADC1 used to convert the output of the DACs to a CD quality file is already better than most studio gear used to record the music we all listen to. Studio A/D converters typically cost about $100 – $400 per channel of conversion. The ADC1 is a reference-grade A/D with a $900 per channel price tag. It generally has better specifications than nearly all studio gear. If the less expensive studio gear is good to enough to capture the subtle difference between say a high-end Steinway and a high-end Yamaha grand piano, the ADC1 should also capture audible differences between DACs.
  
• Most recordings have already been through many more steps and kinds of digital processing yet are still very revealing of subtle differences. So adding one more relatively “pure” step isn’t going to make much difference. The recordings used in these tests are unusually pure and transparent.

PARTICIPANTS: This was a very informal survey. A total of 20 listeners picked their favorite (and sometimes least favorite) tracks. Many ranked them top to bottom. Some only participated in one of the tests and/or one of the songs. Some used ABX and some listened conventionally. More participated in the first (line out) trial than the second (headphone out) trial.

SCORING: Not everyone participated in every trial, some devices were offered in more trials than others, and different listeners provided different sorts of “votes”. Someone who’s better with statistical analysis than I am is reviewing the raw data. But, for now, to summarize the results here’s my attempt at a rough analysis:

• Top choices scored 2 points
  
• Second (“Runner Up”) choices scored 1 point
  
• Least favorite (worst) scored –1 point (several indicated only their least favorite and not a favorite)
  
• The results are summarized by category (line output vs headphone output)

RESULTS (corrected 2 minor errors 3/16): Here are the total points scored for each downloaded file ranked from most favored (highest score) to least favored using the above scoring method:

Taft - Benchmark Line Out Brick House	9
Harrison - NuForce Line Out Brick House	8
Juniper - Benchmark CX300 Brick House	7
Wilson - Behringer Line Out Brick House	6
Jefferson - Reference CD Just Dance	4
Lincoln - Benchmark Line Out Just Dance	3
Oak - Behringer CX300 Brick House	2
Spruce - Mod UCA202 CX300 Brick House	2
Acorn - Benchmark CX300 Tis of Thee	2
Maple - Mod UCA202 CX300 Tis of Thee	3
Monroe - Reference CD Brick House (!)	1
Hawthorn - NuForce CX300 Tis of Thee	1
Fig - Behringer CX300 Tis of Thee	1
Jackson - NuForce Line Out Just Dance	0
Cypress - NuForce CX300 Brick House	0
Adams - Behringer Line Out Just Dance	-2
Pine - NuForce UE SF5 Tis of Thee	-4
Total Benchmark Line Out	12
Total Benchmark Headphone Out	9
Total NuForce Line Out	8
Total Behringer Line Out	4
Total Modified Behringer Headphone Out	5
Total Behringer Headphone Out	3
Total NuForce Headphone Out CX300	1
Total NuForce Headphone Out Ultimate Ears SF5	-4

COMMENTS ON RESULTS (edited 3/16): Even with the small sample size a few things were fairly clear:

• The NuForce headphone output with the Ultimate Ears SuperFi 5 Pro headphones was an obvious “fail”. Several expressed a clear dislike of “Pine” and nobody favored it. I think this is due to the large frequency response variations caused by the relatively high output impedance of the NuForce uDAC-2 with these headphones. Several commented the high frequencies were rolled off or “dull”. This consistent with what the measurements would predict.
• The reference CD tracks didn’t score as well as expected. For “Monroe” it’s likely because the reference track stood out as being different. Early in the forums it was labeled “by far the worst”. And that public comment likely tainted others into also hearing it as the “worst”. Six people rated it as the worst of that group of tracks. I personally think this is a good example of subjective bias--the original piece of music was deemed the “worst”! The Just Dance reference faired much better with a positive score of 4 likely because it wasn’t labeled publically as being bad. And much fewer even bothered to vote on Just Dance so the score of 4 was close to “perfect”.
• In terms of total votes, the Benchmark stood out as a clear favorite. It was included mainly as a “reference” and not necessarily as fair competition to the other two, much cheaper, DACs. But, some might have expected all the DACs to sound roughly similar via the line outputs based on the measurements. The could be many reasons why this didn’t happen. The most likely is the same “peer bias” that caused a reference track to be rated poorly (see above). But it’s also possible the cheaper DACs do have audible problems.
• The NuForce did much better via the line outs than the headphone outputs. Given the output impedance problem and higher distortion of the headphone output, this is what the measurements would predict.
• Any channel balance error was corrected in all tests. So this removed the audible imbalance of the NuForce helping it score better.
• NuForce’s claim of “better sound” doesn’t seem to be true when using the headphones. It faired poorly even with the more common and “impedance friendly” Sennheiser CX300’s.

Trying to draw other conclusions is a bit more difficult, but some things worth noting:

• Nearly half the people couldn’t hear any differences at all.
• The Just Dance track had a lot of inherent distortion that made hearing differences difficult for most. So those results are likely less valid (and there are fewer of them).
• A number of votes were somewhat randomly distributed between all the middle scoring test files. This may suggest more guessing rather than clear preferences.
• The line outputs of  the $29 Behringer didn’t do as well as its measurements would suggest. The NuForce and Benchmark were preferred. This could be “peer bias”.
• The differences among the individual files were more obvious via the headphone outputs rather than the line outputs. This is to be expected due to the impedance interaction.
• For those who argue tests like this mask differences, it’s interesting to note the test may not have masked any subtle advantages of the high-end Benchmark. This is despite the fact that few listened to the files on similarly high-end gear.

BOTTOM LINE: I think the most obvious thing is roughly half the participants couldn’t tell any difference and another group of the “middle scoring” results is almost random with no clear preferences. Comparing the headphone outputs, however, the differences seemed more obvious. And they were very obvious when using balanced armature headphones (the SuperFi’s) on the NuForce. It’s clear the frequency response variations created by the NuForce’s relatively high output impedance creates audible problems.

It’s apparent subjective opinions of differences are easily swayed by public comments. The reference track from the CD of Brick House was strongly disliked by six listeners. They heard something different and were easily swayed into thinking different was worse based on previous public comments. It just took one person to say something negative and several others followed. This is exactly the sort of subjective bias that affects the majority of online subjective reviews in forums. As further proof, others using ABX (which is blind) voted the same track their favorite.

There’s also significant evidence the NuForce uDAC-2 does not sound better—at least using 2 different types of headphones—as NuForce claims it does. But it seemed to do a respectable job via the line outputs—at least with the channel balance error removed.

Overall I think this has been an interesting experiment. I learned a lot about how to run (and not run!) a listening test. And there were some fairly clear results—some expected and some not. If there’s sufficient interest, I may try to build on what I’ve learned here and conduct future listening tests?

COMMENTS WELCOME: Please feel free to add comments to the end of this article on the results—especially if you participated. I’d also like to know how many are interested in future listening tests? They’re a fair amount of work to put together and only really valid if you get a reasonable number of votes. So it’s something I only want to do if there’s enough interest. Please feel free to make suggestions, etc?

---

TECH SECTION:

IMPROVING THE RESULTS: I realize this wasn’t the best run study. It was more an informal experiment than anything. I think a larger scale listening test, that had more uniformity, would be needed to verify some of the closer results. If I do this again, I’ll research better methods, and I also welcome input from others with experience in this area?

DIFFERENT LOADS: It’s been suggested it would be also useful to include high impedance headphones and, except for the complexity that adds to the mix, I agree that would be interesting. I chose low impedance headphones as they’re, by far, the most popular—especially for use with a portable entry level DAC.

PASSWORD: An encrypted 7-Zip file was included with the file descriptions to prevent me from cheating or changing anything after people’s votes. The password is:

CY&YUMN5cZ9x2X8BhNj2t

PINE EXPLAINED: The graph below, in blue, shows the frequency response of the NuForce uDAC-2 using the Ultimate Ears headphones used in the trial:

The 4+ dB of response variation seen above is caused by the relatively high output impedance of the uDAC-2 interacting with the SuperFi headphones (typical of balanced armature designs from many manufactures). For more on this see Headphone & Amp Impedance.

OTHER DETAILS: The other details of how the test was run can be found in the original articles:

NuForce uDAC-2 Listening Test

DAC Listening Sequel

NuForce uDAC-2 Listening Test

BACKGROUND: (revised 3/7/11) After posting my review of the NuForce uDAC-2 the company has emphasized they compromised the measured performance of the uDAC-2 in the interest of better sound. For example, they have said they chose the volume control potentiometer, even when they’re aware it has significant channel balance issues, because it sounds better. They also have argued they intentionally allow the uDAC-2 to clip internally (creating higher distortion) because this makes the product sound better in real world use. So I wanted to provide a way for most anyone to compare the sound of the uDAC-2 against two other DACs under conditions that are as identical as possible.

THREE VERY DIFFERENT USB DACs:

• NuForce uDAC-2 ($129) - This is a popular second generation USB DAC from a company that specializes in mainly DACs. It sits above the $99 uDAC2-hp in their product line. It has line, headphone and digital outputs and uses a 24 bit/96 Khz chip. Here’s my detailed review.
• Behringer UCA202 ($29) – This inexpensive product is the lowest priced USB DAC from Behringer--a pro audio company that makes hundreds of products but only a few DACs. It has line, headphone, and digital outputs and uses a 16/48 chip. Here’s the full review.
• Benchmark DAC1 Pre ($1595) – This is an award winning, very well reviewed USB DAC/Preamp with excellent measurements. It has line and headphone outputs and uses 24/192 chips. The Benchmark is included mainly as a reference to help establish any “sound” the Benchmark ADC1 doing the recording might have. It represents a product well past the “point of diminishing returns”.

REFERENCE TRACKS: One track was chosen from two very different original CD’s:

• Audiophile Track – Sara K.’s “Brick house” from her Hobo CD on the Chesky audiophile label. This very well made recording has a lot of acoustic instruments and sounds, female vocals, and much more room ambience than a typical recording. It also has a wide dynamic range and only rarely reaches the full 0 dBFS digital signal level a few times. So the clipping problem with the NuForce uDAC-2 should be minimal with this track.
• Popular Track – Lady Gaga’s “Just Dance” from her The Fame Monster CD. This represents a typical current pop track that’s mastered to sound relatively loud by restricting the dynamic range. It was played back exactly as it was ripped from the CD with no digital level changes. To the possible benefit of the NuForce, the track itself has a lot of distortion which may well mask the clipping present in the uDAC-2.

WHY SHORT EXCERPTS: To avoid large file sizes, make downloading faster, and comply with copyright laws, a 15 second excerpt was chosen from each of the above tracks. The Brick House portion was chosen to represent as wide of variety as possible. The Just Dance excerpt was chosen from one the louder sections of the track to hopefully expose the 0 dBFS problem with the uDAC-2.

Short excerpts work especially well when doing ABX comparisons. Most people familiar with ABX testing end up comparing only a few seconds of music back and forth. The brain can remember a few seconds of a previous excerpt much easier than longer clips.

THE SOUND FILES: There are four FLAC sound files, and four MP3 files for each test track plus two more small files. If you would prefer to use just the Audiophile track, or just the Pop track, you can download just the four files for that track. There are a total of 18 files but you only needs as few as 4 or at most 10.

• 2 Reference tracks in MP3 & FLAC – These are the original tracks from the CD
• 2 NuForce uDAC-2 tracks in MP3 & FLAC – Recorded from the line output of the uDAC-2
• 2 Behringer UCA202 Tracks in MP3 & FLAC – As above for the UCA202
• 2 Benchmark DAC1 Tracks in MP3 & FLAC – As above for the DAC1 Pre
• 1 Copyright Notice – (keep the lawyers happy)
• 1 encrypted text file – More on this later

HOW THE SOUND FILES WERE MADE: The goal was to minimize as many differences as possible, and document the procedure sufficiently to let someone else reproduce it. That way someone can verify my results and/or compare other USB DACs in a similar way. The following apply to all 6 recorded files (see the tech section below for more details):

• The same two uncompressed native 16/44 reference tracks were played in Foobar 2000 for all three devices.
• The line outputs were used and carefully matched to same level.
• All three were connected to a Benchmark ADC1 professional A/D converter using the same cable and any channel balance differences were corrected.
• The files were recorded at CD quality to uncompressed files.
• The average volume of the files were slightly adjusted as needed to be as similar as possible.
• The files were tagged and compressed to both lossless FLAC and lossy MP3 formats.

MYSTERY FILES: At the suggestion of others, I’ve decided to make this a blind test and initially keep each file a mystery. There are some very good reasons for this. When a listener knows what they’re listening to, several kinds of bias influence their opinions. This has been proven many times in many ways. So to minimize bias, the files are named in a generic way. I’m also including a small encrypted file that contains descriptions of each file. When I reveal the source of each file, I will also reveal a password for the encrypted file. This allows anyone to verify I have not changed the assignments since the start of this trial. Here’s the basic plan (unless there’s consensus on improving it):

• Initially all the files will be “blind”. During this time I encourage anyone to download the FLAC or MP3 files, compare them, and hopefully comment on the differences either at the end of this article or in the appropriate forum thread—ideally pick a favorite or two out of the bunch. You can also download the Descriptions file if you want to keep me honest.
• When there’s enough feedback, I will reveal the two reference files. This will allow anyone to A/B compare each DAC file against the reference track. It will be interesting to see if this changes any preferences among the files.
• After a suitable time, I’ll reveal the remaining 6 files along with the password to decrypt the descriptions and publish a summary of the results.

MORE INFORMATION ON BLIND TESTS: For more information on why blind listening is better, you might want to check out one or more of these links:

• Dishonesty of Listening Tests (some good info by Sean Olive)
• Matrix HiFi Blind Test (a typical blind test and the surprising outcome)
• Blind Listening Tests (Hydrogen Audio – rather technical info)

HOW TO COMPARE THE SOUND FILES:

Download 4 or 8 MP3 Format Files – These are faster to download, and are very high quality, but use lossy compression which could mask very subtle differences. They can be played on most anything.

Or, Download 4 or 8 FLAC Files – These take longer to download and require a player that supports FLAC but use lossless compression for ideal sound quality. For a Windows PC, I recommend Foobar 2000 as it’s free and accurate. But many other players work including VLC, MediaMonkey, etc. If you have a portable player, several will play FLAC files including the newer Sansa players, Cowon, HiFiMan, and most anything running the Rockbox firmware. Some home music players/devices also work such as SlimDevices (Logitech), Sonos, etc.

Listen To The Files – Using a FLAC compatible player (see above), or most anything for MP3, listen to the excerpts. I would suggest using the most revealing setup you have. Most people find even moderately good headphones to be more revealing than speakers, but it all depends on what you have and your preferences.

ABX Compare The Files (optional) – For the most revealing comparison, if you have a Windows PC and some decent headphones or speakers, you can use Foobar 2000 to do your own solo blind test. Here’s how:

• Download and install the small, simple, and free Foobar 2000 player
• Follow these directions for installing the ABX comparator (hint: start from File > Preferences > Components in Foobar 2000).
• Once the ABX component is installed and shown in the list of Components, you can perform an ABX comparison by opening the two tracks at the same time (File > Open then hold down the Ctrl key to select both tracks and click Open), stop the player if it automatically starts playing, and then right click on either of the two tracks in the playlist and select Utilities > ABX Compare. From there clicking the “A” button always plays track A, and the “B” button always plays track B.
• If you’re not familiar with ABX and/or Foobar, this YouTube video (not by me) is helpful: ABX Audio Testing with Foobar 2000 (by homebrewedmusic)

Share Your Observations: Please note what differences you might hear and comment at the end of this article, or in a forum thread if you want. But, for now, if you’re fairly certain you know the origin of one or more of the excerpts, please keep it to yourself to keep it “blind” for the others. Or send me a private message.Ideally, pick your top favorite or two to help me “score” the results.

THE FILES: With the listening test over, the files have been removed to save bandwidth.

HEAPHONE VERSION: (updated 3/7) Some of you asked for a similar test with the headphone outputs, and I’ve done that with a few other twists here: Headphone DAC Listening Test Sequel

RESULTS (updated 3/16): The results have been published. This has been an interesting experiment, and if there’s enough interest, I may do it again as I’ve learned a lot from these two tests.

---

TECHNICAL INFORMATION:

SARA K IN AUDIGY:

LADY GAGA IN AUDIGY: Believe it or not, this is just as it comes off the CD completely unprocessed (by me, obviously the recording engineer at the music label had other ideas):

THE PROCESS:

1. The reference tracks were ripped at their native format of 16 bits/44.1 Khz without any level changes (i.e. no normalization or volume leveling). Steinberg’s Wavelab version 6.1.1 was then used to extract the 15 second segments from both tracks.
   
2. The Just Dance track already had peaks of 0 dBFS and was left exactly as it is on the CD (as you see above!). Like many pop CD’s, this one was clipped during the mastering process. The Brick House excerpt was normalized to 0 dBFS so the same gain settings could be used during recording for both.
   
3. Both tracks were played in Foobar 2000 version 1.1.4 in WAVE format. Foobar was set for 0.0 dB gain with no equalization, DSP, plug-ins, or other processing for each of the recorded files.
   
4. The operating system was Windows XP SP3 with all current Microsoft updates. The master volume slider was set to full and all other inputs were muted. XP and Foobar were configured to deliver a bit accurate stream to all three DACs.
   
5. The Behringer UCA202 has fixed level line outputs which measure approximately 1.2 volts RMS into 100K with a 0 dBFS input signal. So this determined the output level for the other two DACs. The volume controls on the uDAC-2 and DAC1 were adjusted to match the UCA202’s line output within 0.05 dB (matched to approximately +/- 0.01 volts RMS). The levels were set playing a digitally (Wavelab) generated ideal 0 dBFS 1 Khz stereo sine wave file in Foobar 2000.
   
6. The same professional Neutrik cables were used to connect each DAC to the Benchmark ADC1 for recording. The levels on the ADC1 were set to –1.0 dBFS using the same 1Khz 0 dBFS file used to set the DAC output level. The clip hold indicators were enabled on the ADC1 and never illuminated during any of the recordings. The full measured specs of the ADC1 are available here ADC1 Specs. The –1.0 dBFS recording level was chosen to avoid clipping from frequency response variations due to the DACs.
   
7. Any channel imbalance in the DACs (not the excerpts) was removed by adjusting the ADC1’s level controls as necessary to within +/- 0.05 dB.
   
8. Wavelab was used to record each track also at the native format of 16/44. This format was chosen as the majority of people evaluating the files likely do not have the ability to play back 24/96 files without sample rate conversion. This way the test files can remain bit accurate from the output of the ADC1 to anyone who plays them.
   
9. Each recording was trimmed at the start to match the reference track (within about +/- 0.5 mS). The ends of the files were trimmed to match the overall length of the reference track.
   
10. Each track was analyzed for perceived volume using Wavelab’s Average RMS Volume Analysis. The loudest track in each set was then used as the reference and the other 3 tracks were normalized to the same average volume in the left channel to within +/- 0.01 dB. Notes were kept on each track’s data.
    
11. The files were then saved as WAV files, in no particular order, adding unique names to the end of each.
    
12. The files were then tagged in MediaMonkey version 3.2.4 and saved as FLAC with a compression level of 5.
    
13. Another set of matching files was made and saved in 192K VBR MP3 format using LAME with the well established “–alt –preset –standard” settings.
    
14. The Description file was made and encrypted using the free 7Zip file compression utility. The password will be revealed later.