The HD-Audio Challenge II
A Little Wildfire Drama
Just about every place one might choose to live, there’s a chance that mother nature will bring some sort of natural disaster to bear. At 2:30 am this morning my iPhone started beeping loudly due to a wildfire that was raging out of control in the Sepulveda pass. I looked outside and could see the smoke and faint red glow only a few miles away — and the famous Santa Ana winds were pushing the fire southwest towards my home. Initially we weren’t in the mandatory evacuation zone but as the fire continued to burn up and down the hillsides, our home just up the hill from the ocean was inside that zone. So we collected everything we thought would be missed, stacked up boxes by the front door, positioned the car, and prepared for the final notice. Thankfully, it never came. With sunrise at 7:09 am, the large fixed wing DC-10 chemical retardant planes took over from the helicopters and managed to stop the advance — at least for now. The Santa Ana are coming back on Wednesday. We won’t be unpacking the boxes just yet.
The HD-Audio Challenge II Is Coming – The Week!
I spent the weekend gearing up for the second round of the HD-Audio challenge. Some of you may remember the first iteration of this study (click here). The music industry seems intent on continuing to push their claims that “hi-res audio” is a tremendous advance in the evolution of music reproduction. After being involved with real high-resolution audio for almost 20 years, I’m not so sure it matters. I’d love to demonstrate that hi-res music and hi-res audio are delivering a “better” experience, but the studies I’ve read have left me unconvinced.
I believe that I can contribute to the debate by offering up a catalog of real high-resolution tracks in a variety of formats. You — my readers and fellow audiophiles — can download the tracks and play them to your heart’s content. I only ask that you not analyze them to determine which is which. What’s the point of cheating?
I’ve selected 20 tracks from a variety of genres and took into consideration suggestions from many of you. I’ve included solos, small and large ensembles, acoustic and electric, and vocal vs. instrumental http://www.realhd-audio.com/?p=6197 .The tracks that will be available are listed below:
These are full length tracks not merely samples.
I spent all weekend converting the tracks using Sonic Studio’s professional software tool PROCESS to do the conversions. I took the native 96 kHz/24-bit PCM masters and downconverted them to 96 kHz/16-bit, 44.1 kHz/24-bits, and 44.1 kHz/16-bits (CD “Redbook” spec). I will also create constant bitrate MP3 versions at 320 and 256 kbps for those interested in compressed formats. Then I converted all of downconversions back to 96 kHz/24-bits so that all of them are precisely the same size. I have been very careful to ensure that they are the same volume.
I have uploaded all of the files to a folder in my premium Dropbox account and will “Share” the contents with those interested in participating in the study. The files are randomly named and should provide a rich opportunity for those willing to download them and do some serious listening. The goal is to discover if bona fide high-resolution audio recordings can be distinguished from lower resolution formats.
A Preview
I will be doing a thorough analysis of each file and providing the spectra and dynamic analysis to participants. I’ve already done that for a test file by The Latin Jazz Trio. Here’s the spectra of all of the formats:
Sign Up
The more audio enthusiasts that participate in this study, the more raw data I’ll have and the more valid the results will be. I’m prepared to be criticized for the casual nature of this experiment. Some will insist that using my own catalog is too limiting, others will insist that it be done in a state-of-the-art studio, or with mega buck equipment. I don’t believe that any of those things matter. We all have different rooms, systems of differing values, and varying abilities to listen…exactly the diversity that is required to establish whether the marketing claims made by the industry are true.
This should be fun. I’ll leave the files up for a couple of months. I have to report back to my university sometime in early 2020, so you’ll have lots of time. Thanks!
I just signed up for the challenge. I realized however that the number of formats will make this fairly difficult and very time consuming. I’m primarily interested in potential differences between true Hi-Res (96/24) and Red Book (44.1/16). Are we supposed to switch between 5 or 6 versions of the “same” track and attempt to discern audible differences? One could go about it systematically, say 1-2, then 1-3, 1-4 etc. but that sounds discouraging, definitely not fun as intended.
Hi Michael and thanks for you interest. I’m interested in the comparison that you mentioned. I will upload individual folders with just those two items for comparison. There will other options for people that want to spend additional time.
Sounds great (pun intended), thanks for clarifying!
I’m looking forward to taking the challenge and reading the results. Thank you for putting the challenge together.
“And I have the tapes to prove it.” [06.02.24, Stereophile Forums]
—I’m doing the Sam Rivers book, the Sessionography going to press in early 2021.
Could you please contact me (rl@bb10k.com) about the Rivers Trio + Mingus Quintet at SUNY Long Island?
I’d sure appreciate it.
Hi Mark, I feel for your situation with the fires. Being in Central FL, every few years we experience something similar due to a hurricane. Being inland from either coast close to 100 miles, we haven’t had any serious damage here in my memory. None the less we run around preparing as we can and feel the emotional pressure.
Best of luck to you.
Sal
I just signed up for the challenge
Thanks, you should be receiving the credentials for the downloads soon.
I will try the ABX plugin for Foobar 2000.
I downloaded the corrected files and started to listen, but a few don’t match your track list. I don’t have the Chopin, the MVT or the Heart; in their place I have Schubert, Scarlatti, and “Something Like a Broken Heart.” Do I have the right tracks?
Here’s the final list. I had to change a couple of things since the first list was published:
1 MCQ Mozart Clarinet Quintet – Mvt 4 – Allegretto con Variationi 8:32 Mozart – Clarinet Quintet K. 581, Horn Quintet K. 407 & String Quartet K. 169 AIX 86065 The Old City String Quartet Classical Chamber Instrumental
2 IMP90 Impromptu Op.90 #1 In C 9:55 Bryan Pezzone – Piano Pieces from Mozart, Schubert, Elgar and Pezzone AIX 86064 Bryan Pezzone Classical Solo Piano Instrumental
3 FSF Stravinsky – Firebird Suite – Mvt. 4 Finale 3:37 Stravinsky – Firebird Suite 1919 / Ravel – Boléro 1339AX George Enescu Phil. Orch. – Cristian Mandeal, Conductor Classical Full Orchestra Instrumental
4 VS Vivaldi – Sinfonia No. 1 7:07 Marcello- Concerto, SF. 935 / Handel – Organ Concerto, HWV 295 / Vivaldi – Sinfonias RV 719 & RV 146 AIX 81002 George Enescu Phil. Orch. – Cristian Mandeal, Conductor Classical String Orchestra Instrumental
5 SSAM Scarlatti Woodwind Quintet – Mvt 4 2:28 Scarlatti: Six Sonatas for Woodwind Quintet / Beethoven: Quintet for Fl, Ob, Cl, Bsn, & Hn, Op. 71 1341AX George Enescu Quintet Classical Chamber Instrumental
6 BMS Bright Morning Star 3:35 Voices Unbound AIX 80012 Zephyr Classical A Cappella Choir Vocal
7 BSTJ Bubba Shot the Jukebox 3:26 Your Room AIX 86061 Mark Chesnutt Country Electric Male Vocal
8 TIU2 Tear It Up 3:41 Tearing It Up AIX 85054 Albert Lee Country Electric Male Vocal
9 DF Darcy Farrow 4:09 Nitty Gritty Surround AIX 80008 John McEuen and Jimmy Ibbotson with Jennifer Warnes Country Acoustic Male and Female Vocal
10 IG Italian Girls 5:03 The Gypsy Life AIX 85043 John Gorka Folk Acoustic Male Vocal
11 MOR2 Memories of Rio 5:20 The Latin Jazz Trio AIX 80011 The Latin Jazz Trio Jazz Latin Instrumental
12 PRI Primavera 4:41 Primavera AIX 85050 The Banda Brothers Jazz Acoustic Female Vocal
13 MA Moonlight Acoustica (segment) 4:55 Moonlight Acoustica AIX 80041 AIX All Star Band Jazz Electric Instrumental
14 LBIT Lulu’s Back In Town 2:34 Tormé Sings Tormé AIX 83041 Steve March Tormé Jazz Vocal Vocal
15 CIRC Circles 7:12 Gathered Around AIX 80026 Steve Huffsteter Jazz Big Band Instrumental
16 MYSI Mysterious Integratron 5:12 Span AIX 80037 Bad Haggis with special guest Rubén Blades Jazz Electric Small Instrumental
17 MMM My My My 4:52 Guitar Noir AIX 80018 Laurence Juber Jazz Acoustic Instrumental
18 SLABH Something Like A Broken Heart 4:00 Tried and True AIX 80048 Hanna-McEuen Country Rock Electric Male Vocal
19 OYIAIK Only You Know and I Know 3:31 Play Something Sweet AIX 86062 Rita Coolidge Pop Electric Female Vocal
20 LS Lone Star 4:19 The Carl Verheyen Band AIX 83039 Carl Verheyen Rock Electric Instrumental
There seems to be one flaw in this test to make a valid inference about whether there is a difference between the two resolutions.
Most people will not be able to ensure that they are actually hearing the 96khz without downsampling to 44.1 or more likely 48khz on their systems before converting to analog. This could happen in one of several ways. For example:
1. If playing through a media player in a Windows PC, it may be sending it to the default device in shared mode that is configured to convert all sources to 48khz in the device settings
2. The DAC one has internally or externally may not have the ability to process 96khz and may downsample before conversion to analog.
3. If they are playing through an AVR or an integrated amp, it may downsample all inputs to less than 96khz before its DAC either because it cannot do any better or that by default it is going through its DSP (for room eq or whatever) that can only handle max 48khz.
The implicit downsampling in any such cases may actually be worse than the competent downsampling done for the test cases here and so sound worse than the lores version.
My point is NOT to say there is a difference or not in listening to the higher resolution.
If the goal is to test whether most people who download hires would be unable to differentiate with a CD quality version because of any reason including their equipment limitations, then the structure of this test is fine.
But that is a different goal than testing whether a hires content sounds better or different than a CD quality version when both are played as they should be which I though was the motivation behind this test.
Given how difficult it is for anyone but dedicated audiophiles to ensure that their audio chain whether it is just a laptop or a hi-fi stack is going bit-perfect until the actual D-to-A conversion, I suspect the result would be skewed towards no difference or the hi-res sounding worse (from unknown downsampling in the equipment).
The above might seem to prove the point behind the test but is not statistically valid.
Again, I am NOT coming from a viewpoint that hires necessarily sounds better or is distinguishable when each is played bit-perfect until conversion to analog. I am intrigued enough to look at some tests for this but this test does not do so for test design/condition reasons.
The possibility that a large number of people who download hi-res music aren’t actually hearing what hi-res may potentially be capable of because of the typical equipment limitations is probably not that controversial despite marketing claims by the content providers.
Your point is well taken and has been raised by others. I will reinforce the importance of making sure that everyone listens to the files at 96 kHz/24-bits. My assumption — perhaps ill advised — is that as audiophiles, we would know how to insure that we play the files as intended.
I think we often conflate audiophiles with audio-techies. There is a reasonable intersection for sure but there are a lot of audiophiles (as in people who love good sound) who aren’t necessarily savvy with the equipment and how it works.
Even for audio-techies, making things bit perfect as long as possible is a bit of a challenge if there are multiple links in the chain. Except for some boutique desktop DACs and high end AVRs or component DACs, the equipment may not even show the received and/or processing sample rate in its display. Almost none of them show that they are down-sampling if they are doing so.
Yes, asking people to get positive confirmation somehow that they are not down-sampling before analog conversion is a good thing. Not sure how many will be able to get positive confirmation in their equipment though.
Perhaps in the submission form, it might be useful to have as part of data collection, a question that asks if they played it at 96/24 with answers being yes/no/unsure. At least that will give you some additional means to analyze and/or categorize results.
Good suggestion. Thanks.
I, my wife and a friend spent several hours listening and recording our choices. After we finished and I uploaded my results, I checked the files to see how we did.
– All three of us chose the 44/16 version more often than the 96/24. Not identical scores, but we selected the 96/24 files only 5, 6 and 7 times out of 20 (respectively). Interesting if not statistically significant.
– I read the posts about being sure to listen to 96/24 and not a signal downsampled by the playback chain. Though I’m sure my PC sent 96/24 to my preamp, I have Dirac Live room correction running in the preamp and it operates at 48/24.
– I’ve had a thought since I once tried the Philips Golden Ear and failed to distinguish resolutions. If a source is low fidelity, lacking dynamic range and spectral breadth … there might be no difference there to be heard. Or, if a source is of exceptionally high fidelity with wide dynamics and spectral breadth … it might sound so good in both versions that the differences are too subtle to discern.
Admittedly the last comment seems contradictory, but it was a thought. Anyway … we enjoy listening to musuc!
I’ve successfully submitted. Where/how do I download the files?
I’m sending out the access credentials as I get submissions. Look in your email or spam folder.
I just signed up for the challenge. Hopefully it’s still ongoing.
I’ve signed up and am looking forward to the Challenge.
I have little doubt that my equiment will enable me to listen at the supplied sample rates.
If my PC is not capable of this, and I’m fairly sure it will be based on yesterdays listening session using Tidal and Roon, I’ll load them into my Antipodes server/streamer. I know that is capable.
Those who do score above 50% might claim better hearing. Wouldn’t there be a good case for repeating the test for them using reshuffled/renumbered tracks? You would then be able to apply statistics for repeatability. If they don’t identify the same tracks correctly, you have demonstrated another case of lucky guessing. If those tacks have audible “tells,” they should be consistently identifiable. There might be a certifiable golden ear out there, but I doubt enough of them to sustain the Hi Rez market. My club is still looking for the first one.
Good suggestion…in fact, I have another data set of the same tracks in another random order.
How do I submit the results for the tracks?
I use Jriver as my player and I have 3 different systems that I could audition these on.
I assume that I listen to track zxy A and then Track xyz B and say which one I prefer, or do I say its Hi Res or Red book?
Just need some guidance on how to participate.
I didn’t receive the results of my submission, is there a link to download them?
Thanks
Have you had a chance to complete your study?
Yes, I did complete it and prepared a presentation for the Audio Engineering Society (https://www.aes.org/e-lib/browse.cfm?elib=20954). The net result is that despite different experience levels, ages, and equipment costs, high-resolution audio is indistinguishable from CD fidelity.
When all is written and said, the ultimate determinator is the Human Ear and at what age it hears Digital Audio?
Most individuals, by the time they reach 40 – 50, cannot hear much beyond 13,000 – 16,000 cycles. Hence, the amount of digital quality rests, largely, within the Mid-range. A child of, perhaps, 10, might very well hear the various Digital Files and sources but for the average audiophile, the hearing loss will render them moot.