Computer Audiophile “The whole truth about”
WAV vs FLAC vs
MP3 vs M4A
Experiment by Mitchco
FLAC vs WAV vs MP3 vs M4A Experiment http://www.computeraudiophile.com/blogs/mitchco/flac-vs-wav-vs-mp3-vs-m4a-experiment-94/
Mitchco
12-02-2011 at 02:07 PM I wanted to try an experiment of measuring any differences between various media file formats as described in the title. Consider this, if you are hearing a difference when you change media file formats (e.g. from FLAC to WAV), then the audio waveform must have changed, and if it is has changed, then that change can be measured. While the waveform pictures in this article are technical, it really is a case of which picture does not belong with the others.
If you are comparing media file formats, the theory is that FLAC and WAV are lossless file formats and therefore should be identical. Meaning the waveforms are identical. My plan is to use Audacity and a well known procedure to measure the waveforms and identify any differences. But first, how can we check our gear to see if everything is working as it should? I happen to have a function (i.e. waveform) generator and dual channel oscilloscope for viewing analog waveforms on an old school CRT.
Notice on the oscilliscope, the top trace is a sine wave. The top trace is monitoring a 20 Khz sine wave coming out of the fucntion generator and going to the analog input of my Lynx L22 sound card in my PC. Lynx card flat from 15Hz to +50Khz with no phase shift. Then it is converted into digital format and routed through the mixing console and then converted from digital to analog and monitoring on the 2nd scope channel (i.e. bottom trace).
Ok let's line it up the two waveforms a bit and see how close the waveforms are after going through analog input amp --> ADC --> DAC-->Analog output stage -> scope.
I blame any discrepancies on the +20 year old scope and the much older carbon unit operating it. Would be great to have a digital storage scope. Ok now lets record 60 seconds of that 20Khz sine wave in Audacity at 24/96 and save it as FLAC.
This is what it looks like in digital format. Mind you I have zoomed waaaayyyy in, look at the time line. Amazing that it can be reconstructed back into a perfect sine wave again. A technological marvel.
I played back the 60 second 20Khz sine wave and lined the waves on the scope again and they matched. Same procedure for a WAV conversion of the FLAC and MP3 version. I did not want to make this repetitive, so I did not add the pics, they all look the same. My purpose was to see if everything is running as it should be. To really see if the waveforms have been altered we are going to use Audacity and a procedure to take 2 waveforms, normalize their amplitude, massively zoom in and align the two waveforms, invert one, and mix them together. If they are identical, it will "null" out the two waveforms and there will be no signal left. Here is the step by step procedure: 1) Import copies of both the mp3 and Wav files into the same Audacity project. 2) Amplify both to the peak volume, use the amplify effect's default value on each signal separately. 3) Zoom way in on a distinct part of the waveform. Zooming on something percussive will make this part much easier, it's easier to see. 4) Use the Time-Shift Tool to line up the waveforms exactly. Keep zooming in and adjusting until you're lined up as accurately as possible, make sure you can see each sample by the time you're done. 5) Now invert one of the signals, either will work. 6) Highlight both signals and select Tracks -> Mix and Render or Project -> Quick Mix. 7) The signal you have left will be the difference between the two original files. Everything here is what you lost when you went to the mp3 format, it's mostly high frequencies and quick changes in dynamics (such as percussion). I chose Tom Petty's song Refugee in 24/96 FLAC format downloaded from the Tom Petty site as my "master". The reason for my choice is the attached Producer/Engineer note that comes with the download that states: "FLAC is a “lossless” format, which sounds the same as the source files it was created from. We made the FLAC files from the same high-resolution uncompressed 24-bit 96K master stereo files we used for the vinyl and Blu ray versions of Damn The Torpedoes: Deluxe Edition. When we compared those files to the FLAC’s, the waveforms tested out to be virtually identical. With the right system, you’ll be as close to being there as we can get you." So I am getting a copy that that the waveforms are "virtually identical" to the original analog 2 track master. Great I like that. I wish all remasters were like that. Give me an uncompressed and virtually identical waveform copy of the best mix/master that
you can get your hands on. As we will see, one of the files is badly compressed and is the picture that does not belong with the others. I used JRiver MC16 to convert the FLAC to WAV and MP3 (LAME 320Kbs). I also have the M4A iTunes version that I will compare.
The top waveform is FLAC, with the left channel on top and the right channel on bottom. And the bottom two waveforms are the left and right WAV channels. These are the original files unaltered.
Here I have zoomed way in on the waveforms to see the individual samples. Now I selected one of the waveforms and inverted the signal, and then applied the Mix and Render to produce this:
As you can see there is nothing there. It is totally nulled out. Let’s see if there is a frequency spectrum:
Ok, what about a frequency analysis:
Nothing. As expected. Let's move to our comparison of FLAC versus MP3. Here I have imported, amplified, zoomed, and lined the two waveforms up.
Ok let's, invert one of the waveforms and mix the 2 together:
Aha! Look, there is signal. As expected as MP3 is a lossy format. Lets look at the spectrum:
And the frequency analysis:
As you can see, mostly high frequency content. I have attached a 30 second snippet of the file so that you can download and hear the difference with your own ears. Well, so far the experiment is going as anticipated. So what about M4A iTunes format? Well, lets look and see how far we get, given the the iTunes version is 2 seconds longer than the original.
Again, FLAC on top and M4A on the bottom. WOW! Look at the level of compression! And there is a time difference as well. The FLAC is 3:20 and the M4A is 3:22. So if the FLAC is from the original analog master, what is this M4A version? It looks like it is also on the CD as it says 3:22. But since I don't have a copy, I can't validate that claim. The other variable is that the M4A was downloaded from iTunes and I have no way of knowing if it has been processed in some way. What could explain the 2 second difference? Total speculation, but I wonder if it was operator error or if the mastering lab had issues with their gear (out of calibration) or maybe the 2 different analog 2 track tape machines were not speed calibrated. Regardless, the iTunes version is horribly compressed. Time for an aside... Why I want unaltered waveforms. As someone who used to mix bands sound live and on tape, there is a mixing technique called, riding the fader:
Starts off with an up volume snare roll, nice intro guitar solo, then drops down in volume for the 1st verse, crank volume back up during the chorus, then back down for 2nd verse, then up for the 2nd chorus, then up for the bridge and then peak the Hammond B3 and guitar solo's a bit for a taste of what's to come, then back down for 3rd verse and then pull out all the stops for the final chorus and the rock out at the end, riding the faders to max with the peak on the piercing high note on the guitar solo and then a quick ramp down fade out. But see, that's the sound mixer doing his/her job in getting every last bit of emotional content of the bands performance, whether live or recorded. Have another look at the FLAC and M4A waveforms. It takes no special skill to see that the two waveforms do not match or even really resemble each other. But it is the same song! Somehow on the M4A, the low amplitude has been expanded, yet the peaks are lower... Welcome to the world of (badly done) compression. I would speculate that when TP finished the recording and mixed to death the final two track analog master, over and over again, until it was the way they wanted it. I would think that they thought this is how it was going to get mastered on CD. I.e. an unaltered waveform copy. Not able yet to confirm, but if it is the same copy on the CD as the iTunes version, that's a world of hurt. I see on Wikipedia that there is at least 3 masters available, so who the heck knows what the iTunes one is other than it is atrocious. I would further speculate that TP has had to live with a rubbish master job since 1979 or apparently it was remastered in the early 80's and again in 2001. Whatever, it was not till 2010 that TP finally got a copy of the original 2 track analog master tape at 24/96 and as it states the waveforms from the master and the 24/96 are virtually identical. I would even go out on a limb to say that the whole reason that Damn the Torpedoes was remastered was because of the incredibly compressed mastering(s). All speculation of course on my behalf. The point is that the waveform on the iTunes version has been altered so much as to reduce the enjoyment factor to the point where I can't listen to it. Can you imagine what the band must have felt to have heard the iTunes version. Ruined. Again, total speculation. You do not hear the fader riding in the iTunes compressed version. So there is no build. The verse and chorus sounds the same level, as does the bridge, solos, etc. Sounds flattened and knocked some good emotional performance out of the song. To add insult to injury, the waveform is so altered that it has also altered the soundstage. All of the psychoacoustic cues that the mixing engineer has put in place have been destroyed. In the iTunes version, TP's voice sounds flat and two dimensional. In the 24/96 unaltered copy, you can really hear his voice sitting back in the mix, very 3 dimensional sounding as are the rest of the depth cues in the remaster. It is somewhat ironic that this is the reverse situation that plagues current remasters today. In other words, TP got burnt on his master, but his remaster is a virtually an identical copy of the original analog master. Whereas, we are now getting remasters
that have their waveforms altered from the original. Can you imagine a fan playing the heck out of their favorite bands records, then getting the CD versions and now has an opportunity, 20 years later or whatever, to get a special, hi-res remaster only to find that it sounds nothing like what the fan has listened to for the last 20 years. Talk about disappointment and giving hi-res a bad name. My example is the new CCR release. I am hesitant to buy it for exactly that reason because there is no information that I can see that states what master is being used for the remaster and what was the process used to remaster this version. To me as an audiophile, I do not think that it is too much to ask for an unaltered waveform copy of the best master tape available for any given artist/song. Tom Petty did not think it was too much to ask. Just like the record companies did not get on-line music distribution, they are not getting that, with the advent of hi-res Blu ray video, people will pay a premium for hi resolution audio. Assuming it is the real thing and unaltered. Was it the time or era that caused bad masters? I don't think so. Consider this, The Police's Synchronicity album was released in 1983. That's 28 years ago. The song, "Murder by Numbers" has a whopping DR of 18. Stewart Copland's drums sound incredible and in the last minute of the tune, Hugh Padgham, one of my favorite rock producer/engineers, pushes the faders up on Stewart's drum kit so high it's like he is playing right in your room.
At concert volume, the kick drum punches your stomach and the snare has such a crack, your eyes involuntary blink every time Stewart hits it - awesome! Even more so for a 16/44.1 CD mastered a long time ago. Man, I would pay a pretty penny to get a unaltered waveform copy of the original master from Les Studio. Back to work. Let’s compare FLAC vs M4A. When I amplified the waveforms, FLAC takes 2.1db to give a peak of 0db, whereas the M4A versions only requires a .2db to reach peak. That's a factor of 10. Here it is zoomed in:
So when I inverted and mixed the 2 together, I got the following result:
Well, first of all, the red is showing clipping. Note in Audacity Beta 1.3.13, under the view menu is a menu item called Show Clipping. As a side note for the folks doing the music analysis thread http://www.computeraudiophile.com/Fo...ive-Subjective , always a good idea to have this Show Clipping checked on so that you can see if any hi-res download is clipped. I might follow up later and see if I can pitch change the M4A to match the FLAC and re-run the test, but the bottom line is that the iTunes version is massively compressed and waveform altered. Attached is a 30 second snippet of this. Clearly you can hear the echo as one track is 1% out of sync with the other.
I also have the Blu ray version of Damn the Torpedoes. I was going to compare that too, but following Chris's Blu ray ripping video, http://www.computeraudiophile.com/co...ersion-Box-Set I could not get the software to extract the streams.
Conclusion: The big standout to me is how much he waveform has been altered on the iTunes M4A version. And how bad it sounds. I would pay a premium price to get the best "historical" remaster of any music without altering the waveform, which means no compression or noise reduction or eq filtering of any kind. I want the virtually identical copy. I want to hear the music the way it was intended to be heard right from the mixing chair to my listening room. For those that are hearing a difference between lossless file formats of any kind, it is not the file format. I humbly submit that the differences you are hearing are else where. Who knows where as it could be anywhere in or around the signal path. That's the issue. It is not that the files are different, it is how your system is interpreting those files that is different. The problem is diagnosing where the differences are. Or living with the difference and pick your format that sounds best to you and move on. What is most important to me is the erosion of original performances where the original master is gone and we don't have an unaltered waveform of a "new" master. All those analog tapes are eventually going to fall apart. I hope we have unaltered waveform copies. I understand that famous paintings undergo restorations. But even then, the restorers are trying to restore the painting to its original condition. That does not seem to be happening in the music remastering world. Those works of art are being altered, some so bad that it does not even resemble what the artist had in mind. The bands dynamics, nuances, soundstage, tone, have all been altered to the point of ruin. Then again, if the restoration is an enhancement of the original because of modern technology (used professionally), whose to say... Little dip in eq there little bit noise reduction here, wee bit of expansion there - the ol nip and tuck. And if done really well, it should sound like an enhanced version of the original, but restoring more dynamic range, tape hiss reduction, etc. The idea would be to restore the tape as best as the technology allows without massive compression. Instead, if massively compressed, try dialling in the reverse settings in an expander to try and restore the original waveform (a convolver can do it). It takes a trained ear in compression to be able to apply expansion. Maybe we can get some of the original performance back. And unless you were in the mastering room and A/B ing the changes of the ol nip and tuck, you may never know what processing occurred. Unless there are other remasters, you could compare those. But If it sounds good, does it matter? This was an experiment. Others have had similar results. Bruce Brown from Puget Sound Studios, "If I put a wav file on one track and a FLAC file on the other track in Pyramix, I can't tell them apart, sighted or blind". And the Well Tempered Computer. If you are hearing a difference when you change X, then the waveform must be altered in order to produce that difference. Therefore it can be measured. Don't take my word for it, try it yourself.
FLAC vs WAV Part 2 Final Results 02-18-2012 at 04:59 PM http://www.computeraudiophile.com/blogs/mitchco/flac-vs-wav-part-2-final-results-155/ In part 1, I used a null test technique to show that both FLAC and WAV (lossless) file formats are identical. In this post, I have expanded the null test to cover off playing the same FLAC and WAV files dynamically from JRiver and capturing the audio waveform after the Digital to Analog conversion and analog line output stage. Here is a high level block diagram of my test setup:
For playback, I am using the exact same original FLAC and converted (by JRiver) WAV file I used in Part 1. It is Tom Petty and Heartbreakers Refugee at 24/96. JRiver is set up for bit perfect playback with no DSP, resampling, or anything else in the signal chain. I used the native Lynx ASIO driver to communicate between the sound card and JRiver. All sample rates for the tests are at 24/96. My Win7 64 Bit HTPC build is nothing special. No special power supply or SSD or interconnects.
Side note, for Windows users, always invaluable to check your PC for latency with http://www.thesycon.de/deu/latency_check.shtml I have tested the frequency response of my Lynx L22 sound card using REW http://www.hometheatershack.com/roomeq/ and noise levels, distortion, etc., using RightMark Audio Analyzer http://audio.rightmark.org/index_new.shtml
For capturing (i.e. recording) the audio waveforms, I used a Dell M4600 latptop and the onboard HD audio chip and driver. Here is the noise measurement of the on board sound chip. Not as good as my Lynx card above, but a check to see that everything is in working order.
I used Audio Diffmaker http://www.libinst.com/Audio%20DiffMaker.htm for recording the waveforms that were coming off the analog outputs of my playback PC. Here is the process used by Audio DiffMaker:
As an aside, I should point out that you can use this software to objectively measure anything in your audio playback chain that you have changed. Whether that be power supply, DAC, interconnects, music players, SSD, VST plugins, or whatever. Remember, if you are audibly hearing a difference when you change something in your audio system (ABX testing), the audio waveform must have changed, and if it has changed, it can be objectively measured. I find there is a direct correlation between what I hear and what I measure. For me, to form any valid opinion about audio reproduction, I want to correlate my subjective results with my objective results and vice versa. I want a balanced view. In the Audio DiffMaker help file, the software program is able to line up the waveforms if the program material is within 1 second of each other (protip). Here I am capturing the first 40 seconds of TP’s Refugee in Audio DiffMaker:
I did this twice, once playing the FLAC and then the WAV, without making any changes on either computer. To test the DiffMaker software (and everything else) is working correctly, I took the FLAC recording and compared it to itself. Theoretically, it should null itself out completely.
And it does. Ok so now let’s compare the two recordings, one FLAC and the other WAV:
What the result is saying is that the difference signal is almost -90 dB. I repeated the test ten times and obtained the same
results. You can listen to the difference track for yourself as it is attached to this post. PLEASE BE CAREFUL as you will need to turn up the volume (likely to max) to hear anything. I suggest doing this in volume level stages so you can verify there are no other artificats while listening. As you can hear for yourself, a faint ghost track of the music, that nulls itself out completely halfway through the track and slowly drifts back into being barely audible at the end. According to the DiffMaker documentation, this is sample rate drift and there is a checkbox in the settings to compensate for this drift: “Any test in which the signal rate (such as clock speed for a digital source, or tape speed or turntable speed for an analog source) is not constant can result in a large and audible residual level in the Difference track. This is usually heard as a weak version of the Reference track that is present over only a portion of the Difference track, normally dropping into silence midway through the track, then becoming perceptible again toward the end. When severe, it can sound like a "flanging" effect in the high frequencies over the length of the track. For this reason, it is best to allow DiffMaker to compensate for sample rate drift. The default setting is to allow this compensation, with an accuracy level of "4".” Of course this makes sense given that I used a different computer to record on versus the playback computer and I did not have the two sample rate clocks synched together. The DiffMaker software recommends this approach, but I have no way of synching the sample rate clock on the Dell to my Lynx card. So when this is not possible, the DiffMaker documentation indicates to use the sample rate compensation. However, when I tried the sample rate compensation, the DiffMaker program thru the following error:
I sent an email to the software manufacture and will follow up once I hear back. Given that the signal is almost -90 dB from the reference and that the noise level of my Dell sound card is -86 dB, we are definitely nearing the limits of my gear. Also, given that the dynamic range of most music material we listen to is less than 20dB http://en.wikipedia.org/wiki/Dynamic_range#Audio it seems unlikely that I could hear the difference track, relative to the reference level – that’s a 90 dB difference.
Subjective Listening Tests In JRiver, I played the FLAC and WAV (and vice versa) several times through headphones and speakers. I did this sighted and blind. I also played back the recorded reference and compare files in Audio DiffMaker using headphones. Finally, I played back the Reference + Difference track. In my subjective listening tests, I could not hear any differences between the FLAC and WAV files in any combination of the above. Not only from the playback machine but also the recorded tracks. They all sounded identical to me. There seems to be good correlation between objective and subjective results. As a side note, I have been into audio and music for over 40 years. For 8 of those years I was a recording/mixing engineer where I was trained and relied upon to note very small audible changes. http://www.thepikes.com/bio The reason I am saying this is because of psychoacoustic http://en.wikipedia.org/wiki/Psychoacoustics effects, our ears can be easily fooled http://en.wikipedia.org/wiki/Auditory_illusion or put in a positive way, our ears adapt to changes very quickly. In fact, most recording, mixing, and mastering engineers use these psychoacoustic effects on purpose. For example, the HAAS effect http://en.wikipedia.org/wiki/Haas_ef...s_and_findings to make the sound more full, wider, sense of air, etc. All tricks played on our ears: http://www.algorithmix.com/en/kstereo.htm including some remastered material we download from HDTracks. So do we not trust our ears? I am not saying that. What I am doing is bringing a balance of both subjective and objective thoughts together so we can correlate what we hear with what we measure and vice versa. Again, when performing ABX listening tests, if you are hearing an audible difference, then the waveform must have changed. If the waveform has changed then we can measure the difference. Btw, all of the software used in these tests is free. I would encourage you to download the software’s and try this out for yourself as it does not require any special equipment. Further, you can objectively quantify any differences throughout the audio chain in your playback system. In conclusion, using my ears and measurement software, on my system, I cannot hear or (significantly) measure any difference between FLAC and WAV. Not only just file formats, but the rest of the audio playback chain as well. __________________________ In the given URLs you can find audio samples from Michco at the end of both articles __________________________ [In layman terms, according to Thomas Peeke from Funktion One, WAV and FLAC are one and the same]