Encoding and Decoding Ciro A. Noronha, Ph.D. Cobalt Digital
“The more time you have to do a job, the better job you can do…” This is something that not only applies to day-to-day life, but also in many technology fields, such as video Encoding and Decoding. It is not the case that the smart people working in video compression invented time dilation – the smart semiconductor people are continuously coming up with devices that can do more in the same amount of time. The evolution of the video compression standards, from MPEG-1 to MPEG-2, then H.264 (AVC) and H.265 (HEVC) has followed this technology curve. Except from the jump from MPEG-1 to MPEG-2, there have not been major algorithm changes – what has happened are evolutionary improvements. We more CPU cycles to work with – we use them to do a better encoding job. Fundamentally, video encoding works by exploiting the redundancies in the content – instead of sending the same thing again, we just say “repeat this blob” or “repeat this blob with these minor changes”. The key is finding a good selection for “this blob” in the current image, and where it went in the next image. If you have enough CPU cycles (“more time to do the job”), you can look further out, and you can look at multiple frames. Also, if you are doing non real-time encoding (VOD assets, for example), you can take as long as you want to do the best possible job. HEVC is simply the current chapter in this saga. Information theory tells us that at some point, there will be a limit on how far you can go. So far, we have been able to keep the pace of reducing the required bit rate by a factor of two between major compression standards for the same quality (at least for images whose resolution is large enough). However, we may be getting close to the limit – the new compression algorithms coming after HEVC are targeting a more modest improvement, on the order to 20% or so. What makes this whole thing so exciting is the combination of two technology advancements: improvements in compression bring down the required bit rate to produce good quality video, while improvements in infrastructure increase the available bit to consumers. These two trends have come together in the last 10 years to bring first SD, then HD, and now 4K/HDR to people’s living rooms (and to the cell phones in their pockets). Moreover, since now everything you have with a screen on has a powerful CPU inside, decoding can be done in general-purpose software, which allows it to follow improvements in encoding technology. Ciro A. Noronha, Ph.D. is Executive Vice President of Engineering for Cobalt Digital, Inc.