OBS CTO Sotiris Salamouris on the Move to HHD, HDR, IP, and Immersive Audio

OBS CTO Sotiris Salamouris on the Move to UHD, HDR, IP, and Immersive Audio

When the 2018 PyeongChang Winter Games ended, OBS CTO Sotiris Salamouris and his team laid out an ambitious vision for the 2020 Tokyo Games. Not only did they want to transition from SDI to IP, but they also wanted to go all UHD and HDR.

It was an ambitious plan, but the team pulled it off, despite a pandemic-mandated delay that may have allowed more time for testing and development but also meant less time to get ready for the 2022 Beijing Winter Games. Even with multiple years to plan, there was little in the way of wiggle room because there was always another Olympics to plan for, new technology to embrace, and new projects to launch to better meet the needs of rightsholders.

When the Games were delayed by a year, noted Salamouris, it gave the team a bit more time to do testing with broadcasters. But the team tried hard not to change the scope of what they were looking to accomplish as it dealt with other issues, including logistical challenges and, of course, the pandemic.

“When the postponement happened,” said Salamouris, during the Games, “we already had about 150 people here in Tokyo. It was a project in and of itself just to figure out how to freeze things and then get people back home. We needed to deal with the situation here, coordinate with Beijing, and then also deal with new requirements from broadcasters, who were all concerned with how they would work remotely and their space inside the IBC.”

“It was difficult,” he continued, “because, in order to be ready for the event, we have a very well-coordinated plan. And we had two areas of difficulty. One was international logistics, which were also affected by the pandemic with frequent flight changes and cancellations. In addition, as though one bad option was not already enough, sea transports were experiencing challenges due to the overall backlog created by the Suez Canal blockage some time ago.”

And then there were pandemic issues, he added. Because OBS brings thousands of international staff in to work the Games, the Japanese authorities and the Organizing Committee established a thorough regime to safeguard the health and safety of the international personnel and, of course, the local population. This famous “playbook” mandated rigorous testing and other measures to minimize infection in the Games environment.

Salamouris noted that there were some disruptions in OBS operations. Contact tracing on the plane to Tokyo, for example, required some personnel to quarantine, making them unavailable to work. “We had Plan Bs to address such eventualities,” he said. “It is totally impossible to fully predict what kind of impact you may have if someone from your personnel, who have very varying and sometimes unique skillsets, may need to quarantine.

“We have the playbook,” he continued, “and people were tested and tested. The reality was that the percentage of positives found was extremely low. However, there were cases where people may have been in the vicinity of suspicious cases or even positives and then had to quarantine. Suddenly, you have people that are essential to your team disappearing, sometimes for two weeks.” Despite the travails, dozens of rightsholders, thousands of production professionals, and thousands of athletes and volunteers were onsite for Tokyo Games. And, although technical innovation may have taken a backseat to the ongoing concerns around COVID-19, it’s important to look at some of the innovations that made these Games arguably the single most impressive technical achievement in the history of sports production. As rightsholders reshaped their plans for the Tokyo Games, the OBS efforts around cloud-based services started to become more important. “They became a

priority,” said Salamouris, “as they would allow rightsholders to do more remotely and operate from wherever they are located.”

OBS cloud-based services were built on the Alibaba cloud platform, and Salamouris said it would not have been easy to do without their support. “You need specialized support to do this, and we are able to get Alibaba’s attention.”

The more popular cloud services included Content+, Content+ Extra, and Live Cloud, although there were several others that OBS had developed over Alibaba’s public cloud for either its internal consumption or delivery to the RHBs.

“We’ll produce more than 9,000 hours of content, 5,500 hours of which is live competitions, ceremonies, and other scenes and content from the venues,” said Salamouris. “The rest is postproduced. The point is, how do we make that accessible to broadcasters?”

The two main ways for rightsholders to access all that content was Content+ and Content+ Extra, which were basically the same service but with different access rights. Content+ gave access to all the postproduced content, such as features, interviews, and highlights. A file-based system, it gave the user the ability to download an entire clip or part of it or even do some editing in the cloud.

“Content+ Extra is the same things but with access to growing files for competitions as they are happening,” explained Salamouris. “You can browse it and clip whatever you want and download it while the session is still on. You can’t feed it to distribution, but you can build your own highlights once you select what high-resolution file [is] sent to you.”

For rightsholders looking to do cloud-based distribution, there was Live Cloud, which made all the video and audio signals available via IP packets streamed over the public internet. The whole process was controlled by cloud-based applications built and made available by OBS.

“They can select whatever they want from our available 75 HD and 46 UHD distribution channels,” Salamouris explained, “and they can get it wherever they are in the world over the public internet.”

The signals were available at 100 Mbps per UHD feed, exceeding even the compression specs for UHD, which are common via satellite.

“It goes from one part of the world to another with no packet loss, no breaks, and with latency that is similar to satellite transmission,” he pointed out. “That is great news, because one of the big things about UHD is the cost of getting the signals back home. This is a very costeffective way of getting as much UHD as you want.”

The move to IP has been intense, interesting, but ultimately very successful, Salamouris said, because OBS was able to combine the move to IP with the move to UHD. Native coverage is UHD with HDR and wide color gamut (WCG), and an HD SDR version was derived from that UHD HDR production.

“We wanted to move to UHD,” he noted, “and we knew that we could not scale with a standard quad SDI workflow for the volume of content we wanted. Since we are using a substantially large fleet of existing OB units but also fly-away systems (31 OB vans and 22 fly-away systems, quite often with multi-feed outputs), it was impractical and unnecessary to impose the exact type of internal technology that these systems could use. Many of those had migrated to IP, but the majority were still based on quad SDI, ‘legacy broadcast’ technologies for their internal signal routing.

“We had no issue with that,” he continued, “as long as they were engineered to support our UHD workflow, including, of course, our expectations for capacity and resilience. Each production unit, however, had to deliver a double UHD HDR version and a double HD SDR signal in parallel paths. From this demarcation onwards, HD and UHD followed independent paths and were based on totally independent technology stacks. We used our legacy contribution and distribution systems for HD, but, for UHD, we moved fully to IP.”

The UHD contribution — what it takes to move UHD content from the venues to the IBC — was a combination of technologies based on SMPTE ST 2022, which Salamouris said had several advantages over ST 2110 at this stage.

The OBS contribution, distribution, and unilateral signals area was the mission control for a wide variety of content feeds.

TOKYO OLYMPIC GAMES: OBS OBS Head of Engineering Isidoro Moreno on Move to UHD, HDR, IP

As Head of Engineering for OBS, Isidoro Moreno knows his tech. And at the 2020 Tokyo Games he drew on all his skills as well as those of his team. Why? Because not only were the Tokyo Olympics the first ever to be all IP, but the Games were also all UHD, all HDR, and all immersive audio with 5.1.4 channels. Moreno discussed the new developments and more with SVG. Transitioning to UHD, HDR and IP all at once is a pretty big lift. How has it been going and what have you learned? We are learning a lot and it’s an opportunity for us to streamline our workflows. Here in Tokyo, the number of new projects we have started is huge and the number of services we offer to rightsholders has increased by about 50% since the Rio Olympics in 2016. And those new services can be only achieved by applying new technologies.

For instance, in the IP world we have increased the capacity between the venues and the IBC. In London there was just one video feed per fiber and things were extremely big because you needed a lot of lines. But now with IP, we can create trunks and aggregate services on a single fiber. And we also have redundancies we didn’t have in the past as we can have two services and have a guarantee against failure.

Also, the transition from HD to UHD is gradual [for rights holders]. We have to separate the UHD signal from the HD signal in order to stay consistent with the service level we had in the past. We cannot force everyone to change to a new format, so IP is useful in offering both HD and UHD services. I was speaking with Dave Mazza at NBC, and he said it is amazing that you were able to find enough UHD trucks and facilities to go all UHD. How did you do that? It was one of the bigger challenges and how to find enough trucks to do UHD was one of the first studies that we did. We wanted to figure out if we could do 100%, 50%, or whatever and at the beginning it did not seem possible to cover 100%. But we had contracts with companies that were about to build trucks and we were counting on them being built for UHD.

At the Opening Ceremony we used a UHD flyaway system that was designed specifically to do the opening ceremonies and it was based on ST-2110 IP. There is an opportunity to develop new systems that can be used for future games. Did the one-year delay give you a chance to change anything? We went to the market looking to see what could help us fulfill our needs. In UHD we saw a mature market that offered us what we were looking for. We had to offer services like splits in the cameras, replays, super slo mos, and RF systems so we were investigating how to upgrade those to UHD.

So, out of the 1,049 cameras we are using, very few are natively 1080p. And one thing we have learned [in tests] is that using native 1080i equipment and upconverting to UHD did not create content at the level we wanted. But when you start with 1080p and then go up to UHD and then down to 1080i there is a much better result.

One important decision we made, which was risky, was that in the past experiments were done in parallel. For example, in 1992 in Barcelona the Olympics HD production was a completely parallel production as we didn’t want to interact or interfere with the main production. But here we are doing a single production with maximum UHD quality. All the elements are fully native UHD or [upconvert] at a good level. We wanted to have full athletic coverage with the same number of cameras we had in the past and that has been super helpful to broadcasters.

How about working in HDR? Once you understand HDR, it’s a big improvement. When we were talking about UHD SDR, it was more pixels, bigger resolution, but we didn’t explore the whole world of colorimetry and wide color gamut.

We studied how to make HDR compatible with up and down conversion as our main product is 1080i SDR. So, we have three transforms that keep the color spectrum of UHD as much as possible within the 709 color space used for SDR.

With things like country flags in graphics the colorimetry is important. We want to maximize the quality and user experience in HDR without compromising 1080i with things like shifting colors that are not correct. So, we created our own set of Look Up Tables (LUTs) that broadcasters can use free of charge. We want to make our pictures compatible with the personalized pictures they create with their own cameras. It’s been a complex process and we are collaborating with broadcasters to better understand their needs. At the end of the day that is our mission: not what OBS wants but to help the broadcasters. The Beijing Games are only six months away. I am assuming you won’t look to do any major changes. We want to keep things the same in Beijing as much as possible but obviously we will refine something if we need to. But it’s a special situation as usually we have a year and a half and now, we have only months. We do have some things in our road map, but we can’t apply them in Beijing as it would be too risky. – KK This interview has been condensed and edited. To read the full interview, visit the SVG SportsTechLive Blog.

OBS Director of Engineering Isidoro Moreno

continued from page 47

“Within the IBC,” he noted, “all the signal routing and distribution is based on an SDN infrastructure carrying ST 2110, as is all the signal monitoring. We also have PTP timing that has worked very well, and we’re surprised at how robust the whole thing has been when it comes to networking. We’re very, very happy to establish ST 2110 and PTP as our basic technology.”

The production efforts for the Tokyo Olympics were massive, deploying more than 1,050 cameras. About 70% of those cameras were broadcast, CCU/BPU-supported native UHD; the remainder were SDI-output cameras, mostly in native UHD but a few also in 1080p (there are no 1080i camera sources).

“An important innovation from our side,” he said, “was the combined/ common live workflow for UHD HDR and HD SDR. We very soon realized that we had only one option for introducing UHD in the Olympics: to build a unified workflow that will be delivering both UHD and HD from the same higher-quality format, which of course could only be UHD in BT.2020, with HLG HDR and WCG. Of course, such a single workflow will always need to guarantee a premium quality in HD, since this is the format that the great majority of the world broadcasters still use. UHD had also to be visually implacable; otherwise, its introduction would not make sense at all.

“To achieve all these quite aggressive and challenging goals,” he continued, “we had to develop a unique workflow that had to deviate substantially from what had been so for the more common approaches of producing UHD with HD for live sports. We ended up developing three types of our own HDR look-up tables. We realized that, because our own needs are very specific we had to create our own conversion tables.”

Each of the look-up tables (LUTs) had a specific purpose. One took existing SDR sources, such as archival material or specialty cameras, and placed them into the HDR domain. A second table was specifically designed for graphics needs. And then a final table helped convert UHD to HD.

“We incorporated those LUTs in converters in the trucks, or the trucks themselves have their own ability to program the LUTs into equipment like the vision mixer,” said Salamouris. “But it was a very long process with very extensive testing and a rigorous certification phase every time that we were deciding to use existing truck resources.”

The result was a single workflow for all 50+ production units.

“They all have exactly the same workflows regardless of which sport they are doing or what venue,” he explained. “That has proved to be nice, as we have had zero issues with our UHD HDR output and our HD SDR output. It’s the same picture but enhanced in resolution due to 4K, brightness highlights wherever they exist (this is due to HDR), and color fullness wherever it exists (and this is due to WCG), which is what it should be.”

Each truck also had a visual expert who worked with the shading team and was also in contact with a centralized VQC in the IBC, where experts made sure that the results across all the sports are dialed in similarly.

Along with improvements on the video side was the move to 5.1.4 discrete immersive sound, which added four channels above the listener to provide a sense of height. According to Salamouris, 5.1 surround sound, despite being available for some time now, has not caught on with viewers because it requires placing dedicated speakers around a room. He believes soundbars could change the equation, especially with 5.1.4.

“The technology in soundbars has developed so much and they are so sophisticated that they are close in quality to a dedicated surround-sound system,” he explained. “It makes a difference with the sound space on top of you, especially in sports, where you want to feel like you are there.”

For more than a decade, OBS has been working hard to figure out how to make it easier for rightsholders to have a smaller onsite presence. The pandemic caused that to happen in terms of personnel, but the physical space was also smaller, around 10,000 sq. meters less than Rio in 2016.

“It has been engineered to make that happen,” Salamouris explained. “One important way is the consolidation of all the technical spaces in our CTA, or Centralized Technical Areas. It has helped a lot with the overall efficiency of space needed but is even more important in providing efficiency in the cooling and the power system.” The CTAs and all the efforts by OBS to create cloud and other services are also designed to shorten the setup time for an IBC that is creating more content than ever and in more-complex ways. Over the past 20 years, the time to set up the IBC has remained static. Salamouris and OBS hope to change that. “We are always given just some weeks or some days to set up and do our job,” he says. “That is why the cloud is important to us: you can set up a workflow at any time and test it for months before the [event]. You can commission it, switch it down, and then switch it up before the Games. That is a concept that works for us.” – Ken Kerschbaumer

The OBS technical area was home to hundreds of freelancers and staffers who worked on a wide variety of OBS Olympic initiatives.