PICTURE PERFECT PIXELS

By Ron Prince

Today we’re off to Pinewood Studios, where Sony is celebrating ten years since first establishing its Digital Motion Picture Centre (DMPC) on the world-famous lot with the launch of something that’s a little bit special and ever-so au courant – a spanking new Sony Crystal LED wall studio with complimentary and complementary virtual cinematography learning experiences.

Located two floors above Sunbelt Rentals (formerly Movietech) on Goldfinger Avenue, and with direct sight of the famous 007 Stage, the DMPC has offered a mecca for filmmakers to experience a one-stop resource for everything related to Sony’s state-of-the-art technologies and end-to-end production workflow solutions – offering training, seminars, masterclasses, special events and other services for students and industry professionals alike, as well as system integrators and Sony partners.

For cinematographers and their crews, this has meant being able to get hands-on with cameras like the FX series and Sony Venice family of CineAlta cameras. Now it means they can also get up-close and personal with a new toolset, which emerged on extravagant blockbusters with large budgets, like Gravity (2013) and The Mandalorian (2019), and now has the potential to become part of mainstream film and TV production – replacing traditional blue/green screen with in-camera effects and back-projection with realtime virtual or live-action environments.

As such, the LED wall at the DMPC symbolises Sony’s continuous efforts to contribute to the content creation industry by bringing the culture of technology and imaging to current and nextgeneration imagemakers.

I’m welcomed by Will Newman, the company’s affable new business development and DMPC manager, who has taken over from where the selfretired and highly-thought-of Richard Lewis has left off. He ushers me into the newly-installed studio space, that has been put together and colourcalibrated with the help of Lux Machina, a specialist in virtual production, in-camera effects, display technologies and creative screen control systems.

Before scrutiny of the set-up, Newman delivers a salient reminder that filmmaking techniques like in-camera effects and back projection are actually as old as the hills, using a neat PowerPoint and video presentation. This includes a clip of Charlie Chaplin performing a perilous roller-skating routine beside a precipitous drop in Modern Times (1936, dir. Charlie Chaplin, DPs Ira H. Morgan & Roland Totheroh), only to reveal clever matte painting was the basis for the visual trickery back then.

“In-camera effects are nothing new, but the way they can be achieved now using modern digital technology, like the Sony Crystal LED wall here, certainly is, and there’s no doubt that more and more filmmakers are eyeing the exciting opportunities it can provide,” says Newman. “However, it does require a good understanding of the process. You cannot, and should not, turn-up on a set with an LED wall without doing your homework first, expecting to achieve a perfect result. That’s what we’re here to help with.”

Sony currently makes cabinets/panels with 1.2mm or 1.5mm pixel-pitches (which indicates the space between each of the micro LED diodes). The lower the pixel pitch number, the closer together the LEDs are on the panel, and the higher resolution the background image is on-screen.

Newman says a rule of thumb to remember is that the optimal camera/viewing distance is typically two to three times the pixel pitch number in meters. For example, a narrow pixel pitch of 1.5, as is the case here at the DMPC, allows viewing from just three meters away, whereas a pixel pitch of ten would require a distance of at least 20 meters to experience optimal detail. Indeed, standing-up close you can see those tiny LEDs and the joins between the panels, but move further back and they instantly become invisible to the naked eye and the camera.

Newman enthuses that Sony’s microLEDs have a million-to-one contrast ratio for better rendition of light and shade, and that the panels themselves are made using a patented anti-reflective coating. This means that separate, on-set light sources won’t contaminate the look of the LED wall, often the bane of cinematographers and gaffers alike.

The wall itself measures 4.89m x 2.70m and is comprised of 64 Sony CLED Crystal B-series displays, called ‘cabinets’, arranged in 16:9 aspect ratio. Sony has chosen a flat wall, but could have just as easily laid them out in a shallow curve, as the cabinets can be off-set by a margin of 5-degrees. Each cabinet is made up from eight replaceable panels, which can be individually removed by suction cup, so that any light emitting diodes that go rogue or panels that decide to malfunction can be quickly swapped-out and replaced.

As you would expect, the latest flagship 8K Sony Venice 2 is being used in this facility. It is fitted with a Canon 45-135mm zoom that has in-built metadata support and feeds lens data into Unreal Engine where the image-processing happens. The camera is mounted on a smooth-gliding JanJib arm from Egripment, enabling the camera to be quickly repositioned around the studio for framing purposes. Additionally, the camera and studio ceiling are fitted with Ncam’s Reality infrared camera tracking system to deliver accurate XYZ positional data to Unreal Engine, so that it can work out the position the real camera in the virtual space and correlate any real-world camera moves and lens shifts with the background plate, which might be a still frame or moving footage.

“We’re using a high-quality zoom for demonstration and learning purposes in this setup, as it’s versatile for shooting subjects against LED walls,” says Newman, “but we’re actually lens agnostic. Along with Canon, we have partnerships with various lens manufacturers – such as Cooke and Zeiss – and rental houses, like Sunbelt downstairs, and are perfectly happy to order-in whatever spherical or Anamorphic optics cinematographers want to specifically test themselves.”

Moving over to the workstation where the magic happens, Newman explains that, to help improve pre-production planning and on-set workflows, Sony recently launched the Virtual Production Toolset. Essentially these are plug-ins that work with Sony Venice, Crystal LED and other HDR-enabled LED walls to enhance in-camera visual effects, and help to solve common virtual production challenges before they begin, such as irksome moiré patterning and colour matching issues.

Diving in a little deeper into these Newman says that the Camera & Display Plugin for Unreal Engine allows filmmakers to virtually-reproduce the settings of the Sony Venice, Venice 2 and the Crystal LED displays by streamlining colour characteristics and simulating the camera’s exposure index and neutral density filters to recreate shallow depth-of-field on the computer, thereby helping cinematographers identify lens choices during preproduction. This plug-in also displays moiré pattern detection/alerts, depending on the pixel pitch and other specifications of the LED wall. This helps cinematographers make changes to the camera position and movement during the pre-production process, that will result in time-savings and avoiding problems on-set.

“The Camera & Display Plugin means that you can practice, assess and check camera movements in pre-production, before you ever set foot in front of the LED wall,” says Newman. “This also has benefits for filmschools that are teaching virtual production, as they don’t have to run to the expense of running a wall for students as they learn some of the do’s and don’ts.”

To ensure real-world colour accuracy between the LED wall, the camera and on-set displays, Sony’s Colour Calibration plug-in enables Unreal Engine to reduce the colour calibration process from several hours to just a matter of minutes.

Back to the real world, Sony has installed a bevvy of lighting instruments from ETC, such as Fos/4 panel lights, and SumoLight’s Sumosky, in the new studio to help cinematographers and gaffers assess how they can best light people or objects in front of the LED wall. Fixtures like these enable image-based lighting techniques, where the background images are played through the lighting fixtures themselves, to cast appropriate ambient light wherever that is deemed appropriate – from beside the wall or above the set, for example.

“We have assembled all of the tools – LEDs displays, camera, lenses, tracking, lighting and virtual technology – you are likely going to encounter when you shoot using and LED wall. And we’re open to anyone – scriptwriters, directors, actors, cinematographers, ACs, gaffers, students – who wants to learn more about this incredible new technology,” Newman concludes. “We’ll be announcing a range of learning courses in the near future, but we’re also open to individual approaches now, and we can fire up the wall in a matter of minutes.”

All-in-all it’s an impressive set-up, led by experienced people who care. If you want to know how to achieve picture perfect pixels in the emerging world of LED wall technology, you know who to call.

PS: Last year, Sony joined forces with Plateau Virtuel to create a virtual studio in Seine Saint Denis, Paris, that offers a curved screen measuring 18m x 5m, using no fewer than 450 Sony Crystal LED 1.5 pixel-pitch B-series cabinets. This wall is suspended so that alternative flooring, such as LEDs, can be introduced into the mix, and is available for training as well as commercial projects.