The Magic of Digitally De-Aging a Beloved Hero

ILM’s digital animator Matt Shumway tells us about his digital adventures on Indiana Jones and the Dial of Destiny.

By Trevor Hogg

Much been written about Indiana Jones and the Dial of Destiny since the movie premiered at the Cannes Film Festival two months ago. In the fifth installment of the beloved film franchise, Harrison Ford puts on the fedora and lashes the bullwhip for the last time as the adventurous archaeology professor tries to stay one step ahead of the Nazis. Visual effects powerhouse Industrial Light & Magic has been along for the entire death-defying ride beginning with the Raiders of the Lost Ark in 1981. While Henry ‘Indiana’ Jones, Jr. escapes danger haphazardly and with a mischievous grin, nothing was left to chance in the film’s dazzling visual effects work, thanks to the expertise of VFX supervisor Andrew Whitehurst (Ex Machina) and director James Mangold (Ford v Ferrari). The first 20 minutes of the film begins in a Scottish castle, transitions to a car chase and a battle onboard a speeding train, and ends with a bridge blowing up. The trick here is that the action-packed opening is supposed to appear as if it’s been assembled from lost footage shot during the making of Raiders of the Lost Ark

Actor’s Youth Recaptured

“Honestly, Harrison Ford still moves well for an 80-year old!” notes Matt Shumway, animation supervisor at ILM, whose credits include The Mandalorian, Life of Pi and The Revenant. “I hope we all can move like that when we’re 80. He still fit into his original costume. But Harrison is not 38 anymore, so you did have to reflect that in the motion. For the most part, you key off the takes that he has given, and we looked at his walks and runs. Harrison has such a distinctive run, which makes things easier because it’s not like a generic version of itself. It’s a mix of Jack Sparrow and Tom Cruise! He has a wonderful run from Raiders of the Lost Ark that we sourced when doing the younger stuff. It’s much wilder than you would expect.”

Each project that involves de-aging furthers the development of the process, from The Irishman to The Mandalorian to Indiana Jones and the Dial of Destiny. “It wasn’t a one-button approach and we’re good for 20 minutes of the movie,” says Shumway. “We used ILM FaceSwap, which is our internal technology that is similar to deepfake, but not quite. Hours and hours of footage that we had at Lucasfilm of Harrison at that age were sourced. Through machine learning and ILM FaceSwap, we were able to then source that footage and apply it to older Harrison Ford, which gets you to 80 percent quickly.”

He points out that to achieve the final 20 percent, which takes the visuals over the uncanny valley, is a lot of work. “It’s a mix of different techniques. We also had a fully CG-rendered version of Harrison at that age which would fill in the gaps that didn’t work so well with machine learning, or it would be 100 percent animated depending on the shot. We’re able to put all these pieces together so it feels like one performance (thanks to improvements in compositing and digital animation).”

The younger version of the actor did not require much modification. “It was definitely a concern because that was some of the feedback from The Irishman, which was they looked younger but still moved like older men,” remarks Shumway. “That’s a testament to James Mangold on what his choices were and how the shots were chosen to be de-aged; he was aware of that and didn’t want Harrison to move like

‘We used ILM FaceSwap which is our internal technology that’s similar to deep fake, but not quite. Through machine learning and ILM FaceSwap we were able to source that footage and apply it to older Harrison Ford.’

ILM animation supervisor Matthew Shumway

80-year-old Harrison — but like I said, Harrison still has a lot of pep in him. There was some digital double help for the more extreme actions and fighting, like on the back of the train. Other times he would be more hunched over, so we lifted him in the composite a little bit. But it was a lot more subtle approaches than we’ve had to do for de-aging in the past.”

No de-aging was required for Toby Jones (who plays fellow archaeologist and Indy ally Basil Shaw) reveals Shumway. “Toby Jones is playing his age for the most part in the movie and we don’t see him as an older man because he passes away before we see the older Indy.” However, this wasn’t the case for the Nazi antagonist played by Mads Mikkelsen. “We did do some de-aging on Mads as well,” says the ILM veteran. “He has a younger-looking face already, so his de-aging wasn’t as difficult as Harrison’s. Mads is a younger man than Harrison. James liked the de-aging we did because it didn’t look like a younger Mads, but a younger version of the character; that was cool and a different approach.”

Physics was creatively defied for the nerve-wracking rooftop train battle. “It’s definitely in favor of the story and what is most exciting as suppose to the real physics of what that would be,” says Shumway. “Indiana Jones certainly has a lot of experience being on top of trains, so we leaned into that. But you do want to make it feel like you’re on a train. Steven Spielberg is like that, too. It’s all about the excitement and heightened reality of it. If it was real to life on the train physics, you’d be freaking about falling off the whole time! I would certainly be! Even the speed of the train at moments to make it more exciting is going faster than those steam engines can do.”

ILM was responsible for the previz for the third act and subsequently executing the work which features airplanes and a cloud portal. “It was fun because we got to develop and help guide where a lot of that stuff was going and a number of the shot designs that we came up were in the film,” remarks Shumway. “That started with some kickoffs from James as well as storyboards. We would immediately do rounds and rounds of ideas and shot designs. We would get notes back and forth and worked with editorial. Then we moved into animation once they started shooting.”

Some full digital shots were created for the exteriors of the military aircrafts. “Andrew was a stickler for everything feeling real for planes, and we heightened the reality in a lot of cases because it’s an adventure film, but it needed to move and have weight,” he notes. “We treated those planes like any other character because each one had to feel correct. The Nord that Ethann Isidore is flying is lighter and getting bounced around a lot more in the storm, as opposed to the Highball, which is heavier and filled with Harrison and a bunch of Nazis.”

Producing the Portal

Shots featuring the cloud portal that goes back in time were entirely digital. “Some of the last work we delivered was the plane going through the portal because there’s always the mandate to do something that we’ve never seen before,” remarks Shumway. “What does that mean? It means you have to keep trying things and see what James does and doesn’t take to. There was a specific shape that the clouds were making which James wanted for breaking through and feeling natural and not like a magical portal. We looked at a lot of weird cloud formations from volcanoes so we could have something from reality that we were basing it on. Then we took a lot of single frame approaches with our art department, like should the clouds be spiraling out or spin? They gave James a menu of different things and he would narrow it down. We would start doing the effects elements for it and see if we could get that same feel. It was a long process trying to get that.”

Shumway says he’s always game for crazy challenges such as opening Dial of Destiny with a de-aged Harrison Ford. “We at ILM love that stuff because it pushes us, and it gets us excited about things. When people get excited, they do good work.” ◆

Disney’s Indiana Jones and the Dial of Destiny is currently playing in theaters all around the world.