VFX TRENDS: ROUNDTABLE

HOW INTERNAL ART DEPARTMENTS HELP TO SHAPE THE VFX STORY

By IAN FAILES

We regularly hear about the incredible artistry inside visual effects studios to pull off final shots. But a key step in the VFX component not always discussed is the intricate concept, design and visualization processes carried out within visual effects studios, which often helps shape the particular story being told. In this roundtable discussion, representatives from Framestore, ILM, Pixomondo, Scanline VFX, Technicolor Creative Studios, TRIXTER and Weta Digital break down their role in this storyshaping process, especially within their internal art departments.

THE PROCESS BEGINS

TOP: A Framestore concept frame for Blade Runner 2049. (Image courtesy of Framestore. Copyright © 2017 Warner Bros. Pictures) Daniel Matthews, Concept Artist, TRIXTER: “We will get our very first brief via our internal supervisor on a project, who distills a short, open ‘TL;DR’ version of what’s been discussed with the client. This minimal brief helps us to keep our creativity totally free-flowing, and allows us to cast the widest net possible for ideas. It gives us the freedom to mess up – which is a fantastic thing as an artist – and get bad ideas out of our system. It allows us to brainstorm and build on each other’s ideas as a team. At this stage, ‘happy little accidents’ can happen which often work their way into the final result.”

Mujia Liao, Head of Virtual Art Department/Art Director, Pixomondo Toronto: “Generally, we create mood boards, sketches, illustrations and key art frames that describe a particular moment in the show to help drive the direction for other departments to follow. Aside from creating designs from scratch, we collaborate closely with the asset department. We create paint-overs on renders for quick design changes, mocking up texture/color variations, as well as provide reference photography to help the asset team look dev the character/creature/ environment.”

James Clyne, Senior Visual Effects Art Director, ILM: “I don’t think there is a part of the production pipeline that ILM’s art department doesn’t touch. The department is involved in ‘blue sky’ conceptual designs, working directly with the director or with the production designer, sometimes just with the studio. Then we’re involved in everything from character design to vehicle design, prop design, environments and creatures. I think VFX has also allowed filmmakers to make some of the design decisions a little later because we can actually tackle some of those design decisions in post-production like we couldn’t before.”

Martin Macrae, Head of Art Department, Framestore: “As well as designing for projects internally, we’re also able to work independently on projects that don’t necessarily come through the company’s VFX team. Another part of what we do is to help directors put a pitch package together for studio presentations to help get a film greenlit. This will involve discussions with the director to try and establish their vision for the film, taking extracts from early drafts of the script, if there’s one ready, and creating concept art for film beats that evoke the right look and feel for key scenes in the story.”

Leandre Lagrange, Art Director, Technicolor Creative Studios: “We often get pulled in for greenlighting a project. At that point there is no VFX involved at all. Either we’re being reached out to by studios or we already have a VFX supervisor attached to the project who invites us to create images as a pitch to do the final shots. There are lots of different roles we play.”

Jelmer Boskma, Visual Effects Supervisor and Art Director, Scanline VFX: “There have been projects where we were involved early enough to help establish the look and influence the shoot in a way that complemented our ideas and designs wonderfully. Obviously, this is particularly beneficial to us as we can ensure the

TOP TO BOTTOM: Mujia Liao carried out matte painting work on a hollow-like planet in Star Trek Discovery. (Image courtesy of Pixomondo. Copyright © 2020 CBS Television Studios)

A ship and plan design illustration crafted by Scanline VFX for Midway. (Image courtesy of Scanline VFX. Copyright © 2019 Lionsgate Ltd.)

A scene from Prometheus in which the art and the final matte painting work melded into one. (Image courtesy of Technicolor Creative Studios. Copyright © 2012 20th Century Fox)

TOP TO BOTTOM: Rodan emerges from a volcano in a concept piece by Technicolor Creative Studios. (Image courtesy of Technicolor Creative Studios. Copyright © 2019 Warner Bros. Pictures)

TRIXTER crafted its artwork over Black Widow plate photography. (Image courtesy of TRIXTER. Copyright © 2021 Marvel)

The final reveal, in TRIXTER concept form, for this Black Widow scene. (Image courtesy of TRIXTER. Copyright © 2021 Marvel) lighting and general shot setups will translate well later in post. Just as common, though, is for our design services to be requested at a later stage during production. It’s not uncommon for a director to have a change of heart with regards to a certain design, or to find the production’s art department to have simply ran out of time to fully flesh out an idea.”

Sidney Kombo-Kintombo, Animation Supervisor, Weta Digital: “I’m an animation supervisor, but I do lots of previs as well now. It’s even sometimes previs at the end, in the post-production process. It can be previs, postvis and shots all melded into one thing. For example, Marvel always comes up a very strong concept of what they want – ‘Let’s have a dogfight in a canyon.’ They’ll have superstrong art development. But then the missing link is the connection between that strong concept design art and the thrill of the visual. It’s basically taking the strong concept that already exists and adding all the little elements that make that concept visually as interesting as it can be.”

TOOLS OF THE TRADE

James Clyne, ILM: “There are many applications that we use. I would say Photoshop is probably the most dominant, but there’s a lot of 3D applications from Maya to Modo that people are using every day. The strength lies in not just being really good at one thing, but being pretty proficient at not only 2D drawing, but 3D model modelling as well, and being able to do rapid sketching, all the way up to doing photorealistic environments.”

Martin Macrae, Framestore: “It’s a very organic process, every project is done in its own unique way with no set ways to get things

TOP TO BOTTOM: ILM’s James Clyne designed this bomber for Star Wars: The Last Jedi. (Image courtesy of ILM. Copyright © 2017 Lucasfilm Ltd.)

For Spider-Man: Far From Home, Scanline helped establish the smokey look of Mysterio’s suit. (Image courtesy of Scanline VFX. Copyright © 2019 Sony Pictures)

The futuristic, and heavily polluted, Las Vegas in Blade Runner 2049 as imagined by Framestore. (Image courtesy of Framestore. Copyright © 2017 Warner Bros. Pictures) done. Artists can use whatever means they need to get an idea out, be it a pencil sketch or a 3D sketch – it doesn’t really matter what tools they use. Concept artists need to be flexible in the way they work as the creative idea is the most important thing to get out, so they have access to any software they need to use. When it comes to software the trend now seems to be Photoshop and Blender, but, again, anything goes.”

Daniel Matthews, TRIXTER: “Some of my colleagues like to explore 3D with Blender or ZBrush, or even using 3D fractals like Mandelbulbs as a base for their concept art. However, I prefer to start out with free-flow using Photoshop and sketches, which sometimes allows happy accidents to occur. If I want to explore 3D, it’s usually with cool software called 3D Coat.”

Jelmer Boskma, Scanline VFX: “Some artists are more comfortable working in 3D and are able to provide high quality iterations in a highly economic manner that way. Others rely more on their traditional drawing and painting skills to communicate ideas. Certain tasks call for a specific approach, but, in general, whatever gets the job done well in a quick and organized manner wins. Whether that is through painting in Photoshop, sculpting in ZBrush, photo-bashing, a rough 3D render, a mock-up in SketchUp, or most commonly a combination of all of the above.”

Leandre Lagrange, Technicolor: “We do create a lot of FX simulations with 3D tools such as Blender and Vectron to give us a base to work from. With painting skills and photo-manipulations, there’s quite a lot that you can do for concepting those. It’s preferable not to have a Houdini artist go into it on their own and start shooting in the dark because it’s very time consuming and it’s so heavy. The idea is we’re giving them a look and feel and some direction, and then they can work with that. As it goes on, they can create more and more accurate FX simulations based on the images that we create. We have a close relationship with VFX artists for that type of complex FX work.”

TOP TO BOTTOM: Digital matte painting by Mujia Liao for the jawa camp in an episode of The Mandalorian. (Image courtesy of Pixomondo. Copyright © 2019 Lucasfilm Ltd.)

Concept by Mujia Liao for Midway. (Image courtesy of Pixomondo. Copyright © 2019 Lionsgate Ltd.)

A development frame by Framestore for Pan. (Image courtesy of Framestore. Copyright © 2015 Warner Bros. Pictures) Sidney Kombo-Kintombo, Weta Digital: “I storyboard a lot. I always try to have a really quick visual sketch of what I’m after. So Photoshop is a big part of it, just like any concept artist will do. But right after that it’s straight into Maya [for previs or postvis]. The best way to make sure that you’re not going to hit any hiccups is to make sure that you are doing this in the software package that will be used ultimately for the shots.”

KEY PROJECTS, KEY CHALLENGES

Daniel Matthews, TRIXTER: “The photostatic veil in Black Widow stayed in the refinement stage for a long time. As they say, 20% of the work takes 80% of the time. We had a rough idea of the look we were aiming for, but of course we wanted to make something new, exciting and as good as possible. The idea was to make this mask look like one of those peel-off self-care masks, with the ability to make the wearer look like someone else entirely. There was also the matter of finding the right shapes. In the end, we settled for the hexagon pattern, which fans say is a shape that features heavily in the Marvel Universe. We definitely get it – hexagons look great, have a futuristic tech-y feel and fit together neatly.”

Jelmer Boskma, Scanline VFX: “We worked on designs for Mysterio’s smokey FX appearance in Spider-Man: Far From Home. We provided concept illustrations in which we tried to capture the look and feel of the effect as much as possible within the constraints of a still image. For FX like these, illustrations only provide part of the solution. Much of the final design is based on the motion, timing and behavior of the simulation. We find that approaching design challenges like this from multiple angles is essential to finding the answer for a director. As a general principle, we like to instill as much real-world reference and logic into any design to give the idea a grounding in reality.”

Sidney Kombo-Kintombo, Weta Digital: “On the canyon chase in The Falcon and the Winter Soldier, we tried to see what we could

TOP: A sunken boat dining room concept by Scanline VFX for Miss Peregrine’s Home for Peculiar Children. (Image courtesy of Scanline VFX. Copyright © 2016 20th Century Fox)

BOTTOM: An ILM concept for one of the conquistador characters in Jungle Cruise. (Image courtesy of ILM. Copyright © 2021 Walt Disney Pictures) change in the original design of the canyon to make that a more thrilling experience. The first thing I do is go to dogfight reference – Star Wars comes to mind, and Japanese anime. We say, ‘How can we make this interesting?’ I try to come up with the idea for the sequence first and then the environment that makes that idea shine. We rebuilt the canyon from scratch. We did that in the animation department. Usually, the build comes out of modeling and layout, but here it was more a rough model of blocks of rocks done in the animation stage. Animation then passed it back to modeling who adjusted it. It’s a different way of creating the same type of work.”

Leandre Lagrange, Technicolor: “For Prometheus, some of our pieces of artwork were actually concepted by an artist who had a digital matte painting (DMP) background. So, they would actually end up being projected and used in the final VFX shots in the film. Then on Godzilla: King of the Monsters, for example, we worked on an image of Rodan emerging from a volcano. For that we used a design that was already done by Amalgamated Dynamics. We used their design and then built on top of that to make an image. That was more about making a key frame.”

Mujia Liao, Pixomondo:

“On The Mandalorian, we worked on several environments on the planet of Arvala-7, a very barren, desert-like planet covered in wet mud, home to creatures like the iconic Blurrgs. I turned to what I knew best and ended up matte painting the looks we needed for approval. We looked closely at a large number of deserts and found some good images of Atacama Desert after rainfall, although none really provided us with the level of detail we needed. Our CG Supervisor, Winrik Haentjens, then came across this area in Toronto’s Scarborough Bluffs with snow that was just starting to melt, creating these streams that flowed across a muddy surface, giving us some beautiful natural-looking erosion shapes. We took lots of photos of that area and created photogrammetry of the details, and that became the foundation to the closeup areas of Arvala.”

TOP LEFT TO RIGHT: Martin Macrae, Head of Art Department, Framestore

Mujia Liao, Head of Virtual Art Department/Art Director, Pixomondo Toronto

Jelmer Boskma, Visual Effects Supervisor/Art Director, Scanline VFX

Leandre Lagrange, Head of Art Department, Technicolor Creative Studios

Sidney Kombo-Kintombo, Animation Supervisor, Weta Digital

Daniel Matthews, Concept Artist, TRIXTER

James Clyne, Senior Visual Effects Art Director, ILM

A New Dawn in the VFX Art Department

The rise of virtual production has introduced many changes to the way visual effects studios operate. LED volume and stage work in particular require some of the final pixel VFX shot work to be carried out upfront – before shooting – rather than traditionally in post.

This means that some production companies and visual effects studios have now launched virtual art departments, or VADs. For example, Weta Digital established a VAD-like department out of work on AvatarandTintin,with a formal VAD on The BFG, while Lucasfilm and ILM ramped up a VAD with The Mandalorian.Other studios such as Pixomondo have followed suit.

“With the same emphasis in visual development and pre-production, the VAD focuses on not only design but creation of real-time assets from initial lookdev to production-ready models and environments for virtual production,” details Mujia Liao, Head of Virtual Art Department/Art Director at Pixomondo Toronto.

“We process practical scans and create custom assets to block out initial environments, allowing our clients to do virtual scouts and pre-light early on. In addition to the preliminary environment build, the VAD will also supply close to final key visuals by over-painting and hand-off design packages including photographic references and mood boards. This design package and early environment build sets a solid foundation for our virtual production and VFX teams to take further into fully realized worlds.”

Martin Macrae, Framestore: “Blade Runner 2049 was a definite highlight for all of us. For Vegas, we worked initially from designs coming from the production designer and Syd Mead sketches, and working closely with VFX we set to work designing everything from large buildings right down to street-level details, also designing the spinner drone that flew over the city – everything was thought out. We generated hundreds of sketches, paintings and 3D models, all with the purpose to help finalize a look for this sequence so the Montreal team could generate the final shots. As an art department we worked on Blade Runner for over a year and loved every second of it!”

James Clyne, ILM:

“On The Last Jedi we had to design this bomber. We were asking ourselves, what is a Star Wars bomber? How does that work in space? I started developing something that had a vertical bomb bay rather than a typical horizontal one. The director, Rian Johnson, said, ‘I like the idea, but you’re going to have to really show me that it works.’ So we went back to the drawing board and did drawings and models of it. It was story driving the design rather than the design driving the story. Through discussions and weeks and weeks building out what this could be, it became what you saw in the movie, which was amazing.”