VR/AR/MR: MYTH: A FROZEN TALE

IMMERSION IN THE VR WORLD OF MYTH: A FROZEN TALE

By CHRIS MCGOWAN

All images copyright © 2019 Disney.

TOP: Visual development by Brittney Lee. Myth: A Frozen Tale focuses on the duality and balance of the four elemental spirits of Frozen 2: earth, fire, water and air.

OPPOSITE TOP TO BOTTOM: The environments team, led by Michael Anderson, developed simple textures to enliven the vegetation while reducing visual noise, allowing for characters such as the Water Nøkk to stand out.

Visual development by Brittney Lee – the monoliths of Enchanted Forest at night

The monoliths of Enchanted Forest in the daylight. Walt Disney Animation Studios’ virtual reality short Myth: A Frozen Tale explores a world inspired by the feature film Frozen 2 and may point the way to an intriguing new art form: VR experiences that immerse viewers in settings and stories associated with their favorite films.

The approximately eight-minute-long Myth is the second VR short directed by Jeff Gipson, who won critical praise and the 2019 Lumiere Award from the Advanced Imaging Society for directing Cycles, the first VR film from Disney. While Cycles was a poignant look at a couple’s life lived in a beloved house, Myth immerses the viewer in the myths of Frozen 2. In Arendelle [the kingdom in Frozen], a mother reads a bedtime story to her children and the audience is transported to an enchanted forest where the elemental spirits come to life and the myth of their past and future is revealed. “They’re very different films,” Gipson comments. “Myth is more complex with many more effects than Cycles. We really leaned into having the music and the animation tie together, almost like in Fantasia.” Nicholas Russell is the producer, Jose Luis Gomez Diaz (Cycles) is the VR Technology Supervisor and Joseph Trapanese (Tron: Legacy) composed the original score. Evan Rachel Wood (Westworld, Mildred Pierce), who voiced the character of Queen Iduna in Frozen 2, is the film’s narrator, and her participation is an indication of Disney’s serious commitment to the project. Brittney Lee, a key visual development artist on the Frozen movies, is the Production Designer of Myth, and drew inspiration for it from the works of legends such as Disney Art Director Eyvind Earle (Sleeping Beauty) and visual development artist Mary Blair (Alice in Wonderland, Cinderella). She was intrigued by Gipson’s vision of creating a stylized world with the visual influences of pop-up books, graphic silhouettes and stage elements from vaudeville and music hall productions. “Something we tried to push on in Myth was this sense of 2D animation,” says Lee. “I wanted to have hand-drawn animation because Fantasia was one of the main

“One of the things in the film that I really gravitated towards was the elemental spirits. I just loved these characters. … What’s so great is that with the water Nøkk you feel its scale, with the salamander you feel how tiny it is on a rock, and with the Earth Giants you see them massively in scale. That’s the beauty of VR. You’re able to feel things that you’re not able to feel on a flat screen.” —Jeff Gipson, Director

inspirations for this film. So we have a lot of our effects artists on the project. [Effects artist] Dan Lund came in and did beautiful 2D hand-drawn effects.” Gipson adds that the 2D animated work “adds a sense of heritage from Disney animation.” Myth started when Frozen and Frozen 2 writer and co-director Jennifer Lee approached Gipson about exploring the world of Frozen for his next VR project. “When Jenn approached me to create something, I felt a mixture of emotions,” Gipson recalls. “I was nervous, I was excited. I was intrigued. What is cool is that Jenn gave me complete creative control on the film that I wanted to make. I was empowered but I was also nervous. Frozen is one of our most iconic films at the studio, and those worlds and those characters are so special. I wanted to do something that would do that film justice and feel like it belonged to that world.”

Gipson’s concept for the VR film came about a year ago when he was watching some early screenings of Frozen 2. “One of the things in the film that I really gravitated towards was the elemental spirits,” he comments. “I just loved these characters.” These include the Nøkk, mythical water spirits that take the form of stallions with the power of the ocean; Gale, a wind spirit that is playful and curious and can also rage with a tornado’s force; the Earth Giants, massive creatures that are the spirits of the earth; and Bruni, the fire spirit in the form of a tiny, fast-moving fire salamander who can wreak havoc in a forest in seconds.

At the same time, he was thinking about his own family’s

“Something we tried to push on in Myth was this sense of 2D animation. I wanted to have hand-drawn animation because Fantasia was one of the main inspirations for this film. And so we have a lot of our effects artists on the project. [Effects artist] Dan Lund came in and did beautiful 2D hand-drawn effects.” —Brittney Lee, Production Designer

TOP: Visual development artwork of the fire salamander by Production Designer Brittney Lee.

MIDDLE: Lee’s call-out sheet for the fire elemental spirit, a salamander.

BOTTOM: Myth: A Frozen Tale film poster tradition of telling bedtime stories. “We’re all told bedtime stories and we can create our own versions of the stories that we’re told. And that’s kind of what VR does. It takes you into a world, it transports you. So I thought a bedtime story would be a real cool way of framing the elemental spirits.”

Gipson imagined living in Arendelle and being told the film’s bedtime story, which includes the elemental spirits. He continues, “What’s so great is that with the water Nøkk you feel its scale, with the salamander you feel how tiny it is on a rock, and with the Earth Giants you see them massively in scale. That’s the beauty of VR,” he says. “You’re able to feel things that you’re not able to feel on a flat screen.”

That sense of presence creates a certain “wow factor” with well-done VR. “That’s what makes Myth so special,” Gipson attests. “Early on when we pitched the first version of this, one of the things Chris Buck (co-director of the Frozen movies) challenged us on was, ‘what does Elsa feel like when she’s with the water Nøkk, with the salamander, under the Earth Giants, with the wind? How can the audience have that same feeling, that experience of presence with these characters?’ “In Myth, the Nøkk comes up so close to you,” Gipson continues. “You feel the presence of this horse, the same with the Earth Giants towering above you or the wind swooping around you.” Myth also has a brief portion that is interactive, which adds further presence. At one point, if you come close to the salamander, it scurries away. In Myth, as in Cycles, guiding the audience’s eye is a challenge. “That’s one of the biggest challenges in VR currently, and there are lot of different solutions. We did that in Cycles through motion and movement and color and light. And we do something similar in Myth, where we’re always trying to guide you through movement of the characters, or through the sound, or some of the lighting techniques. We had the Gomez Effect in Cycles, and we re-implemented that – it is the technique that if you’re not looking at the area where the action is taking place, it darkens and [desaturates]. So, everything that is in full saturation and beautiful is where you’re supposed to be looking.” Myth was different from Cycles in assigning each character its own score, palette and language. “Each character has its own piece of music, much like in Peter and the Wolf. What’s great is our composer, Joe Trapanese, came on very early in the process and our animators actually had pieces of the score to animate to, which

“[Making Myth with Unreal Engine] was a gamble in general. We had never made anything with Unreal at the studio, but our technology team – Jose Luis Gomez Diaz [VR Technology Supervisor], Mike Anderson [VR Environment Lead] and Ed Robbins [VR Character Lead] – felt strongly about jumping in and trying to use Unreal. They enjoyed making the film and learning this kind of process.” —Jeff Gipson, Director

is rare. At the studio, the animators had this cool challenge of how to match the movements to the music.

“There’s also a lot of art direction choices that we did in Myth,” he adds, “where each character has its own color palette as well, and the world transforms and changes. There’s a changing of seasons in this world that is constantly evolving.” In terms of software, Gipson made Cycles with Unity and Myth with Unreal Engine. “This was a gamble in general,” he notes. “We had never made anything with Unreal at the studio, but our technology team – Jose Luis Gomez Diaz, Mike Anderson [VR Environment Lead] and Ed Robbins [VR Character Lead] – felt strongly about jumping in and trying to use Unreal. They enjoyed making the film and learning this kind of process.”

Gipson’s team also used Swoop, an in-house tool written at Disney. “It’s a kind of tool where basically you’re drawing paths in VR. We’re putting on the goggles and we’re on the set, and our animators are able to grab the VR controller and orchestrate a movement, drawing a path for Gale and for the salamander’s fire trail around this.”

The team continued to use Quill, a VR painting program, for storyboarding, as well as VisDev. Danny Peixe, who was also on Cycles, used it for storyboarding in VR. “We used the same techniques for Myth except much more elaborate. We did this early on just to get the flow, the sense of proximity to our characters. How the story would play out around us.”

Gipson enlisted the help of Skywalker Sound on Myth. “Skywalker is an amazing group of people,” he says. “We worked with sound mixers there and they have their own teams specifically for VR. In Myth, there are 200 different sound elements, which is pretty incredible for a VR film, and each one is spatial. The score changes as you look around, so if you’re not looking in the right space it sounds almost as if you ducked out of the concert.” The climax of this film is a kind of a “visual poem where the music and the animation are all married together.” Myth: A Frozen Tale will screen at festivals and events this year. TOP: Ian Coony, VR Effects Lead (left) and Brittney Lee share their thoughts at an Effects Review in the ‘Myth FX VR Lab.’

BOTTOM LEFT TO RIGHT: Jeff Gipson, Director; Brittney Lee, Production Designer; Jose Luis Gomez Diaz, VR Technology Supervisor; Mike Anderson, VR Environment Lead; Ed Robbins, VR Character Lead; and Danny Peixe, Artist/Illustrator.

THE ACADEMY SOFTWARE FOUNDATION AND THE ADVANTAGES OF OPEN SOURCE SOFTWARE

By CHRIS MCGOWAN

“Now that there is a clear path to inclusion for engineering work, we are seeing a large uptick in new code being written and older bugs fixed. It speaks volumes about the visibility that the Academy Software Foundation has brought to this important work, and the engineering talent that is looking to contribute to the future of filmmaking software.” —Rob Bredow, Sr. Vice President, Executive Creative Director and Head of ILM, and Chair of the ASWF Governing Board

As visual effects have proliferated in the movie and TV industries, open source software (OSS)’s appeal and importance have steadily grown. As a result, the Academy of Motion Picture Arts & Sciences (AMPAS) teamed with the Linux Foundation in August 2018 to form the Academy Software Foundation (ASWF) to create and maintain free open source software for the industry and construct a healthy ecosystem for engineers and artists who want to contribute to this software. The VES is an Associate Member of the ASWF.

The origins of the ASWF trace back to when the AMPAS Science and Technology Council conducted a two-year survey into the use of OSS across the motion picture industry and discovered that almost 84% of the industry uses open source software, especially for animation and visual effects. AMPAS undertook the survey because it identified open source software OSS as a key strategic area for the motion picture industry.

“Open source software is used in the creation of every film, and every frame of film is stored digitally in open source software formats – this software is critical infrastructure for both making and preserving movies,” says Rob Bredow, Sr. Vice President, Executive Creative Director and Head of ILM, and Chair of the ASWF governing board. The initial investigation included an industry-wide survey, a series of one-on-one interviews with key stakeholders, and three Academy Open Source Summits held at the Academy headquarters, according to Andy Maltz, Managing Director, Science and Technology Council, AMPAS, and ASWF Board Member.

Comments Bredow, “They identified the key common challenges they were seeing with open source software. The first was making it easier for engineers to contribute to OSS with a modern software build environment hosted for free in the cloud. The second was supporting users of open source software by helping to reduce the existing version conflicts between various open source software packages. And the third was providing a common legal framework to support open source software.

“The mission of the Academy Software Foundation,” Bredow elaborates, “is to increase the quality and quantity of contributions to the content creation industry’s open source software base; to provide a neutral forum to coordinate cross-project efforts; to provide a common build and test infrastructure; and to provide individuals and organizations a clear path to participation in advancing our open source ecosystem.”

The ASWF has achieved solid early acceptance, with AWS (Amazon Web Services), Animal Logic, Autodesk, Blue Sky Studios, Cisco, DNEG, DreamWorks, Unreal Engine, Google Cloud, Intel, Microsoft, Movie Labs, Netflix, NVIDIA, Sony Pictures, Walt Disney Studios, Weta Digital, Foundry, Red Hat, Rodeo Visual Effects Company and Warner Bros. already on board.

“The ASWF work is an excellent vehicle for healthy open source collaboration and adoption. They are supportive of other industry efforts by the VES and others because all of those groups are collaborating toward a common goal to standardize the areas of our industry that do not give anyone a competitive advantage.” —Steve Shapiro, Marvel Studios Director of Software and Production Technology

WHERE ARE WE NOW?

Says Guy Martin, Director of Open Source Strategy at Autodesk and Board Member of the Academy Software Foundation: “OSS is the lifeblood of VFX departments in the context of how it enables them to use best-of-breed tools – and even write their own in-house applications – while seamlessly moving their effects and shots among these tools. Vendors like Autodesk have been asked why we support open source in the visual effects space. The answer is simple – our customers demand it. They use our tools like Maya and 3ds Max, but also a variety of other tools, and combine them in their tool pipelines to make great content. Any tool vendor or technology partner who is not participating in open source will be behind the innovation curve. There is healthy competition among many of the vendors and technology providers in this space, but there is also universal acceptance that open source is the ‘lingua franca’ that allows the industry to successfully share and collaborate.”

“We are no longer in the infancy of open source software and open standards within the movie, TV and VFX industries,” comments Steve Shapiro, Marvel Studios Director of Software and Production Technology. “Within VFX and post-production, we have been moving from a model of proprietary solutions toward collaborative standards. We are past the infancy of that migration, though there is still a long way to go before it is a mature practice.”

OPEN SOURCE ADVANTAGES

By having an open standard, everybody can contribute their development resources towards that, rather than having many different proprietary formats for common data and workflows, according to Jordan Thistlewood, Group Product Manager for Katana, Mari and Flix at Foundry. “It’s the idea that everyone is invested in a common technology that covers all the basics. This philosophy extends through many types of file formats throughout the industry, so everyone contributes to one massive technological pool,” Thistlewood comments in the essay “Open Standards Will Change the VFX Industry As We Know It” on Foundry.com. “By sharing the code with a large number of engineers facing diverse challenges and allowing those engineers to improve the software and share it back to the community, we get better quality software. In addition, when you think of long-term preservation of data, sharing the algorithms for reading the data openly helps ensure the data is readable long into the future,” explains Bredow.

OPPOSITE TOP LEFT: Rob Bredow, Sr. Vice President, Executive Creative Director and Head of ILM, and Chair of the ASWF Governing Board.

OPPOSITE TOP RIGHT: Guy Martin, Director of Open Source Strategy at Autodesk and Board Member of the Academy Software Foundation.

OPPOSITE BOTTOM: Steve Shapiro, Marvel Studios Director of Software and Production Technology.

TOP LEFT: Jim Jeffers, Sr. Principal Engineer and Sr. Director of Advanced Rendering and Visualization at Intel Corp., and ASWF Board Member.

TOP RIGHT: Andy Maltz, Managing Director, Science and Technology Council, AMPAS, and ASWF Board Member.

BOTTOM LEFT: Daniel Heckenberg, R&D Supervisor - Graphics, Animal Logic, and Chair of the ASWF Technical Advisory Council.

BOTTOM RIGHT: Jordan Thistlewood, Group Product Manager, Katana, Mari, Flix, Foundry.

“Long-term preservation of digitally-created motion pictures is a real challenge. How does one access a digital movie 100 years from now given the pace of technological innovation – and obsolescence? Standardized open source software APIs and file formats are key solutions to this challenge.” —Andy Maltz, Managing Director, Science and Technology Council, AMPAS, and ASWF Board Member.

Maltz adds, “Long-term preservation of digitally-created motion pictures is a real challenge. How does one access a digital movie 100 years from now given the pace of technological innovation – and obsolescence? Standardized open source software APIs and file formats are key solutions to this challenge.”

“The most successful examples of open standards and software will improve the quality of life for large productions or post houses. VFX houses have to collaborate on shared shots and shared assets much more than ever before, which would be significantly harder without some standards,” says Shapiro.

Says Jim Jeffers, Sr. Principal Engineer and Sr. Director of Advanced Rendering and Visualization at Intel Corp., and ASWF Board Member, “Either for speed of development of compelling and complex assets or to take advantage of the latest state-of-the-art techniques in content development, it’s important to have interoperability and interchange become as seamless as possible. Through collaborative open source projects, the industry as a whole benefits from this kind of ‘standardization’ that still is changeable and tunable, which is a natural state for OSS.”

“Either for speed of development of compelling and complex assets or to take advantage of the latest state-of-the-art techniques in content development, it’s important to have interoperability and interchange becomeas seamless as possible. Through collaborative open source projects, the industry as a whole benefits from this kind of ‘standardization’ that still is changeable and tunable, which is a natural state for OSS.” —Jim Jeffers, Sr. Principal Engineer and Sr. Director of Advanced Rendering and Visualization at Intel Corp., and ASWF Board Member

IDENTIFYING CHALLENGES

The AMPAS survey identified various challenges to the use of OSS, including siloed development, managing multiple versions of OSS libraries (“versionitis”), and varying governance and licensing models that need to be addressed in order to ensure a healthy open source community.

Shapiro observes, “Sharing technologies across studios has a variety of challenges, but open source standards do make it easier to collaborate on industry solutions to difficult problems. However, it is not a silver bullet because the standard is just the start. The adoption and evolution of those standards is crucial to their success. If a standard is not leveraged in the right places and improved to adapt to changes within the industry and technology, it will be antiquated before it is widely used.”

“Visual effects is still an innovative industry with a lot of shows breaking new ground each year. The really successful and widely adopted work we have seen in open source software has really been to provide useful core libraries for the basics – reading images, reading geometry, handling looks for shots – and that is key to get right. In terms of sharing something very complicated like a full rigged and groomed character, that’s not yet an area that we are seeing standardized due to the amount of innovation we are still seeing in the space,” says Bredow.

Sustainability, effective collaboration and clarity of purpose are challenges common to OSS projects, according to Daniel Heckenberg, R&D Supervisor - Graphics, Animal Logic, and Chair of the ASWF Technical Advisory Council (TAC). “The ASWF supports sustainability and collaboration by providing established and successful governance and

“Any tool vendor or technology partner who is not participating in open source will be behind the innovation curve. There is healthy competition among many of the vendors and technology providers in this space, but there is also universal acceptance that open source is the ‘lingua franca’ that allows the industry to successfully share and collaborate.” —Guy Martin, Director of Open Source Strategy at Autodesk and Board Member of the Academy Software Foundation

process models for its projects. Key to these outcomes is the reduction of risks and barriers to entry for contributions and involvement through standardization of licensing and engineering resource commitments from members.”

LEGAL AND LICENSING

“The biggest complaints we heard from the community were around the wide variety of licensing requirements and some of the corporate requirements around adopting and contributing back to open source projects. Even free software has a cost if you need a team of lawyers to pick through obscure licenses in order to confirm you can leverage the tool for your next film. One of the ways the Academy Software Foundation can help is ensuring that our projects have common, permissible licenses that are easy for companies to review once and accept for their engineers and artists,” says Bredow.

ACCOMPLISHMENTS TO DATE

Adds Bredow, “One of the great things we have seen in our first year at the Academy Software Foundation is the tremendous increase in code contributions from a much larger number of contributors. In the past, many of these projects only had one or two main contributors, usually from one sponsoring company. Now that there is a clear path to inclusion for engineering work, we are seeing a large uptick in new code being written and older bugs fixed. It speaks volumes about the visibility that the Academy Software Foundation has brought to this important work, and the engineering talent that is looking to contribute to the future of filmmaking software.

“The first projects that the Academy Software Foundation have adopted are OpenColorIO, OpenVDB, OpenEXR, OpenCue and OpenTimelineIO,” says Bredow. “It’s important to note that the foundation is not a standards body and doesn’t act in that role. However, the nature of open source software often leads to de-facto industry standards, since many companies are already contributing to and using the software. So, when our software, like OpenEXR, gets wide adoption and becomes a de-facto industry standard, that’s great news too.”

OTHER OPEN SOURCE ECOSYSTEMS

“I think the ASWF work is an excellent vehicle for healthy open source collaboration and adoption,” notes Shapiro. “They are supportive of other industry efforts by the VES and others because all of those groups are collaborating toward a common goal to standardize the areas of our industry that do not give anyone a competitive advantage.”

The VFX Reference Platform, set up by the Visual Effects Society (VES), also works towards maintaining standards in this space. Initially focused on Linux only, the VFX Reference Platform is a set of tools and library versions to be used as a common target platform for building software for the VFX industry.

Comments Jeffers, “There is a pretty significant open source ecosystem that is related to but not directly managed by the AWSF. Intel is part of the ASWF to support the overall beneficial use of open source software, both the directly managed codes and the large ecosystem of OSS that is not yet an AWSF fit right now or that is standing strong as is. Liberally licensed codes like Intel’s Embree ray-tracing kernel library and our recently launched Open Image Denoise are becoming important tools for animation and VFX workflows. The Pixar-led Universal Scene Description (USD) OSS project for content and asset interchange is enabling multiple studios to collaborate and share the development and expertise of many to make a high-quality film or TV show. A robust mix of open source and advanced commercial software tools is now the norm for making modern films with digital assets.”

INDUSTRY ACCEPTANCE

“The Academy Software Foundation is really a broad industry effort. It’s always free to contribute to and use the software that is hosted by the foundation. And we are supported by members who provide engineering support and funding to ensure a successful foundation. In the first year, our membership base has almost doubled, and five open source projects have applied and been accepted into the foundation. We are seeing growing support for the foundation both at technical and management levels,” says Bredow. More and more people are moving away from closed standards to open standards, says Thistlewood. “There’s now a proven track record of these projects, so there’s now faith in what they will become.”.

“We have gone past the tipping point,” concludes Shapiro, “so it is no longer a question if open source software and standards are worthwhile. The question now is what fits within that model and how do we prioritize those efforts to bring them into the community.”