Unity @ Sundance Film Festival '18

The future of storytelling is real-time Real-time cinematography 3 things you should know

unity.com/sff

How VR and Virtual Rooms are reshaping brand advertising

Oscar-nominated Neill Blomkamp on a new way of creating films

Plus Sneak peeks of Made with Unity projects that are sure to inspire

The future of storytelling is real-time

With its imaginative storytelling, unique art style, and creative interactivity, the award-winning Monument Valley 2 by Ustwo Games is the latest installment of an indie puzzle game that has delighted audiences and critics worldwide.

2

Did you know?

interactive New Frontier projects are made with Unity

2/3 of all global AR/VR is made with Unity

90% of

The real-time revolution W

elcome to the 2018 Sundance Film

truly changing the way we live, work, and play,

Festival! As long-time supporters of in-

whether its training astronauts and surgeons,

dependent artists, we at Unity couldn’t be more

programming lights on a bridge, meeting your

thrilled to be here in Park City celebrating the

favorite superhero in AR, mastering a beautiful

pioneers of storytelling, including virtual and

puzzle game (page left), or shooting a CG film

augmented reality experiences (page 4) and

using photogrammetry.

bold new films created in groundbreaking ways.

Unity’s core value is democratizing develop-

HoloLens projects

Maybe you’ve heard about the ‘real-time revolu-

ment. We believe there’s nothing more sacred

use Unity

tion’ happening in the industry (page 12), or you

for the artist community than the freedom and

know that technology advances in the game

ability to express themselves (page 8). Our

industry are now beginning to power the most

mission is to help you bring your visions to life

More than

efficient, collaborative, and creative workflows

and reach audiences who want to feel inspired,

12 billion

for film, TV, VFX, and CG animation (page

informed, and entertained by what you create.

Made with

26). Maybe, like us, you’re more excited and

Unity game

optimistic than ever about the future of these

We look forward to meeting you here in

installs

industries. Perhaps you’re not quite sure what

Park City. You can track our whereabouts and

in 2017

this shift means for you yet.

panels by following us on Twitter (@unity3d).

We assembled this short collection of tips, case

Happy creating!

100% of

studies, and interviews to introduce you to one

the top

of the largest creative communities in the world,

20 game

which we invite you to join (page 17). Millions

companies

of people around the globe use Unity to create

use Unity

immersive or interactive experiences that are

3 billion devices reached worldwide

P.S. Don’t miss our exclusive Sundance Film Festival Storytellers of Tomorrow interview series. We’ll be interviewing creators all week and posting them on unity.com/sff during the festival.

Cover: ADAM: The Mirror, a short film made in Unity by Neill Blomkamp (District 9, Elysium) and Oats Studios.

3

unity.com/sff

70% of

4

The future of storytelling is real-time

Celebrating New Frontier creators VR, AR, and the landscape of immersive storytelling

L

ast year I was blown away by New Frontier, the Sundance Film Festivals’s exhibition

that explores the convergence of film, media, and virtual and augmented reality. Since Unity’s technology powers the majority of the world’s “XR” (as many of us refer to it now), we are here to support and celebrate our talented creator community once again.

First, let’s reflect on some of the projects that redefined our understanding of immersive storytelling, agency and narrative in 2017.

ASTEROIDS!, by Baobab Studios An interactive VR tale with feature-film quality art, this experience puts you at the center of the action in the form of a helper-bot to two quirky aliens, Mac and

and compassion, directed by Eric Darnell (Madagascar). Released in December.

5

unity.com/sff

Cheez (the latter voiced by Elizabeth Banks). It’s a story filled with humor, wonder,

Celebrating New Frontier creators Zero Days VR, by Scatter Weaving traditional media (video, audio, CG animation, charts) into a powerful VR presentation, this project will unnerve and humble you. Follow a computer virus on a clandestine mission to sabotage an underground Iranian nuclear facility in this story based on the Oscarshortlisted documentary.

Heroes, by Melissa Painter A dance performance unlike any other, this hybrid AR and 360 project is set to the iconic David Bowie song and takes audiences deep inside an athletic and intimate duet at the historic Ace Hotel in downtown L.A. Even your own body movements

The future of storytelling is real-time

are part of the experience.

Life of Us, by WITHIN A billion years in a few thrilling minutes—this piece from Chris Milk and team advanced our understanding of ‘presence’ in VR. As you journey through the story of evolution, while interacting with your multiplayer companion, positional tracking and voice modification surprise and delight you.

6

is growing. As a result, the way audiences consume entertainment will change, and practical business applications will fundamentally change

When the world sees these fundamental shifts,

as well. Fields like automotive, medical, and architecture will soon reimagine what it means to be efficient, safe, and accessible.

Now it’s up to you But it all starts with artists like you. When the world sees these fundamental shifts, booms,

booms, and catalysts, it’s the storytellers who will plot the course we take.

and catalysts, it’s the storytellers who will plot the course we take. Technology advancements mean nothing without the artist sketching an idea at midnight and the writers crafting narrative tricks.

It’s all about the content now, and that’s right where Unity lives. We solve hard problems under the hood, so creators like you and millions of others around the world can focus instead on the craft and joy of storytelling.

Since then

new breeds of art. Fidelity, abstraction, scale —

New platforms, breakthrough artists, design

these will be the stepping stones as we forge

best practices, you name it. I joke with my col-

ahead, moving us from traditional storytelling

leagues and friends that one year in the world

assumptions to literally creating whole new

of XR brings so many new changes that it feels

worlds. These worlds may begin as escape or

like a decade (it’s true). Apple and Google’s

entertainment, but they have the potential to

advancements with ARKit and ARCore, respec-

improve the quality of our lives.

We firmly believe the world is a better place with more creators in it. We look forward to another stunning New Frontier, another year of breakthroughs, and another stepping stone crossed on our way to finally understanding that the only real limit to what you can create is what you can imagine.

as a creator to reach hundreds of millions more

If I can geek out for a second: I want to call out

Tony Parisi is Head of VR/AR

people, and it’s only going to get better. We

XR’s impact on the computing landscape for

Strategy at Unity, the co-creator

believe that AR is going to be able to reach a

consumers and businesses alike. Investments

of 3D graphics standards, and

billion devices by the end of 2018.

are increasing, more industries are getting

Directors are using these mediums to create

the author of several O’Reilly

on board and reaping the benefits, and the

Media books on virtual reality and WebGL, such

quantity and level of involvement of consumers

as Learning Virtual Reality (2015).

7

unity.com/sff

tively, have exponentially expanded your ability

Bringing the ‘art’ to technical artistry Unity’s Yibing Jiang tells you why real-time is a game-changer

I

The protagonist

The future of storytelling is real-time

(right) of Windup, a cinematic art test in progress at Unity.

t’s no secret that real-time content creation

An industry in flux

and filmmaking are merging. Nowadays,

If you work in VFX, animation, or even live

games even feel like films—not just because

action TV or film, you already know that it’s

they have higher visual quality than before—but

not uncommon to experience anxiety about

also because the narratives often unfold with

the state of the industry. Due to foreign tax

very cinematic scenes. Having worked in both

breaks for the arts and a shifting ecosystem

film (Pixar, Disney) and gaming (Naughty Dog,

in general, many studios are moving abroad,

Ready at Dawn), it’s been life-changing for me

downsizing, or rethinking what kind of projects

to witness the growing demand in film for re-

they should pursue. The workforce in film (and

al-time technology. We’re all learning new ways

games) is used to a bit of job insecurity here

of storytelling together, as the two areas merge.

and there—which is no doubt why there are so many passionate people working so hard in

Yet there’s a myth that persists: if one thing is

those industries.

easier to create than another, it must be lower quality. In film, this myth manifests as the

Yet many of my peers are only now realizing

idea: if it doesn’t take a long time to render, it

what I’ve been saying for years: if you want to

must not be beautiful or complex. My standard

increase your relevance and value as a creator,

response to this is that if you want something

it is crucial to familiarize yourself with real-time

beautiful, remember that art school can be a

technology. The credibility for real-time has

more powerful resource than a render farm.

finally reached the point we knew it would; now

8

9

unity.com/sff

it’s a matter of educating creators on how they can develop the next wave of gorgeous stories.

What makes a great technical artist?

The biggest lesson I learned from working at Pixar (which can be very tech-focused) and Disney (which can be very art-focused) is that the foundation of technical artistry must be built with an equal balance of both those elements.

If artists in production don’t have an appreciation for the pipeline and what’s under the hood, it could limit their potential for making magic with it. Conversely, it’s quite common for conversations in San Francisco (where I live and work) to lean too far towards the programming side. Each studio must find the right balance for itself.

I personally believe that you have to be a great artist first. That’s fundamental. A lot of kids and students ask me what programming language they should learn, and I always reply, “Do you

With close-up shots like

want to be a tech artist because it’s easier to

The future of storytelling is real-time

this, Windup needed

get a job, or are you a great artist who is pas-

a high level of detail.

sionate about technology?” You can’t skip the

With fewer performance

basics. Great stories need great art. My advice

limitations than in

for companies putting teams together is not to

gaming, Yibing was able

forget the art part when looking for technical

to craft a realistic 3D

artists. That’s the key to a great story coming to

model for the leaves.

life before your eyes in a magical way.

Another question I get often is how to elevate the level of quality. Image quality, for me, is largely how color is balanced and how unique 10

then worked on the sweater in real-time. The

This is the most I’ve ever felt like a true artist, in that this is the most

result? We can change the color and size of the weaves without waiting.

This is the most I’ve ever felt like a true artist, in that this is the most authentic way to create art: draw, look at it, and draw some more.

authentic way to create art.

Because of technical limitations in traditional CG animation pipelines, artists usually don’t have the opportunity to look at what they’re creating in its final form. You’re part of a pipeline, basically guessing what the final image should

the design is. I also like to see people spending

highlights, which makes the result too noisy.

more time on animation and rigging. Graphical

Removing them makes the cloth materials look

quality is about balance, and it all starts with

much more refined. Unity has more accurate

concept art. If the original concept doesn’t

material and lighting responsiveness (very

have the right aesthetic style and balance,

close to offline rendering solutions), especially

then it’s no surprise that once you put the tech

for materials like stone, metal, and plastic. As

in, it won’t be pretty to look at. Audiences are

a result, we should see more and more visually

familiar with the uncanny valley by now, but

stunning stories that use real-time rendering in

that’s not the only risk. The best filmmaking en-

the very near future.

be, and hoping you’re making the right artistic decisions. Sometimes you’re even told to paint something and you’re not even sure why.

Working on Windup, even though it’s a small team, we are doing everything together: painting, adding lights, balancing color, changing lenses ­­— all at once. I can actually see what my art looks like in its final form, at every second, as I go. Directors and artists will love this,

learning from each other while working closely,

I joined Unity because I saw the opportunity of

which will serve both to provide that balance

real-time storytelling. I’m combining my expe-

and to optimize the pipeline.

rience from film and gaming to show people that you can get feature-film quality with these

because there’s no more waiting for overnight renders. This is what authentic artistry feels like, and it all comes down to freedom. As a technical artist, there’s nothing more powerful than that.

Why real-time storytelling feels more authentic

tools. For example, I’m working on a cinematic

I helped with the cloth, skin, and eyebrows for

art test at Unity now called Windup, in which

the recently released Neill Blomkamp short

the main character wears a fuzzy red sweater.

Yibing Jiang is the Technical Art

films, ADAM: The Mirror and ADAM: Episode 3.

On a big production, I would have had to render

Supervisor on the Made with

To improve the visual quality, we used some

for long periods of time to make things look

Unity team. Her past experience

cleaner habits while creating content. For ex-

right each time I made a change. At Unity, on

at Pixar, Disney, and Naughty Dog

ample, some textures on the cloth have unnec-

the other hand, we created the fabric details,

includes set creation, look development, and

essary information, like shadows and specular

baked them down as small textures first, and

character shading, respectively.

11

unity.com/sff

vironment should have artists and tech experts

Secrets of the real-time cinematography revolution The future of storytelling is real-time

Adam Myhill, Head of Cinematics at Unity, shares three facts you may not know

H

ere’s something to consider while you

It’s something I call “100 years of lens lan-

sip coffee and enjoy the view in Park

guage,” and those who know me also know I

City: why are video game genres named after

could talk about this all day. Because we’re here

camera lingo? First-person shooter, third-person

at the Sundance Film Festival, spreading the

strategy, isometric. As someone who’s worked

word about the real-time revolution that’s taking

on film sets, created trailers, and developed

the industry by storm, I want to share some fun

camera technology for both game developers

facts about cinematography — particularly what

and filmmakers, I see this as no accident. It’s

we have been refining over the past year.

one of many indicators that cameras, and their way of shaping points-of-view, aren’t just how

For those who haven’t yet used Unity to tell a

worlds are experienced—they’re also how we

story, my hope is that this inspires you and gets

experience the world.

your wheels turning.

12

Cinemachine automatically keeps the character in frame as the camera is moving across the dolly track.

Cameras can be smarter than you — and you won’t believe how much that it helps your project.

For example: if I tell Unity that I want this char-

If you’ve ever dreamed of having a top-notch

out of sight. I can also mark where in the frame

team of DPs and editors — the best in their

I want her specific body part to remain (say,

field, to execute your every whim — I can’t wait

the forehead), which ensures even after adding

to introduce you to procedural cinematography.

complex animations for her and nuanced

These ‘smart’ cameras can make decisions

movement for the camera, your target will stay

faster than it takes you to tell them what to do.

framed for you. I could even blend camera A

acter to be kept visible in frame, and she walks behind a wall, the system will automatically cut from camera A to camera B as soon as she is

13

unity.com/sff

1

Real-time cinematography a

b

movement into camera B, as if they are one fluid tracking shot, working together to capture the scene’s action and emotions fluidly, as one. c

Did I mention there’s an infinite amount of dolly track, as well as handheld noise simulators, shot evaluation, collision detection, and more?

These are only a few simple examples of

Doing motion capture

how efficient this tech can be for you when

with actors while seeing

executing your aesthetic style, especially when

the virtual environment

you take into consideration that even a short film may have hundreds of different shots.

in Unity helped Neill and

a

the Oats team iterate

Procedural cameras can literally save you

quickly. Notice the actor

b

hundreds of hours of work, and that is precious

(a) and actress (b) using

time you can now spend on experimentation

props (c) which are

and iteration instead of setup.

digitally replaced.

c

2

Your live-action cinematography skills are transferable to real-time.

The future of storytelling is real-time

Not a week goes by that I don’t hear from someone with a physical cinematography

Rather than shooting motion capture (mo-cap)

The benefits of this workflow extend through

background ­— ­who has never used a real-time

and then adding the world behind the actor

every phase of development, like ‘location

engine before — how delighted and accom-

later, you can build out your world first, connect

scouting’ with a VR headset on inside an

plished they feel trying it for the first time.

the physical camera position to that of the digi-

artist-created 3D world. You can rapidly

tal one in Unity, and then shoot your actor while

experiment with pre-visualizations, and you

Virtual production is another example of this

seeing the CG world around them at the same

can also adjust the look-and-feel without

applicability. Let’s unpack this one. Filmmakers

time. This means you can find the best angles

waiting for post-production; there are famil-

of projects with partial or full CG environments

and get inspired from your environment in the

iar controls for lens emulation like bloom,

are now using Unity to shoot a CG film the

moment, on-set, just as you would in traditional

vignette, tone-mapping, motion blur, depth of

same way as a live-action set.

live action productions.

field, and more.

14

Filmmakers of projects with partial or full CG environments are now using Unity to shoot Because it’s digital, you don’t have to be in the

a CG film the same way as

same physical space either—crew can view the set and footage remotely in headsets and collaborate at a global scale. Compared to tra-

a live-action set.

ditional CG and VFX, this brings exponential increases in creative freedom, rapid prototyping, cost savings, live feedback and decision-making, asset reuse, and even potentially a better aesthetic look and feel to the project, which is

at your fingertips when you can tune camera,

animation studio. Innovative projects require

lighting, and post FX all at once, at any point in

innovative solutions, and designing new tools

The best part? Once you’re done, you can return

the process. This is the revolution of real-time

for precisely your needs is a must for pioneers.

to that world any time, like for sequels or ex-

filmmaking.

tended scenes, and you can reuse those assets

For those who don’t build their own tools, note

for other experiences, such as a companion

that many artists and programmers create and

AR app or game that will help extend the life of your IP. These once-futuristic workflows are becoming

3

offer new solutions on our Asset Store, which

You can build on top of our tools.

Go wild.

has tens of thousands of available packages you can easily access to jump-start your production.

Last year I met two PhD researchers at INRIA,

more mainstream, with dozens of projects currently in the making—including major blockbust-

The value of extensibility is hard to over-

the French Institute for Research in Computer

ers coming soon. Imagine the creative polish

state—I’ve heard this from almost every major

Science and Automation, who brought our

Ready to join the real-time revolution? The Unity Asset Store can help. It’s a massive catalog of over 40,000 free and paid tools and assets for creators and storytellers. You’ll find groundbreaking cinematic tools like Cinemachine and the Post-Processing Stack to help you create captivating content for the next frontier of filmmaking and storytelling. Visit assetstore.unity3d.com/sundance to see a list of powerful cinematic assets to use on your next project.

15

unity.com/sff

closer aligned with your creative vision.

Real-time cinematography

Two PhD researchers at INRIA brought our cinematography technology into VR.

hand-off of artwork; a studio can shoot, edit, and color together simultaneously.

I’m humbled by what we’re already seeing created with these tools. We never tire of hearing users say they are finally able to spend more time on creative problems instead of technical ones. One of my goals when developing our cinematic technology was to make cinematography fun, fast, and iterative, so that you can create shot sequences quickly, change your mind, and rinse and repeat until landing on that

The future of storytelling is real-time

magic sequence that becomes your breakthrough cinematic moment.

cinematography technology into VR. In their

iterating and renders are out of date, making it

demo, you can ‘shoot’ your film with your actual

nearly impossible to keep up with the creative

hands—moving the dolly track and sliding

work. (I’ll also add that with Unity’s Recorder,

Have a wonderful festival, and we can’t wait to

the camera, framing the ‘actors’ who are to-

it’s possible to output 8K HDR files. That means

bring you even more revolutionary features in

scale, while you stand there (as seen above).

near-instant render times. Now you know why

Unity during the year!

It’s delightful, empowering, and yet another

it’s being called a real-time revolution!)

Happy shooting.

example of real-time filmmaking reshaping our understanding of what it means to direct, shoot,

The next steps of this transformation offer un-

and create.

foreseen levels of creative power and productiv-

Heading up Cinematics at Unity,

ity. The ability to create sets, shoot, experiment

Adam has almost 20 years of DP

Last thoughts (for now)

and discover, all while collaborating instantly

and CG work at EA, Pandemic

If you’ve worked in TV/film, you know how pain-

and even globally, is something that has

ful waiting for renders and post-processing can

never before been available. What’s more, this

has been DP on feature films. Cinemachine,

be—not to mention the fact that by the time you

workflow generates an unprecedented synergy

his innovative cinematic and in-game camera

review them, many artists have already been

among teams that are tired of the factory-line

system, was acquired by Unity.

16

and Blackbird Interactive, and

150,000 creators.

3,500 companies.

Join the community hub and talent marketplace dedicated to the full spectrum of Unity creators.

Visit connect.unity.com to seek help on your project, meet fellow creators, or check out the latest jobs and tasks from some of the

unity.com/sff

industry-leading companies like the ones below.

17

The future of storytelling is real-time

Creativi in our D 18

ty is NA

First and foremost, creativity is in our DNA – so naturally we are huge fans of the Sundance Film Festival.

Every year, we see so many powerful films and extraordinary VR/AR experiences pushing the boundaries of what’s possible in storytelling.

That’s why we on the Made with Unity team are delighted to celebrate so many imaginative creators, both at this year’s New Frontier and other projects on the horizon.

The following pages are a sampling of some of the stunning work we’ve seen that leverages art, technology, agency and character. What these pioneers have accomplished fills us with inspiration, and we hope you

unity.com/sff

feel the same!

19

20

The future of storytelling is real-time

Project

Trinity VR Studio UNLTD go-unlimited.co A live-action, immersive, interactive, virtual reality sci-fi head-trip. UNLTD has created a highly innovative VR experience using live-action characters, big-movie VFX, and the latest in interactive design. The studio is working with a new storytelling platform and an amazing new narrative visual language that’s revolutionary in the way we tell stories.

We’ll be talking with John Hamilton as part of our Storytellers of Tomorrow series, which you

unity.com/sff

can catch online shortly after the festival.

21

Project

The future of storytelling is real-time

Retour à Hairy Hill

Retour à Hairy Hill is a linear film with complementary immersive and interactive VR where users can perform puppetry in their environment.

Studio E*D Films

“The project is based on a true story of my grandmother and her family. Having grown up

edfilms.net

deep in the forests of Alberta, I have a life-long appreciation and respect of the land. When I finally pieced together the story of her childhood, I felt a visceral empathy for the experience I imagined, which captivated me,” says Daniel Gies of E*D Films.

22

Project

Coco VR is Pixar’s stunning debut in virtual

Coco VR

reality: an adventure into the beautiful Disney•Pixar film, Coco, released Nov. 2017.

Studios Magnopus, Walt Disney Studios, Pixar Animation Studios

experience, and follow the magical Alebrije into the luminous world of Coco, filled with lovable characters and beautiful settings from the film.

unity.com/sff

oculus.com/experiences/rift

You can choose either a single or multi-player

23

“Advertising in VR combines the storytelling power of immersive content with the guaranteed reach of a media buy.

We’re thrilled to have worked with Unity on this campaign, bringing Jigsaw into an engaging VR ad experience.”

— David Edwards, SVP of Digital Marketing at Lionsgate

You’re now entering the Virtual Room Looking at one of the most immersive brand experiences in history

I

n partnership with Lionsgate, Unity has

a traditional ad. Unlike a traditional ad however,

revolutionized advertising capabilities by

your content can transport them to new worlds.

The future of storytelling is real-time

creating the first-ever advertising platform for VR: the Virtual Room. Recognized as a standard

That’s exactly what VR users got last Halloween

by the Interactive Advertising Bureau (IAB), this

when Lionsgate and Unity debuted the Virtual

innovative experience is the latest addition to

Room for Jigsaw, the latest installment in the

Unity’s advertising platform, which serves

Saw franchise.

1.2 billion devices.

The difference What is a Virtual Room?

In order to measure emotional reactions, we

The Virtual Room is an immersive experience

partnered with Isobar’s Marketing Intelligence

that invites your audience to enjoy memorable

Practice. Using their Mindsight Technology for

interactions with your brand in ways not possible

understanding unconscious emotions, Isobar

before. Never forced, a user in VR can choose to

performed biometric measurement and emo-

engage with your content upon seeing it, just like

tional response surveys, then contrasted the

24

1—

Emotional Response Measurements VR Room experience

Zygomatic muscle

Mobile experience

activation, or smiling.

Muscle between shoulder and neck that tenses during moments of fear.

Galvanic Skin Response or palm-sweating.

Emotional Valence, or approach versus

2—

withdrawal.

The effects on someone going through the VR experience compared to the same user watching the mobile trailer.

3— difference between audiences experiencing the

than for other types of ads. This means that

movie trailer in VR versus mobile video.

you now have the ability to go beyond ‘telling’

The results were remarkable.

Those who saw the VR advertisement experienced substantially greater emotional arousal/ engagement, as indicated by the following biometric evaluations:

customers about your brand — you can invite them inside a world that will leave a unique, lasting impression.

As one of the world’s leading ad platforms,

• Elevated heart rate (+24%)

mobile, AR, 360, and traditional video ads —

• Heightened galvanic skin response

creating a holistic approach that helps ensure

• Sweating (+44% peaks/minute)

you achieve your marketing goals. The creative

• Increased muscle activation (more than 3X)

potential of advertising is boundless, and this is

associated with smiling

4—

Unity bundles these VR advertisements with

just the beginning.

5— someone going through the Virtual Room compared to the same user watching the mobile trailer.

Unsurprisingly, the emotional effect of experiencing a brand in VR is exponentially stronger

Agatha Bochenek leads the Mobile & VR/AR Advertising Sales team at Unity. She has built scalable, interactive advertising for several major publishers, including Zynga.

25

1. Users choose to enter the Virtual Room ad from within VR (in this example, from Nanite Fulcrum: Issue One). 2-4. Thrilling interactions in the world of Jigsaw.

5. Upon completion, the movie trailer plays in VR before users return to their previous VR experience.

unity.com/sff

The two graphs above show the effects on

Big ambitions.

Little studio.

Oscar-nominated director Neill Blomkamp opens up about experimenting with a new way of creating films: real-time

The future of storytelling is real-time

N

eill Blomkamp and his team at Oats Stu-

Isabelle Riva: What inspired you to tackle

dios used Unity to create parts two and

your first real-time film in Unity?

three of the ADAM series. They leveraged a re-

Neill Blomkamp: We have this film studio

al-time workflow and breakthrough techniques,

in Vancouver, which is a miniature machine

as well as photogrammetry, motion capture,

designed to do creative work and off-the-grid

and the latest animation workflows from Unity’s

experimental stuff. One of the main things

new artist toolset.

we wanted to do was cinema in real-time.

While we were contemplating which pieces

The following is an excerpt from a live, on-stage

we should do and how to go about imple-

Q&A between Blomkamp and Isabelle Riva

menting a real-time pipeline, this piece came

(Head of Made with Unity). Some content has

out that Unity had done [the original ADAM

been lightly edited for brevity and clarity.

demo from 2016, created by Unity’s Demo

26

*

This is not a photo.

It’s a digital asset of Neill created using photogrammetry (a technique that uses multiple photos of realworld objects to create high-quality, reusable, film-ready assets). The Oats team shot more than 30,000 photos of environments and props

unity.com/sff

for the films.

27

Big ambitions.

Little studio.

Because it’s essentially live — Team]. It got passed around the office, and a bunch of people sent it to me, and were like, “Have you seen this?” So it was a big

meaning someone could grab

deal. We really enjoyed it, and we also realized that it spoke to something we wanted to move towards.

IR: What is that, exactly?

NB: Basically we’re trying to build an infrastructure that has a very scaled-down version of an entire studio. From production to costumes, production design and visual effects, we wanted to implement the most filmic approach we could, but with real-time rendering.

the scene file and put the cameras wherever they want to — they could change the light, the time of day, they could change the wardrobe.

IR: Can you give an example of this approach?

It never stops living.

NB: Well, in our newest ADAM film, we wanted to try to create an environment that looked as real as we could possibly get it, for the actors to inhabit. We used photogrammetry to do that, by gathering 30,000 photos around an abandoned mine in California, which we rebuilt as a 3D

The future of storytelling is real-time

mesh. That data came in not just three-dimensionally, but also with RGB data, so you kind of have these textures built into the model. It looks incredibly real.

IR: How did this motion-capture process work in terms of facial capture?

NB: We didn’t want to do blend-shapes and morph-targets because there was no need for it

of high-fidelity facial capture that plays back

used to the more traditional way of doing things

like a flip-book. Basically I thought it would

found it quite amazing to be able to instantly

give us the highest fidelity in terms of muscles

move lights around and see the environment

beneath the skin and the way the eyes moved,

sort of paint itself right in front of their eyes.

wrinkles, stuff like that. So there isn’t any animation or deforming that’s happening to the face.

IR: After this experience, what do you think the future of film looks like?

IR: How did you and your team feel while work-

NB: I think that the merging of what’s possible

ing in real-time?

in interactive narrative experiences with that of

NB: A lot of the lighting artists at Oats who are

passive experiences [film] is going to happen

to be procedural. Instead, we wanted to gather a streaming set of individual frames — 30FPS

Learn more about the ADAM series at unity.com/madewith/adam

28

primarily because of younger generations

In the future it’ll be completely normal to watch

playing around with software like Unity. There’s

films produced in real-time.

going to be advances in storytelling where puts it back together in a way that… only time will show what that is. It’s essentially live — meaning someone can grab the scene and put the cameras wherever they want — change the light, the time of day, change the wardrobe. It never stops living.

the Oats team produced in real-time what would

As Head of Made with Unity,

normally take close

Isabelle supports rising stars

to a year using

from Unity’s talented community.

traditional rendering.

A former producer and creative director, she’s delivered TV, film, animation and VFX for several studios, including Ubisoft.

29

unity.com/sff

some kid in a basement rips it apart and

In just five months,

Codeless creation Sylvio Drouin, VP Research Labs, on the liberating future of content creation

I

n the game industry, development can be

That’s just the tip of the iceberg. We’re also ex-

synonymous with engineering — but we

perimenting with object recognition, advanced

believe and are working toward a future where

AI, and other types of machine learning.

content creation doesn’t require a single line of code. Film and VR/AR creators will soon be

Maybe you have a gymnast asset and, by set-

able to build and experiment without dependen-

ting up a few rules for her, you come back in the

cies on technical prowess.

morning to find that she has taught herself new flips and tricks without needing those specific

Our research teams at Unity are experimenting

animations developed.

with what content creation looks like three, five, even ten years from now.

This is the future, and our real-time platform

The future of storytelling is real-time

is the foundation of all of these possibilities.

Imagine ‘smart’ assets. If your scene has a

New forms of creativity will be unlocked (not

metal beam, which should rust over time, what

to mention accessibility and productivity) in a

if that beam asset rusted automatically over

way no one has ever seen before — possibly in

the course of your story? What if trees grew

a way no one has even imagined yet. If you’re

taller on their own? What if you communicated

a filmmaker, know that we are your machete in

verbally with your computer and told it that you

the wild, thrusting forward and clearing the path

wanted your car to change color, or get bigger,

for your creativity to shine.

or shift into a retro style — and it did so, without a line of code, until you were satisfied.

Here’s to the future.

Learn more about Unity’s experimental projects, research, and explorations at labs.unity.com

30

There is something very important going on with Unity and Hollywood that is going to impact media production dramatically in the coming years as game production techniques begin to supplant hundred-year-old narrative entertainment production techniques. This is going to deeply disrupt the film and television industry as we know it. Filmmakers, cinematographers, designers, and technicians have a great opportunity to harness this growing disruption. — Charlie Fink, Forbes

unity.com/sff

Dec. 11, 2017

31

The future of storytelling is real-time

unity.com/sff sundance2018@unity3d.com

32