The future of storytelling is real-time Real-time cinematography 3 things you should know
unity.com/sff
How VR and Virtual Rooms are reshaping brand advertising
Oscar-nominated Neill Blomkamp on a new way of creating films
Plus Sneak peeks of Made with Unity projects that are sure to inspire
The future of storytelling is real-time
With its imaginative storytelling, unique art style, and creative interactivity, the award-winning Monument Valley 2 by Ustwo Games is the latest installment of an indie puzzle game that has delighted audiences and critics worldwide. 2
Did you know?
interactive New Frontier projects are made with Unity
2/3 of all global AR/VR is made with Unity
90% of
The real-time revolution W
elcome to the 2018 Sundance Film
truly changing the way we live, work, and play,
Festival! As long-time supporters of in-
whether its training astronauts and surgeons,
dependent artists, we at Unity couldn’t be more
programming lights on a bridge, meeting your
thrilled to be here in Park City celebrating the
favorite superhero in AR, mastering a beautiful
pioneers of storytelling, including virtual and
puzzle game (page left), or shooting a CG film
augmented reality experiences (page 4) and
using photogrammetry.
bold new films created in groundbreaking ways. Unity’s core value is democratizing develop-
HoloLens projects
Maybe you’ve heard about the ‘real-time revolu-
ment. We believe there’s nothing more sacred
use Unity
tion’ happening in the industry (page 12), or you
for the artist community than the freedom and
know that technology advances in the game
ability to express themselves (page 8). Our
industry are now beginning to power the most
mission is to help you bring your visions to life
More than
efficient, collaborative, and creative workflows
and reach audiences who want to feel inspired,
12 billion
for film, TV, VFX, and CG animation (page
informed, and entertained by what you create.
Made with
26). Maybe, like us, you’re more excited and
Unity game
optimistic than ever about the future of these
We look forward to meeting you here in
installs
industries. Perhaps you’re not quite sure what
Park City. You can track our whereabouts and
in 2017
this shift means for you yet.
panels by following us on Twitter (@unity3d).
We assembled this short collection of tips, case
Happy creating!
100% of
studies, and interviews to introduce you to one
the top
of the largest creative communities in the world,
20 game
which we invite you to join (page 17). Millions
companies
of people around the globe use Unity to create
use Unity
immersive or interactive experiences that are
3 billion devices reached worldwide
P.S. Don’t miss our exclusive Sundance Film Festival Storytellers of Tomorrow interview series. We’ll be interviewing creators all week and posting them on unity.com/sff during the festival.
Cover: ADAM: The Mirror, a short film made in Unity by Neill Blomkamp (District 9, Elysium) and Oats Studios.
3
unity.com/sff
70% of
4
The future of storytelling is real-time
Celebrating New Frontier creators VR, AR, and the landscape of immersive storytelling
L
ast year I was blown away by New Frontier, the Sundance Film Festivals’s exhibition
that explores the convergence of film, media, and virtual and augmented reality. Since Unity’s technology powers the majority of the world’s “XR” (as many of us refer to it now), we are here to support and celebrate our talented creator community once again. First, let’s reflect on some of the projects that redefined our understanding of immersive storytelling, agency and narrative in 2017.
ASTEROIDS!, by Baobab Studios An interactive VR tale with feature-film quality art, this experience puts you at the center of the action in the form of a helper-bot to two quirky aliens, Mac and
and compassion, directed by Eric Darnell (Madagascar). Released in December.
5
unity.com/sff
Cheez (the latter voiced by Elizabeth Banks). It’s a story filled with humor, wonder,
Celebrating New Frontier creators Zero Days VR, by Scatter Weaving traditional media (video, audio, CG animation, charts) into a powerful VR presentation, this project will unnerve and humble you. Follow a computer virus on a clandestine mission to sabotage an underground Iranian nuclear facility in this story based on the Oscarshortlisted documentary.
Heroes, by Melissa Painter A dance performance unlike any other, this hybrid AR and 360 project is set to the iconic David Bowie song and takes audiences deep inside an athletic and intimate duet at the historic Ace Hotel in downtown L.A. Even your own body movements
The future of storytelling is real-time
are part of the experience.
Life of Us, by WITHIN A billion years in a few thrilling minutes—this piece from Chris Milk and team advanced our understanding of ‘presence’ in VR. As you journey through the story of evolution, while interacting with your multiplayer companion, positional tracking and voice modification surprise and delight you. 6
is growing. As a result, the way audiences consume entertainment will change, and practical business applications will fundamentally change
When the world sees these fundamental shifts,
as well. Fields like automotive, medical, and architecture will soon reimagine what it means to be efficient, safe, and accessible. Now it’s up to you But it all starts with artists like you. When the world sees these fundamental shifts, booms,
booms, and catalysts, it’s the storytellers who will plot the course we take.
and catalysts, it’s the storytellers who will plot the course we take. Technology advancements mean nothing without the artist sketching an idea at midnight and the writers crafting narrative tricks. It’s all about the content now, and that’s right where Unity lives. We solve hard problems under the hood, so creators like you and millions of others around the world can focus instead on the craft and joy of storytelling.
Since then
new breeds of art. Fidelity, abstraction, scale —
New platforms, breakthrough artists, design
these will be the stepping stones as we forge
best practices, you name it. I joke with my col-
ahead, moving us from traditional storytelling
leagues and friends that one year in the world
assumptions to literally creating whole new
of XR brings so many new changes that it feels
worlds. These worlds may begin as escape or
like a decade (it’s true). Apple and Google’s
entertainment, but they have the potential to
advancements with ARKit and ARCore, respec-
improve the quality of our lives.
We firmly believe the world is a better place with more creators in it. We look forward to another stunning New Frontier, another year of breakthroughs, and another stepping stone crossed on our way to finally understanding that the only real limit to what you can create is what you can imagine.
as a creator to reach hundreds of millions more
If I can geek out for a second: I want to call out
Tony Parisi is Head of VR/AR
people, and it’s only going to get better. We
XR’s impact on the computing landscape for
Strategy at Unity, the co-creator
believe that AR is going to be able to reach a
consumers and businesses alike. Investments
of 3D graphics standards, and
billion devices by the end of 2018.
are increasing, more industries are getting
Directors are using these mediums to create
the author of several O’Reilly
on board and reaping the benefits, and the
Media books on virtual reality and WebGL, such
quantity and level of involvement of consumers
as Learning Virtual Reality (2015).
7
unity.com/sff
tively, have exponentially expanded your ability
Bringing the ‘art’ to technical artistry Unity’s Yibing Jiang tells you why real-time is a game-changer
I
The protagonist
The future of storytelling is real-time
(right) of Windup, a cinematic art test in progress at Unity.
t’s no secret that real-time content creation
An industry in flux
and filmmaking are merging. Nowadays,
If you work in VFX, animation, or even live
games even feel like films—not just because
action TV or film, you already know that it’s
they have higher visual quality than before—but
not uncommon to experience anxiety about
also because the narratives often unfold with
the state of the industry. Due to foreign tax
very cinematic scenes. Having worked in both
breaks for the arts and a shifting ecosystem
film (Pixar, Disney) and gaming (Naughty Dog,
in general, many studios are moving abroad,
Ready at Dawn), it’s been life-changing for me
downsizing, or rethinking what kind of projects
to witness the growing demand in film for re-
they should pursue. The workforce in film (and
al-time technology. We’re all learning new ways
games) is used to a bit of job insecurity here
of storytelling together, as the two areas merge.
and there—which is no doubt why there are so many passionate people working so hard in
Yet there’s a myth that persists: if one thing is
those industries.
easier to create than another, it must be lower quality. In film, this myth manifests as the
Yet many of my peers are only now realizing
idea: if it doesn’t take a long time to render, it
what I’ve been saying for years: if you want to
must not be beautiful or complex. My standard
increase your relevance and value as a creator,
response to this is that if you want something
it is crucial to familiarize yourself with real-time
beautiful, remember that art school can be a
technology. The credibility for real-time has
more powerful resource than a render farm.
finally reached the point we knew it would; now
8
9
unity.com/sff
it’s a matter of educating creators on how they can develop the next wave of gorgeous stories. What makes a great technical artist? The biggest lesson I learned from working at Pixar (which can be very tech-focused) and Disney (which can be very art-focused) is that the foundation of technical artistry must be built with an equal balance of both those elements. If artists in production don’t have an appreciation for the pipeline and what’s under the hood, it could limit their potential for making magic with it. Conversely, it’s quite common for conversations in San Francisco (where I live and work) to lean too far towards the programming side. Each studio must find the right balance for itself. I personally believe that you have to be a great artist first. That’s fundamental. A lot of kids and students ask me what programming language they should learn, and I always reply, “Do you
With close-up shots like
want to be a tech artist because it’s easier to
The future of storytelling is real-time
this, Windup needed
get a job, or are you a great artist who is pas-
a high level of detail.
sionate about technology?” You can’t skip the
With fewer performance
basics. Great stories need great art. My advice
limitations than in
for companies putting teams together is not to
gaming, Yibing was able
forget the art part when looking for technical
to craft a realistic 3D
artists. That’s the key to a great story coming to
model for the leaves.
life before your eyes in a magical way. Another question I get often is how to elevate the level of quality. Image quality, for me, is largely how color is balanced and how unique 10
then worked on the sweater in real-time. The
This is the most I’ve ever felt like a true artist, in that this is the most
result? We can change the color and size of the weaves without waiting. This is the most I’ve ever felt like a true artist, in that this is the most authentic way to create art: draw, look at it, and draw some more.
authentic way to create art.
Because of technical limitations in traditional CG animation pipelines, artists usually don’t have the opportunity to look at what they’re creating in its final form. You’re part of a pipeline, basically guessing what the final image should
the design is. I also like to see people spending
highlights, which makes the result too noisy.
more time on animation and rigging. Graphical
Removing them makes the cloth materials look
quality is about balance, and it all starts with
much more refined. Unity has more accurate
concept art. If the original concept doesn’t
material and lighting responsiveness (very
have the right aesthetic style and balance,
close to offline rendering solutions), especially
then it’s no surprise that once you put the tech
for materials like stone, metal, and plastic. As
in, it won’t be pretty to look at. Audiences are
a result, we should see more and more visually
familiar with the uncanny valley by now, but
stunning stories that use real-time rendering in
that’s not the only risk. The best filmmaking en-
the very near future.
be, and hoping you’re making the right artistic decisions. Sometimes you’re even told to paint something and you’re not even sure why. Working on Windup, even though it’s a small team, we are doing everything together: painting, adding lights, balancing color, changing lenses — all at once. I can actually see what my art looks like in its final form, at every second, as I go. Directors and artists will love this,
learning from each other while working closely,
I joined Unity because I saw the opportunity of
which will serve both to provide that balance
real-time storytelling. I’m combining my expe-
and to optimize the pipeline.
rience from film and gaming to show people that you can get feature-film quality with these
because there’s no more waiting for overnight renders. This is what authentic artistry feels like, and it all comes down to freedom. As a technical artist, there’s nothing more powerful than that.
Why real-time storytelling feels more authentic
tools. For example, I’m working on a cinematic
I helped with the cloth, skin, and eyebrows for
art test at Unity now called Windup, in which
the recently released Neill Blomkamp short
the main character wears a fuzzy red sweater.
Yibing Jiang is the Technical Art
films, ADAM: The Mirror and ADAM: Episode 3.
On a big production, I would have had to render
Supervisor on the Made with
To improve the visual quality, we used some
for long periods of time to make things look
Unity team. Her past experience
cleaner habits while creating content. For ex-
right each time I made a change. At Unity, on
at Pixar, Disney, and Naughty Dog
ample, some textures on the cloth have unnec-
the other hand, we created the fabric details,
includes set creation, look development, and
essary information, like shadows and specular
baked them down as small textures first, and
character shading, respectively.
11
unity.com/sff
vironment should have artists and tech experts
Secrets of the real-time cinematography revolution The future of storytelling is real-time
Adam Myhill, Head of Cinematics at Unity, shares three facts you may not know
H
ere’s something to consider while you
It’s something I call “100 years of lens lan-
sip coffee and enjoy the view in Park
guage,” and those who know me also know I
City: why are video game genres named after
could talk about this all day. Because we’re here
camera lingo? First-person shooter, third-person
at the Sundance Film Festival, spreading the
strategy, isometric. As someone who’s worked
word about the real-time revolution that’s taking
on film sets, created trailers, and developed
the industry by storm, I want to share some fun
camera technology for both game developers
facts about cinematography — particularly what
and filmmakers, I see this as no accident. It’s
we have been refining over the past year.
one of many indicators that cameras, and their way of shaping points-of-view, aren’t just how
For those who haven’t yet used Unity to tell a
worlds are experienced—they’re also how we
story, my hope is that this inspires you and gets
experience the world.
your wheels turning. 12
Cinemachine automatically keeps the character in frame as the camera is moving across the dolly track.
Cameras can be smarter than you — and you won’t believe how much that it helps your project.
For example: if I tell Unity that I want this char-
If you’ve ever dreamed of having a top-notch
out of sight. I can also mark where in the frame
team of DPs and editors — the best in their
I want her specific body part to remain (say,
field, to execute your every whim — I can’t wait
the forehead), which ensures even after adding
to introduce you to procedural cinematography.
complex animations for her and nuanced
These ‘smart’ cameras can make decisions
movement for the camera, your target will stay
faster than it takes you to tell them what to do.
framed for you. I could even blend camera A
acter to be kept visible in frame, and she walks behind a wall, the system will automatically cut from camera A to camera B as soon as she is
13
unity.com/sff
1
Real-time cinematography a
b
movement into camera B, as if they are one fluid tracking shot, working together to capture the scene’s action and emotions fluidly, as one. c
Did I mention there’s an infinite amount of dolly track, as well as handheld noise simulators, shot evaluation, collision detection, and more? These are only a few simple examples of
Doing motion capture
how efficient this tech can be for you when
with actors while seeing
executing your aesthetic style, especially when
the virtual environment
you take into consideration that even a short film may have hundreds of different shots.
in Unity helped Neill and
a
the Oats team iterate
Procedural cameras can literally save you
quickly. Notice the actor
b
hundreds of hours of work, and that is precious
(a) and actress (b) using
time you can now spend on experimentation
props (c) which are
and iteration instead of setup.
digitally replaced.
c
2
Your live-action cinematography skills are transferable to real-time.
The future of storytelling is real-time
Not a week goes by that I don’t hear from someone with a physical cinematography
Rather than shooting motion capture (mo-cap)
The benefits of this workflow extend through
background — who has never used a real-time
and then adding the world behind the actor
every phase of development, like ‘location
engine before — how delighted and accom-
later, you can build out your world first, connect
scouting’ with a VR headset on inside an
plished they feel trying it for the first time.
the physical camera position to that of the digi-
artist-created 3D world. You can rapidly
tal one in Unity, and then shoot your actor while
experiment with pre-visualizations, and you
Virtual production is another example of this
seeing the CG world around them at the same
can also adjust the look-and-feel without
applicability. Let’s unpack this one. Filmmakers
time. This means you can find the best angles
waiting for post-production; there are famil-
of projects with partial or full CG environments
and get inspired from your environment in the
iar controls for lens emulation like bloom,
are now using Unity to shoot a CG film the
moment, on-set, just as you would in traditional
vignette, tone-mapping, motion blur, depth of
same way as a live-action set.
live action productions.
field, and more.
14
Filmmakers of projects with partial or full CG environments are now using Unity to shoot Because it’s digital, you don’t have to be in the
a CG film the same way as
same physical space either—crew can view the set and footage remotely in headsets and collaborate at a global scale. Compared to tra-
a live-action set.
ditional CG and VFX, this brings exponential increases in creative freedom, rapid prototyping, cost savings, live feedback and decision-making, asset reuse, and even potentially a better aesthetic look and feel to the project, which is
at your fingertips when you can tune camera,
animation studio. Innovative projects require
lighting, and post FX all at once, at any point in
innovative solutions, and designing new tools
The best part? Once you’re done, you can return
the process. This is the revolution of real-time
for precisely your needs is a must for pioneers.
to that world any time, like for sequels or ex-
filmmaking.
tended scenes, and you can reuse those assets
For those who don’t build their own tools, note
for other experiences, such as a companion
that many artists and programmers create and
AR app or game that will help extend the life of your IP. These once-futuristic workflows are becoming
3
offer new solutions on our Asset Store, which
You can build on top of our tools. Go wild.
has tens of thousands of available packages you can easily access to jump-start your production. Last year I met two PhD researchers at INRIA,
more mainstream, with dozens of projects currently in the making—including major blockbust-
The value of extensibility is hard to over-
the French Institute for Research in Computer
ers coming soon. Imagine the creative polish
state—I’ve heard this from almost every major
Science and Automation, who brought our
Ready to join the real-time revolution? The Unity Asset Store can help. It’s a massive catalog of over 40,000 free and paid tools and assets for creators and storytellers. You’ll find groundbreaking cinematic tools like Cinemachine and the Post-Processing Stack to help you create captivating content for the next frontier of filmmaking and storytelling. Visit assetstore.unity3d.com/sundance to see a list of powerful cinematic assets to use on your next project.
15
unity.com/sff
closer aligned with your creative vision.
Real-time cinematography
Two PhD researchers at INRIA brought our cinematography technology into VR.
hand-off of artwork; a studio can shoot, edit, and color together simultaneously. I’m humbled by what we’re already seeing created with these tools. We never tire of hearing users say they are finally able to spend more time on creative problems instead of technical ones. One of my goals when developing our cinematic technology was to make cinematography fun, fast, and iterative, so that you can create shot sequences quickly, change your mind, and rinse and repeat until landing on that
The future of storytelling is real-time
magic sequence that becomes your breakthrough cinematic moment.
cinematography technology into VR. In their
iterating and renders are out of date, making it
demo, you can ‘shoot’ your film with your actual
nearly impossible to keep up with the creative
hands—moving the dolly track and sliding
work. (I’ll also add that with Unity’s Recorder,
Have a wonderful festival, and we can’t wait to
the camera, framing the ‘actors’ who are to-
it’s possible to output 8K HDR files. That means
bring you even more revolutionary features in
scale, while you stand there (as seen above).
near-instant render times. Now you know why
Unity during the year!
It’s delightful, empowering, and yet another
it’s being called a real-time revolution!) Happy shooting.
example of real-time filmmaking reshaping our understanding of what it means to direct, shoot,
The next steps of this transformation offer un-
and create.
foreseen levels of creative power and productiv-
Heading up Cinematics at Unity,
ity. The ability to create sets, shoot, experiment
Adam has almost 20 years of DP
Last thoughts (for now)
and discover, all while collaborating instantly
and CG work at EA, Pandemic
If you’ve worked in TV/film, you know how pain-
and even globally, is something that has
ful waiting for renders and post-processing can
never before been available. What’s more, this
has been DP on feature films. Cinemachine,
be—not to mention the fact that by the time you
workflow generates an unprecedented synergy
his innovative cinematic and in-game camera
review them, many artists have already been
among teams that are tired of the factory-line
system, was acquired by Unity.
16
and Blackbird Interactive, and
150,000 creators. 3,500 companies. Join the community hub and talent marketplace dedicated to the full spectrum of Unity creators. Visit connect.unity.com to seek help on your project, meet fellow creators, or check out the latest jobs and tasks from some of the
unity.com/sff
industry-leading companies like the ones below.
17
The future of storytelling is real-time
Creativi in our D 18
ty is NA
First and foremost, creativity is in our DNA – so naturally we are huge fans of the Sundance Film Festival. Every year, we see so many powerful films and extraordinary VR/AR experiences pushing the boundaries of what’s possible in storytelling. That’s why we on the Made with Unity team are delighted to celebrate so many imaginative creators, both at this year’s New Frontier and other projects on the horizon. The following pages are a sampling of some of the stunning work we’ve seen that leverages art, technology, agency and character. What these pioneers have accomplished fills us with inspiration, and we hope you
unity.com/sff
feel the same!
19
20
The future of storytelling is real-time
Project
Trinity VR Studio UNLTD go-unlimited.co A live-action, immersive, interactive, virtual reality sci-fi head-trip. UNLTD has created a highly innovative VR experience using live-action characters, big-movie VFX, and the latest in interactive design. The studio is working with a new storytelling platform and an amazing new narrative visual language that’s revolutionary in the way we tell stories. We’ll be talking with John Hamilton as part of our Storytellers of Tomorrow series, which you
unity.com/sff
can catch online shortly after the festival.
21
Project
The future of storytelling is real-time
Retour à Hairy Hill
Retour à Hairy Hill is a linear film with complementary immersive and interactive VR where users can perform puppetry in their environment.
Studio E*D Films
“The project is based on a true story of my grandmother and her family. Having grown up
edfilms.net
deep in the forests of Alberta, I have a life-long appreciation and respect of the land. When I finally pieced together the story of her childhood, I felt a visceral empathy for the experience I imagined, which captivated me,” says Daniel Gies of E*D Films. 22
Project
Coco VR is Pixar’s stunning debut in virtual
Coco VR
reality: an adventure into the beautiful Disney•Pixar film, Coco, released Nov. 2017.
Studios Magnopus, Walt Disney Studios, Pixar Animation Studios
experience, and follow the magical Alebrije into the luminous world of Coco, filled with lovable characters and beautiful settings from the film.
unity.com/sff
oculus.com/experiences/rift
You can choose either a single or multi-player
23
“Advertising in VR combines the storytelling power of immersive content with the guaranteed reach of a media buy. We’re thrilled to have worked with Unity on this campaign, bringing Jigsaw into an engaging VR ad experience.” — David Edwards, SVP of Digital Marketing at Lionsgate
You’re now entering the Virtual Room Looking at one of the most immersive brand experiences in history
I
n partnership with Lionsgate, Unity has
a traditional ad. Unlike a traditional ad however,
revolutionized advertising capabilities by
your content can transport them to new worlds.
The future of storytelling is real-time
creating the first-ever advertising platform for VR: the Virtual Room. Recognized as a standard
That’s exactly what VR users got last Halloween
by the Interactive Advertising Bureau (IAB), this
when Lionsgate and Unity debuted the Virtual
innovative experience is the latest addition to
Room for Jigsaw, the latest installment in the
Unity’s advertising platform, which serves
Saw franchise.
1.2 billion devices. The difference What is a Virtual Room?
In order to measure emotional reactions, we
The Virtual Room is an immersive experience
partnered with Isobar’s Marketing Intelligence
that invites your audience to enjoy memorable
Practice. Using their Mindsight Technology for
interactions with your brand in ways not possible
understanding unconscious emotions, Isobar
before. Never forced, a user in VR can choose to
performed biometric measurement and emo-
engage with your content upon seeing it, just like
tional response surveys, then contrasted the
24
1—
Emotional Response Measurements VR Room experience
Zygomatic muscle
Mobile experience
activation, or smiling. Muscle between shoulder and neck that tenses during moments of fear. Galvanic Skin Response or palm-sweating. Emotional Valence, or approach versus
2—
withdrawal.
The effects on someone going through the VR experience compared to the same user watching the mobile trailer.
3— difference between audiences experiencing the
than for other types of ads. This means that
movie trailer in VR versus mobile video.
you now have the ability to go beyond ‘telling’
The results were remarkable. Those who saw the VR advertisement experienced substantially greater emotional arousal/ engagement, as indicated by the following biometric evaluations:
customers about your brand — you can invite them inside a world that will leave a unique, lasting impression. As one of the world’s leading ad platforms,
• Elevated heart rate (+24%)
mobile, AR, 360, and traditional video ads —
• Heightened galvanic skin response
creating a holistic approach that helps ensure
• Sweating (+44% peaks/minute)
you achieve your marketing goals. The creative
• Increased muscle activation (more than 3X)
potential of advertising is boundless, and this is
associated with smiling
4—
Unity bundles these VR advertisements with
just the beginning.
5— someone going through the Virtual Room compared to the same user watching the mobile trailer. Unsurprisingly, the emotional effect of experiencing a brand in VR is exponentially stronger
Agatha Bochenek leads the Mobile & VR/AR Advertising Sales team at Unity. She has built scalable, interactive advertising for several major publishers, including Zynga. 25
1. Users choose to enter the Virtual Room ad from within VR (in this example, from Nanite Fulcrum: Issue One). 2-4. Thrilling interactions in the world of Jigsaw. 5. Upon completion, the movie trailer plays in VR before users return to their previous VR experience.
unity.com/sff
The two graphs above show the effects on
Big ambitions. Little studio. Oscar-nominated director Neill Blomkamp opens up about experimenting with a new way of creating films: real-time
The future of storytelling is real-time
N
eill Blomkamp and his team at Oats Stu-
Isabelle Riva: What inspired you to tackle
dios used Unity to create parts two and
your first real-time film in Unity?
three of the ADAM series. They leveraged a re-
Neill Blomkamp: We have this film studio
al-time workflow and breakthrough techniques,
in Vancouver, which is a miniature machine
as well as photogrammetry, motion capture,
designed to do creative work and off-the-grid
and the latest animation workflows from Unity’s
experimental stuff. One of the main things
new artist toolset.
we wanted to do was cinema in real-time. While we were contemplating which pieces
The following is an excerpt from a live, on-stage
we should do and how to go about imple-
Q&A between Blomkamp and Isabelle Riva
menting a real-time pipeline, this piece came
(Head of Made with Unity). Some content has
out that Unity had done [the original ADAM
been lightly edited for brevity and clarity.
demo from 2016, created by Unity’s Demo
26
*
This is not a photo.
It’s a digital asset of Neill created using photogrammetry (a technique that uses multiple photos of realworld objects to create high-quality, reusable, film-ready assets). The Oats team shot more than 30,000 photos of environments and props
unity.com/sff
for the films.
27
Big ambitions. Little studio.
Because it’s essentially live — Team]. It got passed around the office, and a bunch of people sent it to me, and were like, “Have you seen this?” So it was a big
meaning someone could grab
deal. We really enjoyed it, and we also realized that it spoke to something we wanted to move towards. IR: What is that, exactly? NB: Basically we’re trying to build an infrastructure that has a very scaled-down version of an entire studio. From production to costumes, production design and visual effects, we wanted to implement the most filmic approach we could, but with real-time rendering.
the scene file and put the cameras wherever they want to — they could change the light, the time of day, they could change the wardrobe.
IR: Can you give an example of this approach?
It never stops living.
NB: Well, in our newest ADAM film, we wanted to try to create an environment that looked as real as we could possibly get it, for the actors to inhabit. We used photogrammetry to do that, by gathering 30,000 photos around an abandoned mine in California, which we rebuilt as a 3D
The future of storytelling is real-time
mesh. That data came in not just three-dimensionally, but also with RGB data, so you kind of have these textures built into the model. It looks incredibly real. IR: How did this motion-capture process work in terms of facial capture? NB: We didn’t want to do blend-shapes and morph-targets because there was no need for it
of high-fidelity facial capture that plays back
used to the more traditional way of doing things
like a flip-book. Basically I thought it would
found it quite amazing to be able to instantly
give us the highest fidelity in terms of muscles
move lights around and see the environment
beneath the skin and the way the eyes moved,
sort of paint itself right in front of their eyes.
wrinkles, stuff like that. So there isn’t any animation or deforming that’s happening to the face.
IR: After this experience, what do you think the future of film looks like?
IR: How did you and your team feel while work-
NB: I think that the merging of what’s possible
ing in real-time?
in interactive narrative experiences with that of
NB: A lot of the lighting artists at Oats who are
passive experiences [film] is going to happen
to be procedural. Instead, we wanted to gather a streaming set of individual frames — 30FPS
Learn more about the ADAM series at unity.com/madewith/adam
28
primarily because of younger generations
In the future it’ll be completely normal to watch
playing around with software like Unity. There’s
films produced in real-time.
going to be advances in storytelling where puts it back together in a way that… only time will show what that is. It’s essentially live — meaning someone can grab the scene and put the cameras wherever they want — change the light, the time of day, change the wardrobe. It never stops living.
the Oats team produced in real-time what would
As Head of Made with Unity,
normally take close
Isabelle supports rising stars
to a year using
from Unity’s talented community.
traditional rendering.
A former producer and creative director, she’s delivered TV, film, animation and VFX for several studios, including Ubisoft. 29
unity.com/sff
some kid in a basement rips it apart and
In just five months,
Codeless creation Sylvio Drouin, VP Research Labs, on the liberating future of content creation
I
n the game industry, development can be
That’s just the tip of the iceberg. We’re also ex-
synonymous with engineering — but we
perimenting with object recognition, advanced
believe and are working toward a future where
AI, and other types of machine learning.
content creation doesn’t require a single line of code. Film and VR/AR creators will soon be
Maybe you have a gymnast asset and, by set-
able to build and experiment without dependen-
ting up a few rules for her, you come back in the
cies on technical prowess.
morning to find that she has taught herself new flips and tricks without needing those specific
Our research teams at Unity are experimenting
animations developed.
with what content creation looks like three, five, even ten years from now.
This is the future, and our real-time platform
The future of storytelling is real-time
is the foundation of all of these possibilities. Imagine ‘smart’ assets. If your scene has a
New forms of creativity will be unlocked (not
metal beam, which should rust over time, what
to mention accessibility and productivity) in a
if that beam asset rusted automatically over
way no one has ever seen before — possibly in
the course of your story? What if trees grew
a way no one has even imagined yet. If you’re
taller on their own? What if you communicated
a filmmaker, know that we are your machete in
verbally with your computer and told it that you
the wild, thrusting forward and clearing the path
wanted your car to change color, or get bigger,
for your creativity to shine.
or shift into a retro style — and it did so, without a line of code, until you were satisfied.
Here’s to the future.
Learn more about Unity’s experimental projects, research, and explorations at labs.unity.com
30
There is something very important going on with Unity and Hollywood that is going to impact media production dramatically in the coming years as game production techniques begin to supplant hundred-year-old narrative entertainment production techniques. This is going to deeply disrupt the film and television industry as we know it. Filmmakers, cinematographers, designers, and technicians have a great opportunity to harness this growing disruption. — Charlie Fink, Forbes
unity.com/sff
Dec. 11, 2017
31
The future of storytelling is real-time
unity.com/sff sundance2018@unity3d.com
32