Interview: Baby X creator Mark Sagar

Mark Sagar is a pioneer in the creation of ‘digital people’. He talks to Helen Vause about the scope for such technology, his years in the movie world and enjoying life back in the place he grew up.

When Mark Sagar was a kid, tearing around the village and jumping off the Stanley Bay wharf, who would have thought he might bag a couple of Oscars in Hollywood one day?

Well, they were scientific and engineering Oscars, Sagar says modestly, not the sort the big movie stars get.

But they are coveted Academy Awards nonetheless, and the ceremonies he attended in Los Angeles to collect them were the flashest events he’d ever been to, even without the movie stars.

Sagar settles in to tell the Flagstaff how some big names in the backrooms of movie-making came to hear about him and the days he spent at the top of the animation game in Los Angeles.

While the movie story begins in Hollywood, the Oscars were won after he was back in New Zealand, working for the renowned Weta Workshop in Wellington on Peter Jackson productions, as supervisor of special effects.

His two Oscars were won in 2010 and in 2011 after he’d developed world-leading new technology for the creation of digital characters in blockbusters including Avatar, King Kong and Spiderman 2.

With his unique facial-rendering system, Sagar had moved the industry capability forward.

His fascination for the expressions on the human face and his passion for working to recreate that could have a bit to do with his parents, Neil and Nell Sagar, who brought him to Devonport from Kenya as a child.

He says his systems-analyst father had a talent for observing human quirks and was known for sharing colourful yarns “with everyone in Devonport”.

His mother was an artist who taught him how to draw the human face.

When Sagar first travelled as a young student, he’d earn money by sketching people wherever he went. “I have always loved people-watching.”Sagar started out

Future-facing... Mark Sagar drew on an artistic understanding along with technology to advance the creation of lifelike digital characters

in mechanical engineering studies at the University of Auckland.

His PhD research in the late 1990s was groundbreaking work, demonstrating how he could make a virtual human eye.

“The eye is the window to the brain and the interface with the world. It’s the hardest part of the face to make really believable,” he says.

From eyeballs, he moved on to creating faces, joining a laboratory group at the Massachusetts Institute of Technology (MIT). He drew on his artistic side and understanding of creating portraits, and dreamed of building virtual actors so believable they could fool an audience. His work, combining graphics, maths and physiology, revealed how highly realistic human features and expressions could be created for use on screens.

24 Hour Towing24 Hour Towing Devonport OwnedDevonport Owned and Operatedand Operated 1 Fleet Street, Devonport1 Fleet Street, Devonport Phone 445 0483Phone 445 0483 email: fleetst@ihug.co.nzemail: fleetst@ihug.co.nz email: office@fleetstpanel.co.nz www.fleetstpanel.co.nzwww.fleetstpanel.co.nz

Independent Financial Advice Individuals and Businesses

Call me fora no obligation chat email: david@davidsmart.co Phone: 027 543 4455 www.davidsmart.co

It added an extra layer of thinking to mathematical modelling, as art met science, with human psychology added for good measure. Things moved fast for Sagar when word of the exciting work in the MIT Laboratories caught the attention of entrepreneurs, who could see the huge potential of very lifelike artificial characters.

Soon, he had relocated to Hollywood and the world of the big-budget movies.

“They were crazy, exciting days full of the most incredible real individuals with amazing ideas. And then there were all the stars and the famous directors. It was the who’s who of Hollywood, and people were excited by what we were doing.”

Sagar and his team combined all the many tiny movements of the human face with animation techniques to produce a digital Jim Carrey, with a face pulling a whole range of complex expressions.

The project created a big buzz and “everyone in Hollywood” came to look at it.

But beyond the movie world there was big potential for the technology too, and Sagar these days continues to work at the leading edge of his field.

Since 2012, he has been the director of the Laboratory for Animate Technologies at the University of Auckland Bioengineering Institute. Her is also chief scientific officer of the company he co-founded in 2016, Soul Machines, which is headquartered in San Francisco but conducts its research and innovation in Auckland.

Soul Machines, which claims to have created “the world’s most astonishing digital people”, employs around 150 people, most of them in the United States.

At last count, more than US$130 million had been invested in the company’s work.

The potential of digital people as a workforce both on-site and on-screen has already been well established.

They are deployed in healthcare applications, such as their use by the World

Fast learner... Soul Machines’ lifelike ‘Baby X’

Health Organisation for Covid messaging, and in an alcohol-consumption programme where a virtual person runs Zoom calls with clients.

They are examples of digital ‘workers’ –being used in use in sectors from education and entertainment to finance, healthcare, retail and the public sector –that have been successfully programmed to interact and ‘talk’ in such a believable manner that they have become acceptable to large audiences – The digital assistant has huge potential to becoime part of our lives, says Sagar. The goal is to capture a sense of consciousness in them that is indistinguishable from the real human being.

Soul Machines started young, with Baby X, a virtual, artificially intelligent infant.

She is the very lifelike character they work with to explore human co-operation with machines. In a YouTube demonstration, Sagar sits chatting and observing the baby’s tears and discomfort as he takes his attention from her. But she’s seen soon smiling again after a few soothing words from him.

“Her behaviour is generated by neural networks running live. So she is seeing me and listening to me and responding to what I do.” Sagar says the leaders in the field of human co-operation with machines are working to enhance the feedback systems between people and digital assistants, making interactions faster and more relatable for the real participants.

But his team observes important ethical distinctions to guide the intentions of what they are creating.

“The aim is to make digital assistants that can help people and enhance the workforce.”

He says the concept of creating artificial friends, for example, could bring with it the possibility of dependency for real humans and that would not be ethically acceptable.

“The intention in this field is to create digital assistants for socially positive purposes.”

And for those yet to visually encounter a digital assistant on a Zoom call, Sagar points to Siri as a simple example of how life might be in the near future.

‘Asking Siri’ or any of her peers will become an increasingly sophisticated experience, as computer systems become more intuitive in their responses and therefore make Siri more ‘human’ to talk to.

Though Sagar is in the thick of this fast-evolving field, he also appreciates recreational activities back in the same environment he grew up in.

To relax on the weekends at home in Devonport, he swaps computers for swimming, paddle-boarding and appreciating nature.

Landscaping

in Devonport We specialise in all outdoor construction. Decks, paving, retaining, fences and more. We also handle all your design and planting needs. Call Steve on 021 345 694 steve@naturalgardens.co.nz www.naturalgardens.co.nz