Vr ar project state of industry paper 5 1 17 by Kevin Clark

IMMERSIVE MEDIA A STATE OF THE INDUSTRY M ay 2 0 1 7

VR/AR Project

VR/AR Project vrarproject.com contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

M ay 1 , 2 0 1 7

Rev 3 .0

Editor: Seth Shapiro Deputy Editors: Br yce Paul & Andriy Pishchalenko C o n t r i b u t o r s : Pe r i s a B r ow n , John Canning, Amy Dai, Xingting G u , H u d s o n L e i s e r, F r a n c i s c o Serna, Kinsey Thompson

V2 Seth Shapiro Andriy Pishchalenko V1 Seth Shapiro Andriy Pishchalenko contributions@vrarproject.com

2016 2015

VR/AR Project

New Amsterdam Media

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

T HI S I S A N OP E N C O M M U NI T Y PRO JEC T. T HI S I S A S N A P S H OT O F A R A P I D LY EVO LV IN G LA NDS C A P E. TO OP T IN FOR F U T U R E V ER S I O N S OF T HIS PA P E R, J O I N U S AT VRARPROJECT.COM EMAIL US AT CONTRIBUTIONS@VRARPROJECT.com

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

TABLE OF CONTENTS I 1. Preface 2. Overview  3. Enterprise Use Cases Advertising & Marketing Aerospace & Defense Construction Education Entertainment Fashion Finance Gaming Healthcare Live Events Real Estate Retail Training Travel & Hospitality 4. Computer Rendered VR 5. Live Action VR 6. Web VR 7. Social VR 8. Location-Based VR Theme parks VRcades 9. Smartphone VR/AR 10. VR Head Mounted Displays (HMDs) 11. AR/MR Head Mounted Displays (HMDs) 12. Spatial Audio 13. Haptics, Accessories, and Control Systems Omnidirectional Treadmills Exercise and fitness Haptic Suits Galvanic Vestibular Stimulation (GVS)

contributions@vrarproject.com

VR/AR Project

7 8 9 9 11 12 14 15 18 19 21 24 26 28 29 31 32 35 37 38 39 42 42 44 46 49 54 59 63 66 29 64 67

New Amsterdam Media

TABLE OF CONTENTS II Advanced movement tracking 14. Cameras and Capture Systems Custom Rigs The Manufactured Solution Proprietary Solutions DIY 15. Spotlight on China Government Support Leading Companies Electronics Companies Startups VR/AR Theme Parks 16. Investments 17. Notable Projects

66 68 68 68 72 73 74 74 75 77 78 79 80 93

INTERVIEWS 18. Grant Anderson, Jaunt (2016) 19. Gabo Arora, United Nations and LightShed (2017) 20. Willow Bay, USC Annenberg School for Communications and Journalism (2017) 21. Xing Chung, Mindmaze (2016) 22. Chris Edwards, Third Floor (2017) 23. Robert Hernandez, USC Annenberg School for Communications and Journalism (2017) 24. Aaron Koblin, Within (2016) 25. Yelena Rachistsky, Oculus (2017) 26. Ted Schilowitz, FOX Innovation Lab (2016) 27. Eric Shamlin, Secret Location (2017)

contributions@vrarproject.com

VR/AR Project

105 110 115 119 125 131 135 141 146 153

New Amsterdam Media

PREFACE

This project began in June of 2015, as a collaboration between my student Andriy Pishchalenko and I. We both wanted a reason to learn more about the rapidly-changing vir tual and augmented reality space, where I’d been spending more time, and had friends doing great work. Although the idea of vir tual reality had been around since the 1980s, this time there was a platform. The rise of mobile, the increase of inexpensive processor power, and the ubiquity of both web and social meant that there was a floor beneath the movement. It felt the same way the Internet had felt around 1992—before the browser, when believing in the transformative power of the Internet was still a minority view—a cottage industr y on the threshold of a tsunami. The best way to learn about something is to write about it, so Andriy and I dug in, and tried to summarize the industr y. Version 1 took about eight months; even then, the results were rudimentar y. We came back to it six months later, and released Version 2 in March of 2016. For this 3rd edition, Br yce Paul’s editorial contributions were invaluable, and Andriy continued to play a critical role. Perisa Brown, John Canning, Amy Dai, Xingting Gu, Hudson Leiser, Francisco Serna, and Kinsey Thompson provided great research that drastically expanded version three of the project.

Are you a con t e n t p ro d uc e r o r t e ch n o lo gy s o l u t i on s p rovid e r? To b e a d d e d to the d a t a b a s e , e m ail us a t con t r i bu t io ns @ v r a r p ro j e ct .c o m.

We would also like to credit Alice Shen and Michael Hoydich for the valuable community resource included in our Investments section. Many thanks to Casey Jones for his elegant design and layout work, and to De Ivett of 5D Spectrum for her web development work. Many thanks to the USC School of Cinematic Ar ts, my depar tment patrons Larr y Auerbach and Bonnie Chi, and our inter viewees: Grant Anderson, Gabo Arora, Willow Bay, Xing Chung, Chris Edwards, Rober t Hernandez, Aaron Koblin, Yelena Rachistsky, Ted Schilowitz, and Eric Shamlin.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

OVERVIEW

Vi r t u a l, a u g me n t e d , and mi xe d r eality r ep r esent the next gr eat er a of m e dia beyond T V and o nl i ne v id eo. Virtual reality refers to computer-generated experiences designed to mimic reality in an enclosed environment. A well-constructed VR environment embeds physiological cues to meet the human perceptual requirements for feeling real as opposed to traditional, 2D screen-based experiences. VR can be accessed from head-mounted displays (HMD) powered by a mobile device, computer, or game console. Unlike 3D, with which VR is often compared, the best VR experiences represent a new type of experience, in which the viewer is surrounded or immersed in an environment. While VR encloses the viewer in a different world, augmented reality opens and extends the user’s real environment. Whereas VR generally transpor ts viewers into an otherworldly experience, AR adds new digital layers onto one’s existing field of vision, enhancing the physical world with more information or choices.  AR and VR may be best understood as the two endpoints on the continuum of mixed reality. In a true mixed reality environment, physical and digital objects co-exist and interact in real time, and are anchored to a point in space. MR enables users to move around a digitally rendered object, just as if they were walking around a real physical object. In MR environments, physical and digital content co-exist and interact in real time.

While VR closes the viewer in a different world, augmented reality opens the world and extends the user’s real environment .

The massive implications of these three sectors are outlined below.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

ENTERPRISE USE CASES

T h is sec t io n p r e s e nts a n ove r v iew of som e of the m ost innovative co m p a n ie s i n the i r r e s p e c t i ve field s. I t highlights established b r and s tha t a re in c o r p o r ati n g V R a nd AR , as well as star tup s w ho ar e deve lo p in g t h i s e me r g e nt t e c h. Thi s i s by no m eans exhaus t i ve .

ADVERTISING & MARKETING

Companies are using VR and AR to extend brands— the best works focus on creating entirely new experiences. Effective VR generates visceral reactions because the audience is completely immersed in an environment, free from distraction. Studies show interactive and immersive ads outperform static pre-roll in completion, engagement, awareness, and brand recall rates. The strength of immersive adver tising lies in its intimacy. For instance, it is much more effective to offer a vir tual firstperson test drive than to watch someone else do it on TV. For example, Lexus, Volvo, Audi, and BMW are heavily invested in this form of marketing. VR/AR technologies also enable eye tracking. This information can be useful in constructing heat maps to fur ther refine, personalize, and target ads. Moreover, the same ad experience can be deployed in a variety of different environments: instore, in the home, in social media, and in B2B demos.

V i r t u a l S K Y r e p o r ts h i g h ly e n g age d a u d i e n ce s ; fo r the ir cu s t o m bu i l t 360 V R a d s , co m ple tio n r a t e s a re a t ab o ut 30-50%.

Marketers must select tactics that align with brand’s overall strategy, to achieve maximum ROI.

CreateVR

Sony Pictures teamed up with CreateVR to create a companion experience for their movie The Walk. In The Walk VR, the user relives Philippe Petit’s terrifying walk between the two World Trade Towers. When CreateVR originally demoed The Walk VR, they added haptic elements: a cut hose on the floor to simulate the wire, and a fan to simulate wind. As a result, at least 30% of viewers just could not bring themselves to walk more than a few feet.

MediaMonks

In a multi-million-dollar campaign, Etihad Airlines hired MediaMonks to showcase their new onboard, first class lounge contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

in VR. This lavish, high-budget ad with Nicole Kidman has the production value and sheen of a major motion picture. They have also created experiences for Google Tango, Audi, Toyota, Chr ysler, Samsung, and Lego.

Mssng Peces

Digital agency Mssng Peces leverages their video stor y telling exper tise, creating VR and 360° for brands including Chevrolet, NBA, and Dos Equis.

VirtualSKY

Airpush, a mobile adver tising leader that is integrated into over 150,000 mobile apps, recently announced a new platform called Vir tualSKY. Vir tualSKY repor ts highly engaged audiences: for their custom built 360 VR ads, completion rates are at about 30-50%. In comparison, Airpush typically sees 2-4% rates on their traditional mobile video ads.

Vertebrae

Ver tebrae is a VR adver tising network that connects brands with audiences in a similar fashion. Their platform helps content creators and publishers monetize by connecting them with consumers. Ver tebrae’s major publishing par tners include CBS, NFL, Facebook, Huffington Post, and Fox News.

VNTANA

VNTANA, a AR Hologram star tup, uses data-driven results to ser vice brand clients. They create hologram adver tisements that consumers can interact with. Data is then collected about

contributions@vrarproject.com

VR/AR Project

“The Walk VR” Create VR

New Amsterdam Media

how consumers reacted to and engaged with the experience. Additionally, VNTANA collects email addresses for brands, from potential customers.

AEROSPACE & DEFENSE

Aerospace & Defense are early adopters of VR/AR, as the incredible value of flight simulators became an early driver for VR R&D. While VR has been especially valuable for flight and combat training, AR is proving its value here as well. For example, soldiers in the field can view real-time information from a variety of sources (satellites, cameras, etc.) in a hands-free manner. A HUD information system enables them to navigate a combat situation or even civilian encounters. For pilots, the F-35’s $400,000 helmet is a major AR advancement. The display integrates angling, noise canceling, and night vision, so that the pilot is fully aware of the jet’s vitals, and surroundings.

Cubic Global Defense

Cubic Global Defense’s Immersive Vir tual Shipboard Environment (IVSE), is an advanced learning product for the Littoral Combat Ship training program. IVSE puts trainees in a photo-realistic 3-D vir tual environment that teaches tasks in settings vir tually identical to real-life scenarios.

Experius

VR/AR production studio Experius is creating advanced combat simulators. Their solution leverages volumetric capture from segment leader 8i, photogrammetr y, and 3D audio. Users can switch between a standard video game environment and an Experius photogrammetric environment, in order to see the stark difference between the two technologies.

contributions@vrarproject.com

VR/AR Project

“Destination Mars” NASA

New Amsterdam Media

Experius’ recent Navy SEAL Experience provides lifelike combat situations. In addition, users will also be able to stand eye to eye with a battle-worn Navy SEAL soldier, and walk around him to view him at any angle as he performs basic duties.

NASA

For over 60 years NASA has been training astronauts to fly, work, and live in conditions beyond the Ear th’s atmosphere. NASA uses VR/AR technologies to train astronauts to spacewalk via a solution called SAFER (Simplified Aid for EVA Rescue) with simulated jetpacks. Using a combination of an HMD, haptic feedback gloves, and motion trackers, NASA’s VR Lab gives the astronaut a vir tual experience of working outside the International Space Station. NASA has not stopped experimenting, and has added VR technology to train, educate, and inspire. Anybody can now experience astronaut training, a walk on Mars with a HoloLens, or see a rocket launch from a unique perspective.

CONSTRUCTION

VR enables companies to show clients a finished building before a single brick has been laid. The idea of vir tual construction sites is time honored: construction companies have long used laser scanning and rendering via building information modeling (BIM) and CAD. However, because drawings and renderings fall shor t in capturing the three-dimensional nature of buildings, designers would construct scaled down physical models. This is time consuming and costly. Now, companies and their clients can explore both the interior and exterior of a project, and make changes to the design plan before construction begins. Fur ther, VR’s social capabilities enable those involved in a project to meet in a vir tual jobsite and walk through it together in real-time.

VR enables companies to show clients a finished building before a single brick has been laid.

Simultaneous Localization and Mapping (SLAM) technologies are being integrated into AR, resulting in a reduction of time spent comparing blueprints to ongoing construction sites. Instead of using slow laser tripod scanners, SLAM systems use sensor data to track location and map the space around it, usually through LIDAR, which measures distance to an object using laser lights. The key advantages here are real-time rendering and non-dependency on GPS. Lastly, another issue that VR offers a remedy for—is dispute from local citizens who are worried about the effects that a construction site will have on the local environment. VR can be helpful in this regard by offering VR tours to locals, showing them contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

what building will look like, and how it will affect proper ty prices, landscape, traffic, safety, and collective health. Sur veys at the end can generate valuable feedback in early stages.

Gilbane Building Company

With the ability to identify future problems during construction, AR headsets simply pay for themselves. When constructing the Dearborn STEM Academy in Boston, Gilbane Building Company identified a serious issue beforehand: the steel frames they had on order were slightly too long to fit the design. On blueprints and 2D digital displays, the mistake slipped through the cracks. But when the company’s senior manager put on the HoloLens to see a mockup of the project, the mistake was obvious: “That one catch paid for the HoloLens,” he said. Among other construction and architecture companies who have announced experimentation with the HoloLens: the engineering

Designing with Microsoft HoloLens

firm AECOM, the design and architecture firm Gensler, and the China State Construction Engineering Corporation.

McCarthy Building Companies

One example of a successful and efficient usage of VR is the construction of the Mar tin Luther King Multi-Ser vice Ambulator y Care Center in LA. Prior to construction, doctors and nurses wore VR headsets, and were encouraged to provide input on logistical details of the rooms. As stated by the VP and CIO of McCar thy Building Companies, “even seemingly simple decisions like where the equipment connections are located on the wall behind the bed, where a trashcan is located, how wide the gaps are between beds, or what furniture goes can be extremely impor tant. In this project, we contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

saved a significant amount of time making these decisions early, and avoiding going in circles or installing something that would need to be changed later.”

Paracosm

Paracosm, a robotics company, plans to take 3D scanning even fur ther, using robotics and drones do full building scans.

E D U C AT I O N

VR has been used already in tens of thousands of classrooms, in progressive schools and learning labs throughout the U.S. and Europe. So far, VR usage has centered on the hard sciences, enriching interaction and focus on dimensional objects: human bodies, animals, molecules, environments, and architectural models. However, educators are also finding VR to be beneficial in the realm of vir tual field trips.

CGO Studios

CGO Studios is a vir tual reality production company focusing on creating high-quality, photo-realistic moments of histor y in VR. The team’s initial experience, appropriately titled FIRST, was the first-ever historically curated recreation in VR. This piece showcases the Wright Brothers epic achievement of flight, and incorporates curation from the leading exper ts on the subject at the Smithsonian National Air and Space Museum. CGO’s current projects include an Anne Frank experience (with curation from the Anne Frank Foundation in Basel) and a recreation of the first summiting of Mt. Everest by Sir Edmund Hillar y and Tenzing Norgay, called Everest ’53.

contributions@vrarproject.com

VR/AR Project

Google Expeditions

New Amsterdam Media

Google Expeditions

VR can give students a glimpse into the past and/or other places in the world. To expand on this, Google launched Expeditions, accessible via smar tphone. This project gives schools the materials they need to take students on a VR field trip.

Nearpod

In a similar vein, Nearpod is creating pre-made lessons based on VR field trips. There are over 4,000 ready-to-teach interactive lessons for all K-12 grades and subjects—from, U.S. histor y and digital citizenship to algebra and science— created by Nearpods’ community of educators and par tners. These lessons are available in the proprietar y Nearpod Librar y. Moreover, Nearpod is already in use in 10,000, or 10% of, U.S. schools. They are also working to bridge the gap for the largest growing student demographic: non-native English speakers. In one of the largest English Language Learner districts in the countr y, Nearpod provides teachers with more than 500 lessons designed for these students. for this group of students

YouVisit

YouVisit is a vir tual reality touring business with several applications. Their platform allows prospective students to vir tually tour the campuses of collaborating universities. The ser vice claims to result in increased physical campus visits, applications, and yield rates. Har vard, Columbia, Princeton, and Yale are among par ticipating institutions.

ENTERTAINMENT

Enter tainment is clearly the most widely discussed sector of VR/AR. Content is produced for and distributed across a range of form factors, including 360 video, mobile, and HMD/room scale. The content already occupies a wide range of genres, including games, fiction, fantasy, journalism, politics, and travel. Littlstar, Jaunt, NextVR, Transpor tVR by Wevr, im360Vr, Digital Domain, Within, SteamVR, Vivepor t, Oculus Store, Daydream, and YouTube360 are some of the most prominent platforms. INVASION! Baobab Studios

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

The following is a small sample of leading content companies, and some well-known projects. A fuller list can be found in section 17. Notable Projects.

Baobab

Baobab is a VR studio that creates cinematic and character driven narratives. The company was founded by Maureen Fan, former VP of games at Zynga, and Eric Darnell, the director of many DreamWorks animated features including the Madagascar franchise. Baobab’s cinematic experiences include Rainbow Crow, Asteroids!, and Invasion!.

Felix and Paul

Felix and Paul Studios focuses on creating high-end cinematic vir tual reality experiences. Founded by Emmy-award winning directors Felix Lajeunesse and Paul Raphael, the studio creates original stories, including Strangers and the series Nomads. The studio collaborates with franchises and famous personalities, including Jurassic Park, LeBron James, and President Barack Obama. Their signature emotive cinematic experiences are made possible by the company’s proprietar y 360 camera rig and accompanying software.

Google Spotlight Stories

Google created Spotlight Stories to develop experimental shor t films and animations in vir tual reality. The 360 videos are distributed via the Spotlight Stories app on both iOS and Android. Currently, all the videos are free. Their landmark film Pearl was nominated for an Academy Award for Best Shor t Film (Animated).

contributions@vrarproject.com

VR/AR Project

The Martian VR Experience Virtual Reality Company & 20th Century Fox

New Amsterdam Media

Here be Dragons/Within

Chris Milk, (with Aaron Koblin) co-founded production company Here Be Dragons and its sister company, Within. Both focus on live-action and animated VR content. The content ranges from documentaries to music videos, including many award winners. Content from both companies is distributed via Within’s iOS and Android app.

Jaunt

Jaunt develops both hardware and software to enable cinematic VR. The Jaunt One, Jaunt’s premier camera, allows for high quality 360-degree capture coupled with 3D sound-field microphones. At their Los Angeles based production studio, Jaunt creates both live action and animated content, and distributes through its app. In September 2015, Jaunt received a major investment from Disney.

Oculus Story Studio

Oculus Stor y Studio is a vir tual reality film studio that creates original animated experiences. The company was founded in 2014 by Edward Saatchi and Pixar veterans Saschka Unseld and Max Planck. The company is best known for its interactive VR shor t, Henr y, which won an Emmy for Original Interactive Program. The studio also developed Quill, an “infinitely scalable VR canvas” where users can sketch and paint “immersive worlds and tell stories in an entirely new way.”

Penrose Studios

Penrose Studios is focused on both augmented reality and vir tual reality stor ytelling. The studio wants to be “the Pixar of VR/AR”, creating massive cinematic experiences. Founder Eugene Chung put together an ace team with employees from DreamWorks, Oculus, Disney, and Google. Their most acclaimed work is Allumette, the hear twarming stor y of a girl, her mother, and a city in the clouds.

contributions@vrarproject.com

VR/AR Project

theBlu: Whale Encounter Wevr

New Amsterdam Media

Secret Location

Secret Location is an award-winning studio that creates content across the web, mobile, tablet, VR, and AR. The studio developed Sleepy Hollow: VR Experience, the first VR experience to win an Emmy. Secret Location has worked for many clients including Red Bull, Samsung, Mitsubishi, Nickelodeon, MGM, Endemol, Sony Pictures, and Toyota.

Virtual Reality Company

The Vir tual Reality Company is a content studio that focuses on VR experiences by combining “technology, ar t, and stor ytelling”. The Vir tual Reality Company is founded by an all-star team and focuses on episodic and long-form content. Advisors to the company include Steven Spielberg and James Howard. Look out for the company’s VR experience, Raising a Ruckus, later this year.

Wevr

Wevr is both a content creation studio and a distribution platform for live action and animated VR content. Wevr’s most notable VR experiences include theBlu: Whale Encounter, which premiered at the 2016 Sundance Film Festival, and Gnomes and Goblins, in conjunction with John Favreau. Recently, Wevr created Transpor tVR, an independent network for VR creators to share their work.

FASHION

In fashion, augmented reality is emerging as a new way for brands to delight and engage customers. Many retailers are developing AR apps, allowing customers to tr y on vir tual items before they buy. The reach doesn’t stop at traditional fashion brands— even niche markets within retail are jumping on the oppor tunity to deliver unique shopping experiences.

Perfect Corp.

The YouCam app is created by the app developer Perfect Corp., which has a range of mobile beauty apps including YouCam Makeup, YouCam Perfect, YouCam Nails, Beauty Circle, and YouCam Fun. The apps have collectively generated over 300 million unique downloads globally, establishing Perfect Corp. as a leading figure in the AR beauty movement. Through its suite of mobile apps, the company connects brands to consumers across the entire retail experience, from product discover and vir tual tr y on, to social sharing and mobile commerce. Since its launch in March 2014, users have taken a combined 10 billion photos across the apps, and spend over 500 million minutes per month taking photos with the camera feature.

contributions@vrarproject.com

VR/AR Project

“Sephora Virtual Artist” Sephora

New Amsterdam Media

Gap

Gap unveiled an AR app, “The Dressing Room by Gap,” which allows users to visualize Gap products on a vir tual mannequin. Gil Krakowsky, VP Global Strategy and Business Development at Gap, explained in an inter view: “This is how it works: shoppers choose a Gap style that they might be interested in purchasing. Next, they select one of five body types featured in the app so they can “tr y on” the piece of clothing from anywhere on a Google Tango-enabled device, and if they love it, they can buy it online.”

Sephora

Cosmetics company Sephora introduced the “Sephora Vir tual Ar tist” in an update to its existing iOS app. Developed in a par tnership with AR company ModiFace, the feature allows users to tr y on makeup looks through facial scanning. Once a user has found a look they like, they can purchase the products used within the app itself. The app also features “vir tual tutorials” which overlay instructions of how to apply the products on the user’s face. Fur ther, product review and tutorial videos account for a large share of YouTube’s popular content, suggesting that Sephora’s style of vir tual tutorials are strongly positioned to become a leading AR genre.

FINANCE

VR and AR “represent new channels for financial ser vices providers—wealth managers, brokerage firms, and personal bankers—to deliver financial content in a novel and easily consumable visual format.” Citi Holographic Workstation 8Ninths

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

Traditional financial institutions are losing the trust of their core, wage-earning customer base. At 84 million, millennials are the largest demographic in American histor y. According to the Millennial Disruption Index, a three-year study that sur veyed 10,000 millennials about 73 companies in 15 industries, the banking sector was found to be at the highest risk of disruption. In par ticular, millennials have fundamental dissatisfactions with the retail-banking customer experience. Whether it’s sending a wire, applying for a loan, or signing up for a credit card, millennials are inclined to do it all online, avoiding brick-and-mor tar banks at all costs.

According to 8ninths, these were the pain points addressed: lack of prioritization within six to eight screens of 2D information, lack of easily discernible However, most will eventually need some type of professional financial advice for complex dealings, such as home mor tgages centralized and trusts. To address this, customers could visit vir tual reality knowledge, branches from inside of their home. Vir tual branches have huge inefficiency in capacity to revolutionize customer relations by offering a new navigating between ser vice for clearer and enhanced communications. windows and tabs, Just as online banking enabled customers to use live chat and email inefficiency in to contact bankers, vir tual reality branches will give customers recognizing critical a new way to interact. Ultimately, banking will be easier, more engaging, and more efficient for each par ty. Interactions would patterns and market be avatar to avatar, whereby bankers can present offers, advice, changes, loss of simulations, spreadsheets, graphs, char ts, and other examples in oppor tunity for real-time. collaboration and Comarch dialogue, and loss Comarch uses VR in their wealth management software to make of the “human trading tools easier to use and more accessible. Being able to element” and the visualize data and potential outcomes helps their customers make better investment decisions, rather than looking at spreadsheets. “feel of what is going on in the Citi market.” 8ninths has created holographic workstations for Citi’s stock market traders, utilized through Microsoft HoloLens. According to 8ninths, the pain points addressed include lack of prioritization within six to eight screens of 2D information, lack of easily discernible centralized knowledge, inefficiency in navigating between windows and tabs, inefficiency in recognizing critical patterns and market changes, loss of oppor tunity for collaboration and dialogue, and loss of the “human element” and the “feel of what is going on in the market.”

Fidelity Investments

Fidelity Investments ideated StockCity for Oculus Rift (also viewable via browser), a tool that applies vir tual reality and data visualization to help investors manage their finances. It turns any contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

given por tfolio into a vir tual city, in which each stock is represented by a building. The height and footprint represent the price, trading volume, and outstanding shares. This is just one way in which Fidelity is attempting to connect with younger generations. “If you want to deliver a great customer experience, you have to be in the same kinds of environments where they are,” says Seth Brooks, Director of Product Management at Fidelity Labs.

GAMING

Games are the low hanging fruit in VR enter tainment. They have been the enter tainment industr y’s main driver, holding the largest growth and revenue promise. Game worlds have always been about transpor ting the player to different worlds and environments. VR provides a natural extension of the worlds gamers have enjoyed exploring. On the production side, the most robust VR environments are created with game engines. Both Oculus and Vive have their base in the gaming community. All one has to do is look at the launch and acceptance of the PSVR to see the dominance: in 2016, the PSVR sold more units than Oculus and Vive combined.  All three platforms are expanding their content offerings beyond the core gaming categor y. Some of the leading game companies include the following.

Capcom

Capcom is a Japanese game publisher that has been publishing games since the ‘80s, with titles including Street Fighter, Devil May Cr y, and Resident Evil. The latest release of the Resident Evil franchise is Resident Evil 7. Released on PSVR in 2017, the game received a lot of buzz and positive reviews. Many players claimed that they couldn’t complete the initial chapters because of how intensely frightening it was. Audiences can feel trapped inside the headsets while seeing through the eyes of the protagonist. This reaction is something that even the best movies haven’t been able to elicit, and bodes well for the development of horror VR projects going forward.

Game worlds have always been about transpor ting the player to different worlds and environments. VR provides a natural extension of the worlds gamers have enjoyed exploring.

CCP

Icelandic publisher CCP created EVE Online, a highly successful space based MMORPG (Massively Multiplayer Online Role Playing Game). The company then developed two new titles based on the same IP—Gunjack, a vir tual arcade shooter for Samsung Gear VR (and subsequently Vive and Oculus) and a more sophisticated VR experience, EVE: Valkyrie. EVE: Valkyrie is a multi-player dogfighting space game that is achieving mainstream popularity. Despite initially mixed critic reviews, consistent free updates and contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

“EVE:Valkyrie” CCP Games

cross-platform suppor t have spurred sales and enthusiastic audience response, recently breaking even on their $30m investment into the space.

Crytek

The German video game and software developer Cr ytek is known for games like Far Cr y and Cr ysis, which both showcase the companies powerful Cr yEngine. The Climb is a popular Oculus exclusive featuring VR climbing. This game shows off the power of VR to transpor t users into hyper real environments, POV perspective, and adrenalin pumping action. Subsequently, Cr ytek has released Robinson which is a sci-fi adventure. Both games highlight the power of Cr ytek’s game engine for developing VR games.

Epic Games

Epic Games is widely known for their Unreal game titles, and even more so for their Unreal engine, which powers many CGI based VR experiences. Robo Recall, an Oculus exclusive, is an arcade-style shooter that has players dodging and returning gunfire while taking advantage of the powers of telepor tation to navigate the universe. The game offers simple yet addicting gameplay and competes with similar titles like Arizona Sunshine. Robo Recall is the evolution of one of Epic Games’ earlier tech demos for the Rift called Bullet Train, which was one of the first to take advantage of the newly designed Oculus Touch controllers.

NUX Studios

This Swedish based game company was established in 2016 with a core team focused on developing the VR and AR space. In their first game, Wands, players use one of several wands, each with its unique spells, and duel other wizards online. In between battles, users can develop their skills and explore new spells and combinations. The more battles one wins, the more spells and abilities are unlocked. There is also a spectator feature to watch others battle in real time. It is available on Google Daydream, Samsung Gear, PSVR, and Vive.

Owlchemy Labs

Owlchemy was bought by Google in May 2017. The studio is best known for their breakout VR hit, Job Simulator : The 2050 Archives. This car toon-style game consists of several “job simulations,” contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

which stick players in various work environments such as a garage, restaurant kitchen, or a corporate office. They can interact with different objects found in the space, as well as complete tasks such as fixing an engine or preparing food. Players have the freedom to accomplish those tasks in a wacky or unor thodox way if desired. Actions are performed via the hand controllers with relative ease. Job Simulator was well received for its uniquely fun take on an interactive VR experience, and managed to make over $3M in sales in roughly half a year. It also spawned a spin-off game based on the Rick & Mor ty TV show called the Rick and Mor ty Simulator : Vir tual Rick-ality. This game puts players in the shoes of the character Mor ty, and allows them to explore and interact with crazy objects and characters from the Rick and Mor ty TV universe.

Survios

Sur vios was established in 2013 by a group of USC students with the focus on VR games. Their first game release, Raw Data, is one of the most polished titles on the market, and the first consumer VR game to reach $1M in sales. The multiplayer co-op shooter effectively takes advantage of the room-scale capabilities of the HTC Vive to give the player freedom to perform evasive maneuvers while blowing up robots. The game illustrates the potential of multiplayer VR gaming and the potential for E-spor ts to grow out of competitive social VR experiences.

contributions@vrarproject.com

VR/AR Project

â&#x20AC;&#x153;Job Simulator: The 2050 Archivesâ&#x20AC;? Owlchemy Labs

New Amsterdam Media

Valve Corporation

Valve Corp’s content distribution platform Steam is the largest digital game distribution platform for PCs, reaching 14 million concurrent users this Januar y. In 2015, they par tnered with HTC to create the Vive platform, and established the Vive as the leading VR gaming platform. Valve has now expanded their par tnership to other HMDs star ting with LG.

HEALTHCARE

Healthcare is currently one of the most fer tile and rapidly growing areas of VR and AR. Use cases include surger y assistance, remote diagnostics, surger y simulation, phobia treatment, phantom limb pain (PLP) treatment, robotic surger y, data visualization, and skills training.

AppliedVR

AppliedVR is focused on bringing therapeutic practices to hospitals. Their mission is to offer “an enjoyable escape from scar y and painful experiences in healthcare.” Their platform offers games, vir tual travel, music visualization, nature mediations, and spor ts. AppliedVR expects their technology to reduce anxiety in pre-op waiting rooms; increase the comfor t of patients who are receiving an IV, local anesthesia or regional block; and reduce pain, anxiety, and discomfor t over the course of cer tain procedures as well as during post-op periods. Overall, they aim to increase patient satisfaction. AppliedVR cites that games reduce general clinical anxiety at a 63% rate, and found that patients repor ted reduced pain by 24% after experiencing VR for 20 minutes.

BioFlightVR

BioFlightVR offers a wide range of medical VR and AR medical training ser vices. One of BioFlightVR’s main applications is

contributions@vrarproject.com

VR/AR Project

Patients using Applied VR therapy

New Amsterdam Media

the uploading of data in VR to increase accuracy of diagnoses; their platform allows doctors to view CT scans and MRI data as interactive, immersive, 3D images. These imaging tools reveal details that are often overlooked. Another one of their main applications is the creation of VR experiences which take users through organs in the body. This helps patients understand their illness or health issues. This kind of experience stimulates a lasting, positive change. For instance, the company created an experience designed for cigarette smokers tr ying to quit, taking users through the mouth, lungs, hear t and other organs to see firsthand the damage done by smoking.

ImmersiveTouch

ImmersiveTouch is a surgical vir tual reality company that trains and improves the precision of surgeons for minimally-invasive procedures, and is FDA approved. It is used as an education platform for pre-surgical planning, and mission rehearsal for several types of procedures. The company cites that it reduces surgical errors by about 54%.

One Caring Team (Aloha VR)

Aloha VR is focused on assisting the elderly community through VR. There are no puzzles, objectives, or stor ylines. Aloha VR offers several beautiful and soothing scenesâ&#x20AC;&#x201D;a simple escape from reality for those who have difficulty leaving home.

OssoVR

OssoVR is a platform designed for surgical training, par ticularly in the or thopedic market, with plans to expand into other specialties and procedures. OssoVR is equipped with advanced hand-tracking on the Oculus Rift of HTC Vive. Ultimately, OssoVR is an affordable alternative to surger y simulators that can cost up to $250k.

contributions@vrarproject.com

VR/AR Project

Doctors using the Oculus Rift to view 3D images of the body Surgical Theater

New Amsterdam Media

Scopis

Scopis is a next-generation Target-Guided Surger y System that overlays digital images onto a surgeon’s endoscope, which informs them of the mapped-out pathway.

Touch Surgery

Touch Surger y already boasts a community of 1.5 million users of its mobile surgical simulations, and recently formed a par tnership with DAQRI to add compatibility with their hardware. This par tnership has great potential as it is fundamental that a surgeon has as much information as possible during a procedure; if a doctor is wearing an AR headset, he could navigate several bits of vital information about his patient without having to move or look away. When combined with other sensing technology, AR can greatly enhance the surgeon’s perception of the procedure by overlaying real time info.

Virtually Better

Vir tually Better’s mission it to treat common fears such as heights, storms, public speaking, elevators, bridges and flying. They offer programs to help clinicians treat a specific phobia using different scenarios. Each program gradually exposes the user to a phobia, as a supplement to graduated exposure. Such a controlled environment allows practitioners to expose their patients to a gradient of simulations and coach them on how to cope with their feelings. By putting patients through the memories over and over, the patient overcomes the memor y as repetition and expectation reduce their impression of the situation.

LIVE EVENTS

Livestreaming in 360 is now possible over Facebook, YouTube and Periscope, for owners of consumer 360/VR cameras. Samsung has led this effor t for their Samsung Gear camera, featuring famous YouTubers like Casey Neistat in commercials, and having them create 360 content. Samsung’s hope for this campaign is that it will bolster interest in creating 360 content (using Samsung’s camera). New consumer 360 cameras releasing in 2017 will have relatively simple workflows with automatic stitching and sleeker post-production editing tools.

New consumer 360 cameras releasing in 2017 will have relatively simple workflows with automatic stitching and sleekerpostproduction editing tools.

Most custom professional cameras like the ones used by NextVR and Voke have their own unique post-production software for stitching and editing, and are more resource heavy as they deal with a greater number of lenses shooting in higher resolutions. Other companies offering professional systems that are capable of livestreaming include Nokia’s Ozo, 360 Rize Black Magic setup, contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

Liveplanet and Vuze. The simplest “pro-sumer” 360 camera is the Orah 4i with a form factor similar to consumer cameras, utilizing only four lenses and shooting in 4K.

IM360

IM360 is Digital Domain’s VR branch. They specialize in live 360 streamed experiences, and offer a full proprietar y end to end solution.

LiveLike

LiveLike takes a different approach to viewing “in VR.” They utilize traditional cameras with wide angle lenses to capture gameplay, and then stream the gameplay in a vir tual viewing box through a headset, where multiple friends can join and pretend they’re sitting in a suite at the actual stadium. The experience allows for multiple viewing angles and social interaction between viewers, and has been received good reviews via the Fox Spor ts VR mobile app. In the past year LiveLike has streamed multiple NCAA football and basketball games for early adopters, and plans to stream the Roland Garros tennis championship this summer.

Livit

Livit is a 360 streaming par tner that works with a variety of live camera solutions and stream content via a white label player. Their player can be embedded into either web or mobile app solutions.

NextVR

NextVR has closed the most deals in the space so far, teaming up with FOX Spor ts, NBA, NCAA, NHL, ICC Soccer, NBC Spor ts, US OPEN Tennis, and LiveNation to stream live games and concer ts in VR. These livestreams will be made available through the NextVR mobile app on iOS and Google Play. They currently offer compatibility with NBA League Pass, providing access to 3-4 livestreamed games a month. Their goal is to grow their subscription ser vice and offer them through larger channels in the future. Their main goal is to scale, but one bottleneck persists: current smar tphones and consumer HMDs can’t stream at high resolution, due to their relatively low res displays and the bottleneck of current 4G data networks. However, in the next year and going forward, 4K resolutions and higher will become the standard, and by 2021 5G coverage should be gaining traction.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

Pixvana

Encoding also plays a major role in streaming content through the data pipeline. Pixvana built their own processing pipeline to improve encoding and editing speeds for high resolution 360 content. Improvements in display resolution, data network speeds, and encoding will all significantly improve the quality of live action and livestreaming 360/VR content in the future.

Voke

Voke was acquired by Intel for their “Immersive Spor ts” offering. Among several other projects, Voke has streamed professional and college spor ts games, the Critics’ Choice Awards, and the “Project Runway” finale.

VRLive

VRLive is a 360/VR production entity focused on live events (music, events, spor ts, etc.). They use a wide variety of gear, based on the project and output to clients’ desired platforms.

REAL ESTATE

The main use case for VR in real estate is in offering potential home buyers fully immersive vir tual tours. These vir tual tours are extremely effective in preselling spaces, and giving consumers a true feel for the space.

Macerich

Macerich used VR to showcase some of its early stage projects, including its 500,000-square-foot luxur y Fashion Outlets of San Francisco near Candlestick Park.

contributions@vrarproject.com

VR/AR Project

Matterport camera 3D scanning

New Amsterdam Media

Matterport

Currently, Matterpor t is leading the way with volumetric 3D scans of entire homes. Matterpor t’s 3D scans can be viewed on a desktop or thru an HMD. According to Matterpor t, these vir tual tours generate 95% more leads and 3x time spent on site. They’ve created over 200,000 vir tual spaces, which have generated over 50 million visits. However, the process is ver y expensive: the camera itself is $4,500 on top of a $50-150 monthly fee.

Sotheby’s International Realty

Sotheby’s International Realty has star ted to produce 3D scans to enable vir tual walkthroughs, viewable both via web browser and HMD. Currently they are taking advantage of the Samsung Gear VR technology to distribute.

Transported

Transpor ted is also developing solutions for VR tours, available on Samsung Gear and Oculus, with HTC Vive, Google Daydream and Cardboard apps coming soon.

VR Global

VR Global provides the commercial, mixed use, and residential real estate markets with easy to use applications. Their applications combine 3D rendered models, CGI, and 360° content that is accessible via VR and AR environments as well as web and mobile. They also offer customized white label applications designed specifically for real estate agencies, sales agents, developers, designers, architects, and creative agencies.

RETAIL Alibaba

Alibaba is the world’s largest retailer, and has recently launched a VR store called Buy+. This allows consumers to wander through a VR mall with big name stores like Macy’s, Target, and Costco. The stores have inventor y that consumers can interact with and purchase. Additionally, Alibaba has opened a VR research laborator y called the Gnome Magic Lab, which works on VR and AR technologies to assist sellers on Alibaba platforms to build their own 3D product inventories. The long-term goal here is to give any business the tools to create its own 3D inventor y, and to set up an e-commerce store in VR.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

eBay

eBay teamed up with Australian retailer Myer for what was touted as “the world’s first VR depar tment store.” When users enter the vir tual store, several categories of retail items are shown. Users then select an area of interest and browse without leaving the comfor t of their home. Discover y and recommendation are a primar y par t of the experience, as algorithms analyze selection and rejection to identify preferences and predict future actions. Users select items through a system called “eBay Sight Search.” Users hold their gaze for a couple of seconds, and are offered a more in-depth menu of available actions. Items can be added to the basket or analyzed in a more detailed way. To check out, users remove the headset and return to the eBay app.

Ashley Furniture

Ashley Furniture VR and AR enables users to create interior layouts and experience living spaces in 3D. “Augmented and vir tual reality are essential to our growth and vision for the future,” said Todd Wanek, Ashley’s President and CEO. “Our data shows that a combination of 3D visualization, seeing, touching and feeling actual products, combined with the consultation of our knowledgeable salespeople, will lead to a stand-apar t customer experience that is location-flexible.” To capitalize on the emergent medium, Ashley is working closely with Marxent, a leader in augmented reality and vir tual reality for products and spaces. Ashley is utilizing Marxent’s VisualCommerce platform to create and scale their product catalogue and publish new 3D product experiences for VR and AR. Beyond Ashley Furniture, Marxent’s clients include For tune 500 retailers, builders and manufacturers including American Woodmark, AZEK Building Products and USG. IKEA VR

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

IKEA

IKEA VR enables customers to tr y out a variety of home furnishing solutions before buying them, ensuring optimal functionality and customer satisfaction. The app is available on the Steam platform, and is capable of room-scale motion tracking so that a user can perform basic tasks like open doors, drawers, and cabinets for the space they are creating. It is also equipped with a telepor t function for flexible and quick mobility. The app also features the ability to shrink oneself to child size in order to test whether any of the door frames or counter tops would pose a danger. Likewise, users can enlarge themselves and experience it as 6.4 ft. (195 cm) tall adult.

TRAINING

VR training is “learning by doing.” Flight simulators for training pilots are the most well-known and successful applications of VR, but other sectors are enhancing their training programs through the use of AR and VR.

Lockheed Martin

Lockheed Mar tin awarded a grant to Brigham Young University to develop training for engineers, teaching them how to replace and repair aerospace and defense units. The system leverages immersive 3D technology from the Unity game engine, to create training environments accessible to engineers from any location in the world. This vir tual technology can reduce the costs of training with constantly changing hardware, and significantly reduce the costs of having to transpor t and house employees during training.

Modal VR

Modal VR, co-founded by Nolan Bushnell (founder of Atari and Chuck E Cheese), is working on a versatile platform that, among several other uses, could help train law enforcement. Police force training is currently simulator heavy, so the ability to cut training costs with a digital environment might revolutionize testing and scenario practice.

contributions@vrarproject.com

VR/AR Project

MODAL VR

New Amsterdam Media

Virtual Heroes

Vir tual Heroes has several programs that gamify training in medical, militar y, education, and government. One of their more successful programs is called HumanSim. It enables doctors, nurses, and medical personnel to interact with patients in a 3D environment. The simulation enables health care professionals to sharpen their assessment and decision-making skills without risk to patients in realistic, challenging, immersive environments.

Campfire Union

Another example of training simulators available today is Tower Crane VR, Trainer Edition by Campfire Union. This is a multiplayer learning system that allows trainees to come together in a vir tual construction site. One user operates the tower crane, while another user acts as signaler from a nearby building, providing both visual and auditor y cues via the company’s online multiplayer VR technology.

TRAVEL & HOSPITALITY

VR allows consumers to explore locations around the world, helping hotels sell and upsell travel products and accommodations. A technology that enables total immersion, exploration, and sense of presence is ideal for this industr y. With VR tours, mass audiences can be reached at once; a travel agent or tour guide no longer needs to ser ve people one-by-one About 44% of travel sales and bookings are expected to occur online over the next five years, according to data from Euromonitor International. This is 4x higher than the retail industr y (11%) and nearly 5x higher than restaurants and dining (7%). It is clear that prospective guests are using a travel company’s digital presence to determine if their hotel is the right pick for their stay. While mobile apps and social media platforms have become the norm, VR tours have yet to reach critical mass. Offering prospective guests something new and exciting gives par ticipating travel and hospitality companies a significant marketing edge.

ASCAPE

ASCAPE is a vir tual tour app with professional and usergenerated 360 content aggregated around travel destinations and experiences.

About 44% of travel sales and bookings are expected to occur online over the next five years, according to data from Euromonitor International. This is 4x higher than the retail industr y (11%) and nearly 5x higher than restaurants and dining (7%).

Jaunt

Jaunt’s Home Turf series transpor ts you to adventure-spor t hot spots, with pro athletes guiding you along. “Rock-climb in the Needles, kayak a waterfall in Iceland, or slackline across a canyon in Moab.” contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

Marriott Hotels

Marriott is test-driving VRoom Ser vice, whereby guests order a headset to watch “VR postcards” of different Marriott locations. They also have 4D experiences of their locations that consumers access through booths they call the “Telepor ter station,” which travel to Marriot locations throughout the world. The telepor ters have been an extremely popular component of Marriott’s “Travel Brilliantly” campaign.

Qantas Airlines

Australia’s Qantas Airlines par tnered with Samsung to create the first in-flight VR experience; strap on a Gear VR headset to take a helicopter tour of the Hamilton Islands, or snorkel off the Great Barrier Reef.

Radisson Hotels

Radisson Hotels repor ted that hotels offering a VR tour are seeing online revenue increases of 135% over hotels without one.

Solfar Studios

Solfar studios has created a VR experience to take you to one of the most unreachable locations on ear th: Mt. Everest.

TimeLooper

TimeLooper is a VR/AR mobile app that is reshaping how we experience histor y and sightseeing. TimeLooper has recreated and documented historic events at iconic locations across the globe so that when a user physically arrives at one of the historic contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

sites, they can relive moments that made them historic. There are several sites in London, New York, and Washington D.C ., with other sites being added frequently. When one visits the par tner sites around the world, they pick up a headset at the ticket office, and experience the historic moment. To use in AR mode, just skip the headset and align the smar tphone against the on-site surroundings to see the site in different time periods.

Thomas Cook

Thomas Cook offers their own VR app that gives travelers a glimpse of vacation offers, and gives immersive over views of various resor t hotels.Â Bloomberg Business repor ts that VR marketing increased revenue for a Thomas Cook New York tour by 190%.

Virgin Atlantic

Virgin Atlantic lets potential customers experience an Upper Class flight.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

COMPUTER RENDERED VR

Co m pu t er r e nd e r e d V R r e fe r s to the use of a gam e engine such as Un it y, Un r e al , o r C r y E n g i n e , to r end er —in r eal-tim e—a vir tual wo r ld t h a t c an b e mad e i n t e r a ctive thanks to the ab ility of the sy st em t o r e n d e r c h a nge s i n l ighting, m otion, d istance , tim e , etc . T h i s i s n ot po s s ible wit h t r a dit io n a l 360 video, or live action V R, w h i c h i s p re -re n de re d. A s a re fer ence point, one fr ame o f a n a n i m a t ed film c a n t a ke u p t o 12 hour s to be r ender ed; a g a m e en g i ne re n de r s a f r a me e ver y 10-15 milliseconds. C G V R ca n ta ke a dv a n t a g e o f t h e 6 degr ees of fr eedom (D O F ) e n a bl e d by f u ll bo dy mo t io n tr acking. T he companies b e h i n d t h e ma jo r g a me e n g in e s h ave r eal i zed the potenti al o f t h ei r t e c h no lo g y t o de ve lo p V R and AR applicati ons and c o n t en t , a n d h ave a lre a dy be e n gi ving maj or suppor t to e d i t i n g t o o l s de s ig n e d fo r n a t ive V R/AR devel opment. T h e m a i n d r awba c k o f C G v s live acti on is the “uncanny va l l e y t h esi s,” wh ic h po s it s t h a t people find r ender i ngs of na t u r a l fe a t u r e s t o be a ppe a lin g —u ntil they l ook almost, but not ex a ct ly, l ike t h e s u bje c t t h e y ’re mi mi cki ng. For example ,

contributions@vrarproject.com

VR/AR Project

“Raw Data” Survios

New Amsterdam Media

i t i s c o m for ta ble fo r u s t o wa t c h a car toon ani mati on of a h u m a n . B u t if t h a t ima g e g e t s t oo close to a human, it fe e l s we i r d , un t il it re a c h e s e x a c t repr esentati on—w hi ch i t never do e s. A A A g a me de ve lo pe r s have still not r eached th e po i n t o f qu a lit y wh e re c h a r acter models and vi deo ga m e e nv i r on me n t s lo o k e x a c t ly like they do in l i fe as well.

S o ph i st i ca t e d mo t io n -c a pt u re t e chniques pi oneer ed by b ig bu d g et films h ave s pu r re d t h is ar ea of technological d e vel opm e n t , a llowin g n a t u r a l h uman movement to be c a p t u r e d a n d re n de re d in a v ir t ual space . Now, moti on c a p t u r e t e c h n o lo g y is e vo lv in g t o enable full r ender i ngs o f h u m a n bo die s in t o V R, c re a ting extr emely detailed m o de l s t h a t c a n be wa lke d a ro u n d in full 6 DO F, pr actically by pa ssi n g t h e “u n c a n ny v a lle y ”. Th i s is cal l ed volumetric c a pt u re o f v ide o a n d c a n t e c h n ical ly be consi der ed both l i ve a c t i o n a n d C G; da t a po in t s a r e captur ed by a camer a s y st em . Th en , t h a t da t a is fe d in t o a game engi ne to be p r oc e sse d a nd v is u a lize d. An ot h e r st e p be yo n d vo lu me t r ic captur e is l i ght fi el d c a p t u r e , w h ic h u t ilize s a c a me r a system that captur es d a t a o f a l l l i g h t in a s pa c e , a n d t r ansl ates that data i nto a ga m e e n g i n e , a llowin g fo r t h e manipulation of indivi dual p ixel s f r o m t h e s c a n n e d e nv iro n ment to cr eate i nter acti ve e x p e r i en ce s. Th is ma r r ia g e —c a pt u r i ng a r eal envir onment an d eve r yt h i ng in it , a n d t h e n be in g abl e to mani pul ate i t— i t i s t h e H o ly Gr a il o f V R a n d A R c r eati on. I t w ill enable the re - cr e a t i on of o u r c u r re n t u n ive r s e and the gener ati on of c o m p l et ely n e w o n e s . contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

LIVE ACTION VR

As o p po se d to c o mp ute r- g ener ated V R , live action V R r efer s t o th e c a p t u re o f l i ve e ve nts i n to a navigable exp er ience , in w hich t he viewe r c a n e x p l o r e t he p hy s i c al sp ace of the p iece .  In t h e pa st , live V R h a s be e n s u bj ect to l i mi ted r ange of m o t i o n , i n w h ic h u s e r s c a n move thei r head to explor e a s p a ce - bu t ca n ’ t n e c e s s a r ily move for w ar d to navi gate their e nv i r on m en t . Th is is c h a n g in g a s mor e advanced “stitchi ng” (th e edi t i n g o f V R file s t o fo r m c o nti nuous, seaml ess fi l es) have be g u n to a dd t h e Z A x is in to live exper i ences. T he d e vel opm e n t o f vo lu me t r ic s c e n e captur i ng technol ogy w i l l fur t h er e vo l ve t h e c a pa bilit ie s o f t his medium.    S u p e r i or p o s t pro du c t io n pro c ess is alw ays essential fo r m a k i n g a s u pe r io r V R pro du ct. Tool s such as i mage s t a bi l i z a t i o n a re pa r t ic u la r ly c r u ci al to r educi ng moti on s i c k n e ss a n d improv in g t h e ove r a ll exper i ence .

“Inside the Box of Kurios” Felix and Paul/Cirque Du Soleil

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

WEB VR

We bVR e n a bl e s 3 6 0 ° / V R c o n t ent to b e viewed on m ob ile , d esktop, an d in VR h e a d s e t s t hr o ugh web ap p lications. WebV R en a ble s 360°/V R c o n t e n t t o be vi ewed on mobile , desktop, and i n VR head sets th r o u g h web a pplic a t io n s . A -Fr a me is the fir st open sour ce l i br ar y pi oneer ed by M o z i l l a fo r we b de ve lo pe r s t o u s e for bui l ding websi tes w ith VR vi ew i ng capabil i ti es usi n g a s l i t t l e a s o n e lin e o f H TML code . I t eliminates the need for develope r s to kn ow t h e co mple x We bG L 3D A P I other w ise r equi r ed to code an exper i enc e to l i fe . O t h e r o pe n s o u rc e libr a r ie s have been cr eated si nce A-Fr ame including Vi zor, P layC a nv a s, Ba by lo n JSo r, Pr imro s eVR , Zeovr, and O culus’ ow n R eactVR . Web VR i s cu r r en t ly s u ppo r t e d by a ll ma jor web br ow ser s as wel l as those compati bl e w i t h Oc u l u s Rif t , HTC V ive , Sa ms ung Gear, and Daydr eam VR . WebVR w i l l be th e fast e st w ay for v ir t u a l re a lit y t o prol i fer ate thr oughout society and enable seam l ess m ove m e n t t hro u g h o u t e x pe r ie n c e s. As a p t ly ex pl ain e d by Th o ma s B a lo uet, cofounder of WebVR -based star tup Luci d Web, “You k n ow t h a t fe e lin g yo u h ave w hen you just w ant to check a cooki ng r e ci p e o n t h e I n t er n e t , a n d t wo h o u r s later you’r e r eading an ar ti cle about Patagon i a o n W i k i pe di a? We ll, ima g in e yo u put on your headset to w atch a cool r ela xi n g e x p e r i en ce , a n d 15 minu t e s la t e r you fi nd your sel f tal king w ith fr i ends on top of a m o u n t a i n o r f ly in g ove r yo u r h o me tow n. T hat’s w hat the Metaver se would be , an d th i s k i n d o f sit u a t io n wo u ld be possi ble only w i th deep- l i nki ng, one place br i n gi n g yo u t o a n ot he r o n e re la t e d t o it , w i thout you havi ng to come back to a com m on p la c e , o r e ven re a lizin g yo u ’re le av ing the fir st place . O nly WebVR can enable t h at.”

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

SOCIAL VR

B o t h A R a nd V R ( ul t i mate ly, m ixed r eality) w ill fund am entally evol ve t h e way we c o n n e c t w i th e ach other. W h i l e AR a n d M R will de ve lo p i nto a hol odeck type e x pe r i en ce t h a t will a llow peopl e to visualize their f r i e n d s a n d f a mily wit h t h e m wher ever they ar e , VR w i l l pr ov i d e a t y pe o f e x pe r ie n c e de r i ved fr om w hat we know t oday a s M M ORPGs o r ma s s ive ly mul tipl ayer online r ol e pl ayi n g g a me s , e x c e pt o n a mu c h gr ander scal e and w ith m a ny m o re dif fe re n t a c t iv it ie s t o par take i n. A l t h ou g h c e r t a in s o c ia l s pa c e s i n VR w i l l end up being m o r e po pu la r t h a n o t h e r s , e ve ntual ly all vir tual wor lds w i l l b e sea mle s s ly n av ig a ble . Th e ter m for the fr amewor k c o n n e c t i n g a ll s u c h v ir t u a l s pa ce together i s called the m e t aver se .

AltspaceVR

AltspaceVR wa s s t a r t e d in 2013 a nd devel oped usi ng U nity a n d A - f r a m e . It wo r k s wit h Rif t , Vive , and Daydr eam and S a m su n g Ge a r H M D s . Th e ide a is for Al tspace to â&#x20AC;&#x153;l i veâ&#x20AC;? in WebV R a n d fo r a nyo n e t o be a ble to build and dr op 3D a sset s or e n t ire in t e r a c t ive e x p er i ences using A- fr ame , t o a s m a ny de v ic e s a s po s s ible . A l t sp a ce ha s fo c u s e d o n h o s t ing l i ve events to bolster e n g a g e m e n t a n d e x plo re s u c h a possible avenue for m o n et i z a tio n , h owe ve r t h e c o m pany acknow l edges that t h e soc i a l V R s pa c e is s t ill ve r y much i n i ts i nfancy. I n t h e n ex t se ve r a l ye a r s t h e g o a l is to build an engagi ng, m o n et i z a ble , a n d s c a la ble s o c ia l ecosystem.

contributions@vrarproject.com

VR/AR Project

AltspaceVR

New Amsterdam Media

Facebook Spaces

Spaces, t h e fir s t ma jo r s o c ia l V R exper i ence r eleased by F a c e bo o k , h as be e n h ig h ly a n t ic ip ated and gener ally well re c e i ve d. O c u lu s Rif t u s e r s c a n e nter a vi r tual space that th e y ca n sh a r e wit h o t h e r Rif t we a rer s, as wel l as w ith fr i ends vi a F a ce bo o k . 360 v ide o s a n d movi es can be exper i enced tog et h e r, a n d u s e r s c a n c re a t e v ir tual objects in the space an d ch oo se f ro m s e ve r a l f a c ia l e x p r essions for thei r avatar to p e r for m ( wh ic h is c re a t e d by a n al gor i thm that analyzes yo u r F B ph ot o s , a n d c a n be c u s t o m ized w ith accessor i es) . T h e av a t a r s c a n e ve n t a ke s e lfie s together w i th a vi r tual s e l fi e st i ck , wit h t h e a bilit y t o po s t photos onl i ne i nstantly.Â F a c e bo o k i s s lowly bu t s u re ly re a liz ing i ts vision of enabl i ng s o ci a l V R. Wh ile t h e c o mpa ny may not have announced any t h i n g a s a mbit io u s a s Sa n s a r t o date , i t has a l ong-ter m ro a d m a p for in t ro du c in g s o c ia l V R to F acebook user s i n th e m o st n a t u r a l a n d c o mpe llin g way possi ble .

High Fidelity

P h i l l i p Ro seda le , t h e fo u n de r o f Second Life , left Linden L a b i n 2 0 0 9 , s e ve r a l ye a r s a f t e r u ser gr ow th stagnated at ar ou n d on e m illio n . H is own s pir it ual successor i s based on hi s v i si on t o de mo c r a t ize t h e c re ation and shar ing of VR e x p e r i en ce s. High Fidelity c a n be dow nl oaded by anyone w ith a PC c a p a bl e o f r u n n in g it a n d a llow anyone to essential ly ho st t h e i r ow n V R do ma in a n d a llow anyone to cr eate vi r tual e xpe r i en ce s wit h ba s ic c o din g s k ills. A sandbox cr eation to o l k i t e n a bl e s t h o s e wit h n o c o d i ng skills to cr eate and s h a r e c o n t e n t . Ro s e da le s t ill c o lla bor ates consi stently w i th L in d e n L a b i n t h e de ve lo pme n t o f Sansar, seei ng Sansar and F i d e l i t y a s a n a lo g o u s pro je c t s with differ i ng appr oaches tow a r d u l t i m a t e ly c re a t in g s o c ia l c onnecti on w ith VR .

360 vid eos and m ovies can be exp er ienced together, and user s can cr eate vir t ua l ob jects in the sp ace and cho o se fr om sever al fac i a l exp r essions for their avatar to p er for m ( w hi c h is cr eated by a n algor ithm tha t analyzes your F B p hotos, and can be custom ized wi t h accessor ies)

Project Sansar

T h e c o m pa ny be h in d Sa n s a r c a lle d Linden Lab al so cr eated ar g u a bly t h e de e pe s t v ir t u a l o n line soci al exper i ence to d a t e c a l l ed Se c o n d Life . La u n c h ed i n 2003, the vir tual wo r l d co u l d be a c c e s s e d by a ny basel i ne computer w i th i nt e r n et , a l l owin g u s e r s t o c re a t e their ow n avatar s and c o m mu n i ca t e wit h e a c h o t h e r t h rough text or voi ce chat w h i l e e x pl or in g t h e v ir t u a l wo r ld. S ec o n d L i fe h a d it s own e c o n o my and cur r ency ( the Linden d o l l a r L $ , w h ic h c o u ld be e x c h a n ged for r eal dollar s) as we l l a s i t s ow n c o n t e n t c re a t io n t ool set. U ser s developed c o m mu n i t i es a n d e ve n t s in Se c ond Li fe that touched o n ever yt h i ng f ro m politics a n d education to religion and relationships. contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

B u i l d i n g o n t he s u c c e s s o f Se c o n d Li fe ( w hi ch i s still acti vely s u p po r t e d a nd will be in t h e fo reseeable futur e) , Pr oject S a n sa r w i l l l a u n c h in o pe n be t a s o m etime i n summer 2017 an d w i l l st a r t t o a c t ive ly e x pa n d the inter activi ty of the vi r t u a l spa ce a s mo re u s e r s jo in t h e pl atfor m. Li nden Lab w ill m o n et i ze by t a k in g a s ma ll pe rc e n t age fr om al l tr ansactions m a d e by u ser s . Su ppo r t is pla n n e d not only for O culus R i ft an d V i ve bu t a ls o PC s (in 2D mo de) and eventual ly mobi l e H M D s.

Pluto VR

Inst ea d o f a t t e mpt in g t o c re a t e a social vir tual wor l d and e n t i ce u ser s t o in t e r a c t wit h e a c h other i n it, Pluto VR takes a co m pl et ely dif fe re n t a ppro a c h t o social VR by enabl i ng s o ci a l i n t e r a ct io n a c ro s s c u r re n t VR exper i ences that ar e al r e a d y av a i l able . Us e r s c a n jo in their fr i ends and w atch as t h e y sc r o l l t h ro u g h e x pe r ie n c e s on their headsets w hi l e th e y a r e i n side t h e v ir t u a l s pa c e r epr esented as avatar s. T h e l evel of in t e r a c t io n is c u r re n t ly l i mi ted to tal king and vi su a l g est u r e s wit h t h e av a t a r s , and Pluto i s devel opi ng ad di t i on a l fe a t u re s t o e n a ble t h e s har ing of other content.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

LOCATION-BASED VR

THEME PARKS Global Malaysia VR

Global Malaysia VR Theme Park is bei ng devel oped by gami ng company EXA Gl ob al w i t h M edi a sof t a s a c o n t e n t pa r t ner, in Puchong Mal aysi a. T he par k w i l l star t of f w i t h t h r ee wide o pe n ro o ms wh e r e gr oups of 4 r oam fr eely, fending off i ncom i n g w aves of a t t a c ke r s wh ile a ls o be ing abl e to upgr ade weapons and ar mor in a g am e s i m i l a r t o Z e ro La t e n c y. Th e e x per i ence i s expected to cost r oughly $18 and l ast ab ou t o n e h ou r.

SPACES

SPACES w as founded by DreamWor ks vets Br ad Her man and Shir az Akmal in summer 2016, as a $ 3 0 M j o in t ve n t u re be t we e n SPACES and Songcheng Per for mance Developm en t C o. t o br i n g m ixe d a n d v ir t u a l re a lity to existi ng show s and r i des at Songcheng th em e p a r k s. S p a ce s will a ls o be wo r k in g to enhance Soncheng’s existi ng live str eaming si te , 6Ro o m s, w h i c h a t t r a c t s 33 millio n user s a month str eami ng thei r ow n show s an d c r ea t i n g m ore t h a n 50, 000 live broadcast hour s each day.

“The Void”

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

The Void

The Void i s a V R t h e me pa r k ba s ed i n Lindon, U tah w i th ad di t i on a l l o c a t io n s in Ne w Yo r k and Dubai . Essenti ally a hi g h - t e c h m a ze in a wa re h o u s e k itted w i th movement and p r essu r e se n s o r s , pa r t ic ipa n t s pu t on a w i r el ess backpack w i t h a t et h e re d h e a ds e t a n d pla s ti c gun, and explor e the vi r t u a l ex pe r ie n c e wit h f r ie n ds . Ut ilizing a l ayer i ng of r ealti m e i n t er a ct ive e nv iro n me n t s , a n d blending the r eal wor ld w i t h t h e di g i t a l—pa r t ic ipa n t s a re placed i nto “Hyper-R eal i ty” e x p e r i en ce s. Th e fir s t ma jo r e x pe r i ence devel oped for T he Voi d i s t h e Gh o s t bu s t e r s D ime n s ion: VR Exper i ence , w hi ch l au n c h ed a t Th e Vo id’s NY lo c a t ion i n Madame Tussauds Wa x M u se u m . Th e fe e dba c k f ro m t hose w ho’ve tr i ed i t has b e e n ove r w h e lmin g ly e x c it in g . Th e ir cur r ent r evenue model i s ch a r g i n g $ 20 t o u pg r a de yo u r Wax Museum ticket for an ad di t i on a l 1 5-minu t e e x pe r ie n c e . The Void pl ans to open an ad di t i on a l 2 0 lo c a t io n s in 2017 a n d China’s Shanda Gr oup ha s i nvest e d $350 millio n in t o Th e Void to br i ng VR theme p a r k s t o C h in a , wh ic h s h o u ld h e l p accel er ate T he Voi d’s goa l o f ope n i n g 230 locations by 2020.

Zero Latency

Zero Latency is a n o t h e r h ig h -pro file VR theme par k based i n M el b o u r n e , A u s t r a lia , wit h lo c a t ions in O r lando, Flor i da an d To k yo, J a pa n . Th e ma in c o n s u m er exper i ence r evolves ar ou n d a fou r-pe r s o n s h o o t e r in w hi ch player s defend a b a t t l e g r ou nd f ro m zo mbie s . Th e company’s pl an i s to p r ov i d e a “ bu s in e s s in a box ” re venue par tner shi p model fo r a ny i n t e r n a t io n a l pa r t n e r s t o set up and oper ate . T h e y co n si d e r t h is f r a n c h is e mo d el to be most effi cient, r a t h er t h a n bu ildin g a n d o pe r a t in g i ts ow n vi r tual r eal i ty e x p e r i en ce c e n t e r s . Z e r o L a t en cy h e lps wit h t h e de s ig n and instal l ati on of the att r a c t i o n , su pply in g bo t h o n g o in g s uppor t and content. T hey ut i l i ze o n l i n e t o o ls f ro m s y s t e m d i agnostics to a booki ng e n g i n e , p r ov idin g a c o mple t e t u r n key package . O ther than mu l t i p l ayer g a min g , t h e Ze ro La t e ncy space has al so been use d for o t h e r in du s t r y a pplic a t io ns such as 3D r eal estate vi su a l i z a t i o n s. T h e M e l b o u r n e s pa c e is 400 s q uar e meter s of space , p e r c e i ve d t o be mu c h big g e r du e to r edir ected w al king. T i cket s a r e $ 88 pe r pe r s o n . A s t a ndar d ticket w i l l get you ap pr oxi m a t ely 45 minu t e s in -g a m e , depending on how fast you m ove t h ro u g h it . Wire le ss motion tr acking w i th 1 2 8 c a m er a s e qu ips e a c h ro u n d for up to 6 pl ayer s. I n th e n e a r f u t u re , t h e y a re ro llin g out customi zed HMD’s from Sensics u s in g t h e OSVR platfor m, as wel l as new c o n t r ol l e r s. contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

Viveland HTC

VRCADES IMAX

IMAX VR h a s o pe n e d t h e ir fir s t e xper i enti al VR center i n L o s An g e l es, c h a r g in g $10 fo r 10-15 minute play-thr oughs o f p o pu l a r St e a mV R g a me s o n t h e HT C Vi ve and Star VR hea d set s. I M A X h a s re c e n t ly a n n o unced a par tner ship w i th Wa r n e r B r os . t o c re a t e V R e x pe r ie nces for thei r super her o m ov i e I Ps a n d br in g t h e m t o t h e V R center.

Modal VR

Modal VR h a s de ve lo pe d a s y s t e m that l ets up to 10 user s e x p l or e v i r t u a l re a lit y in la r g e s paces—as big as 900,000 s q u a r e fee t , wit h du r a ble , u n t ether ed headsets. T he c o m p a ny h a s a lre a dy de ve lo pe d thr ee games for their s y st em a n d is wo r k in g wit h de vel oper s to make mor e ga m e s for i t s pla t fo r m. Th e y a re al so fi ndi ng par tner s to d e vel op e n t e r pr is e a pplic a t io n s fo r the system such as for re a l e st a t e a n d c o n s t r u c t io n /a rchi tectur e vi sual i zati ons, i m m er si ve edu c a t io n a l pro g r a ms , a nd tr aini ng si mulator s for j obs o f a l l t ype s . Th e s y s t e m is be in g devel oped to be semip o r t a bl e , i n orde r t o e mpowe r c lie nts to take i t mobile and s e t i t u p i n v a r io u s lo c a t io n s a s n ecessar y.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

Two Bit Circus

Two Bit Circus is bu ildin g t h e n e xt gener ati on of mi cr oamu sem e n t pa r k s , re c e n t ly a n n o unci ng a 30,000 squar e fo ot spa ce i n Lo s A n g e le s , wh ic h w i l l house al l of thei r e x p e r i en ce s. Two B it C irc u s a ls o has tr avel i ng pr ogr ams, b r i n g i n g i m m e r s ive e n t r a in me n t t o you. O ne of their most p o pu l a r i n t his c a t e g o r y is STEAM carnival, w hich gami fi es s c i e n ce , t e c h n o lo g y, e n g in e e r in g , ar t, and math. Amongst s e ve r a l ot h e r e x c it in g pro je c t s , t hey have a pr opr i etar y ha pt i c t u r r et mo t io n pla t fo r m t hat enables fi ghter- pilot s t yl e g a m e s.

Viveport Arcade

Vivepor t Arcade is HTC ’s pro pr ie t ar y content management an d d i st r i bu t io n pla t fo r m fo r lo c a t ion-based enter tai nment. It se r ve s b o t h o pe r a t o r s a n d developer s by i dentifying, s e cu r i n g , a n d s a fe ly dis t r ibu t in g c o ntent, as wel l as managing p ay m e n t s t h r o u g h a c e n t r a lize d s y stem. T he oper ator gets a l i br a r y o f V R c o n t e n t t h e y c a n use , w hi l e HT C tr acks usa g e a n d pl ay t ime dire c t ly, a n d s h ar es that w ith the ar cade o p er a t or. C o m m e r c i a l a rc a de o pe r a t o r s c a n dow nload the softw ar e to t h ei r venu e s , a llowin g t h e m access to a l i br ar y of l e g a l ly a cqu i re d c o n t e n t , wh ic h t hey can confi gur e acr oss th ei r V i ve st a t io n s v ia a c e n t r a lize d h ost cl i ent tool . O per ator s an d de vel ope r s t h e n s plit t h e revenue . T he softw ar e i s bu i l t t o se r ve a s a t u r n ke y me t h o d for developer s to easi ly d ist r i bu t e a n d mo n e t ize t h e ir c o n t ent to physical l ocati ons l i ke a r ca d e s, In t e r n e t c a fe s , t h e a t er s, and shoppi ng mal l s ar ou n d t h e wo r ld, wh ile g iv in g o per ator s an end-to-end s y st em t o dis c ove r a n d de live r g reat VR content to their c u st om e r s.

T h e s o f tw ar e is bu i l t t o s e r ve a s a t u r n ke y m e t ho d for d e velo per s to e a s i ly d i s t r i bu t e a n d m on e t i ze t he ir con ten t to phy s ic al loca t i o n s l i ke a rca d e s , In t e r n e t c afe s , th eater s, an d s h o ppin g m a l l s a ro u n d t h e wor l d , w h i l e giving op er ato r s an e n d-to -e n d s ys t e m t o d i s cove r a n d d e live r gr eat VR c on t e n t t o t h e i r cu s t o m e r s .

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

SMARTPHONE VR/AR

Th o u g h le ss p owe r f u l t han p r em ium H MDs, m ob ile V R lower s th e ba r r ier to a d o p ti o n by an or d er of m agnitud e . T he evolution of m o bile V R / A R w i l l b e s p ur red by the enor m ity of the glob al sma r t ph o n e mar ke t, as manu f a ctur er s and ser vice p r ovid er s look for way s t o d r i ve a d o p ti o n .  Google Cardboard

S i n c e 2 0 1 4 , G o o g le h a s dis t r ibu ted over 10 mi l l i on Cardboards. “ C a rdbo a rd” re fe r s t o smar tphone holder s th a t m ou n t ove r a mo bile ph o n e , si mi l ar to a pai r of b in o c u l a r s. C a rdbo a rd is s ma r t ph one agnostic; the user j ust dow n l oa d s a n a pp like W it h in o r J aunt, or can br ow se th e g r ow i n g s e le c t io n o f Yo u Tu be 36 0. Car dboar d enabl es th e m o st ba sic V R e x pe r ie n c e s , a n d at a pr i ce point fr om free t o $ 1 5 , it is powe r in g t h e mass adopti on of VR in th e n ea r- t e r m. Mo s t pe o ple ’s fir s t VR exper i ence w ill c o m e t h r o u g h a C a rdbo a rd.

Google Daydream

C om po sed of bo t h h a rdwa re a n d s o ftw ar e , Daydream i s the hi g h e r- t ec h su c c e s s o r t o Go o g le Car dboar d. T he devi ce i s a m obi l e VR s y s t e m c o mpa t ible w i th the gener ati on o f An d r o i d s ma r t ph o n e s r u n n ing Andr oid Nougat. W h i l e C a r d bo a rd wo r k s wit h a lmost any smar tphone , D aydr e a m will be c o mpa t ible with exclusi ve phones:

contributions@vrarproject.com

VR/AR Project

Daydream VR Google

New Amsterdam Media

G o o g l e P i xel, Mo t o ro la M o t o Z, Huawei Mate 9 Pr o, ZT E Ax on 7 , A S US Ze n Ph o n e A R, wit h mor e Daydr eam r eady p h o n es co m i ng t h is ye a r. T h e co m po n en t s o f t h e s e ph o n e s offer a smoother, lowerl at en cy e x pe r ie n c e t h a n wo u ld be possibl e w ith a si mple s o f t w a r e u pda t e t o a lt e r n a t ive smar tphones. Daydr eam re t a i l s fo r $ 79.

Google Tango

Google Tango is a n A R c o mpu t in g platfor m w hich enabl es s m a r t p h on es a n d t a ble t s t o detect their r elative p o si t i o n t h r ou g h t h e u s e o f 3D c amer as and odometr y. I t b r i n g s h u m a n -like v is io n t o yo u r mobile devices, w hich can b e u sed for a wide v a r ie t y o f a pplications such as AR , MR , i ndo o r n av i ga t io n , phy s ic a l s pa c e measur ement, and 3D s c a n n i n g . L e n ovo ’s Ph a b2Pro wa s the fi r st Tango-power ed d e v i c e br o u gh t t o ma r ke t in 2016. I n J anuar y 2017, Asus l au n c h ed Ze n Fo rc e A R a t C E S. Sin c e i ts r el ease , developer s have bu i l t 3 D s c a n n in g a pps wit h Tango. Among them ar e: 1. RTAB-map 2. Evr yway 3. Open Constructor 4. Simple scanner 5. Google Constructor 6. Matterpor t 7. Voxxlr

Intel RealSense

RealSense su pplie s 3D de pt h ma pping. T he technol ogy uses va r i ou s sen sin g t e c h n o lo g ie s t o a c hi eve depth per ception,

contributions@vrarproject.com

VR/AR Project

Tango Google

New Amsterdam Media

3 D i m a g i n g , in t e r io r ma ppin g , a n d featur e tr acking. I n conjunction w i th Movidius, Int e l i s p u sh in g t h e bo u n da r ie s o f computing i nnovati on, to supply VR , AR , and M R d e vel ope r s w it h s t re a mlin e d 3D s canning and a si mpler pr ocess for captur i ng 3 D m o de l s. I n t e l Re a lSe n s e is c e r t a in ly power i ng the next w ave of immer sive media w ay b e yon d sm a r t ph o n e s .

Samsung Gear VR

Samsung’s Gear VR prov ide s u s e r s a high-qual i ty VR exper i ence w i thout the h i gh c o st of spe c ia lize d, mu c h mo re power ful alter natives. Samsung has str ategi cal ly p o si t i o n ed t h e ms e lve s a t t h e fo re f ront of mobile VR , usi ng their massive r esour ces to d o m i n a t e d i st r ibu t io n a n d a t t r a c t new customer s by keeping har dw ar e compati b i l i ty e x cl u si ve .  Th e y s o ld a t o t a l o f 4.51 million units in 2016, outpaci ng ever y compe ti tor. S a m su n g h a s re le a s e d a n u pda t e d model of the Gear VR headset to accommodat e i ts new G a l a xy S8 a n d S8 Plu s ph o n e s. A main di ffer ence in the upgr aded model is th e ad d- on o f a wire le s s c o n t ro lle r, wh i ch allow s i ncr eased i nter activi ty, and a smoo th er e x p e r i en ce . T h e n e w mo de l a n d c o ntr ol l er ar e backw ar d-compati ble w i th the S6 an d S 7 , a n d t h e co n t ro lle r c a n be pu rchased separ ately as an add- on to a consum er ’s o lde r G e a r VR. Th e pr ic e is $130 for the new model of the Gear VR , i ncl udi ng th e c o n t r ol l e r. F ea t u r es i n clu de bu ilt -in s e n s o r s and smar tphone- compatibl e contr ols, as we l l as fo ca l a n d l a t en c y a dju s t me n t , a n d even a cool i ng fan. F acebook’s O cul us i s pr ov i d i n g c o n t en t su p po r t by wo r k in g wit h d evel oper s and gr ow ing the O cul us Stor e . H er e , th e u se r c a n a c c e s s a ll V R c o n t e n t for the headset, i ncl udi ng O culus 360 and Ci n em a. F a c e bo o k i s cu r re n t ly fo c u s e d o n devel opi ng social featur es w ith O culus, i ncl u d i n g av a t a r s a n d o t h e r mu lt iplaye r f u n cti onaliti es. T his gi ves the Gear VR headse t an i m m en se l on g-t e r m a dv a n t a g e ove r i ts competi tor s.

Gear VR Samsung

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

VR HEAD MOUNTED DISPLAYS (HMDS)

HTC Vive

T h e HTC Vive wa s de ve lo pe d by Va lve and HT C as a push fo r cr e a t i on of V R c o n t e n t a n d wider-spr ead adoption of V R. I t w a s r e le a s e d in A pr il 2016, i ts 15,000 pr e- or der s s e l l i n g o u t i n 10 minu t e s . C u r re n tly the cost i s $800 fo r t h e H M D, t wo mo t io n c o n t roller s, and SteamVR tr a c k i n g syst e m. Th e s e , in t a n de m w i th the Vive’s fr ont faci n g c a m er a , le t t h e h e a ds e t a nd contr oller s tr ack th ei r ex a ct l o c a t io n a n d move me nt. For an additi onal $1 0 0 , t h e Vive Tracker c a n a t t a c h t o any r eal-wor ld object, b r i n g i n g po si t io n a l t r a c k in g t o it . HTC i s l o o k i n g fo r a lt e r n a t e fo r ms of consumer adoption r a t h e r t h a n s o le ly v ia re t a il, a n d has par tner ed w i th o n e o f C h i n a’s la r g e s t in t e r n e t c afe softw ar e pr ovi der S h u n Wa n g Tec h n o lo g y. Th is is s mar t consider i ng that th e r e l a t i vely s h o r t le n g t h o f c u r rent VR content lends i ts e l f wel l t o t h e ma s s ive ly s u c c e s s ful business model of c h a r g i n g for p lay t ime by t h e minu te i n China, as wel l as th e f a c t t h a t Sh u n Wa n g c la ims t o reach over 100 million p eo pl e i n m o re t h a n 100, 000 c a fe s , cur r ently contr ol l i ng a m a r ket sh a re o f a ro u n d 70%. For ten mi nutes, user s p ay a ny w h er e f ro m $2-$5. In O c t obe r 2 016, HTC a n n o u n c e d a par tner ship w ith In t er C on t i n e n t a l H o t e ls G ro u p (IHG) to br i ng the HT C Vi ve t o si t es in C h in e s e c it ie s like Bei j i ng, Shanghai , an d S a nya . G u e s t s c a n h e a d t o a ‘ V ive Zone’ or be in thei r ro o m s t o sa mple t h e h e a ds e t wit h a sel ecti on of games an d e x pe r i en c e s t h ro u g h St e a mV R and Vi vepor t. T hat

contributions@vrarproject.com

VR/AR Project

HTC Vive (left), Oculus Rift (middle), Sony PSVR (right)

New Amsterdam Media

s a m e m o n t h , HTC o pe n e d it s t hi r d “Vi ve VR Café” i n S h en z h e n , C h in a a n d h ave c o n fir med hundr eds mor e to co m e i n 2017 wo r ldwide . Th ey have al so suggested fr a n ch i si n g o ppo r t u n it ie s fo r t h e s e cafes.

LG/Valve Developers Kit

Int r o du ce d a t GD C 2017, LG is Va lve’s second HMD par tner. T h e L G / Va l ve D K will be s imila r to the Vive that Val ve c r e a t e d w i t h H TC . It will u t ilize t h e SteamVR platfor m, and Va l ve’s r oya l ty -f re e SteamVR 3D tracking system. T his tr acking s y st e m l e t s t h e de v ic e k n ow, in re a l - time , w her e i t is w ithin a room. T h e H M D w ill h ave a 3. 64-in c h d iagonal displ ay, 1440 x 1 2 8 0 r e sol u t io n pe r e ye , re f re s h r ate of 90Hz, and 110° fi e l d o f v i ew. Th e big n e w fe a t u r e: the goggles flip up w i t h o u t n e e din g t o re move t h e whole headset. T his w i l l p r ove p a r t i c u la r ly u s e f u l fo r de vel oper s, w ho constantly ne e d t o g o i n a n d o u t o f V R wh e n developing content. T he p r o j ec t ed pr ic e po in t is $700-800.

Microsoft Windows 10 VR HMDs M i cr o sof t i s t a k in g V R/MR e x t re me ly ser i ously. Expect to see th e m m a ke big wave s ve r y s o o n . Microsoft has par tnered w i th Ac er, L en ovo, A s u s , D e ll, a n d HP t o cr eate a VR /MR HMDs fo r P C a n d t h e ir XBOX c o n s o le . Micr osoft is maki ng VR / M R a p r i or i t y fo r it s Pro je c t Sc o r pio, the codename for the nex t X box On e mo de l, wh ic h is c a pabl e of tr ue 4K gami ng, hi g h - en d v i r t u a l re a lit y, a n d H D R content. Acer is the fir st m o de l r ea d y t o s h ip, a s it s H M D, Wi ndow s 10 I nsi der, and S D K a r e a l l av a ila ble t o de ve lo pe r s now. T he Acer headset c o m es equ i ppe d wit h t wo h ig h -resolution l i qui d cr ystal, 1 4 4 0 x 1 4 0 0 dis play s a n d a 90 H z r efr esh rate.

Ra t h e r t h a n h av i n g a n e x te r nal ca m e r a re a d the p o s i t i on o f LEDs o n yo u r h e a d , M i cros o f t H MDs i n cl u d e s e ns o r s t h a t a re built d i re ct ly i n t o the h e a d s e t . This is ca l l e d “ i n s i de o u t “ t r a ck i ng a s o p p os e d to “ o u t s i d e - i n ,” and e n a bl e s 6 d e gre e s o f f re e d om .

R a t h e r t h a n h av in g a n e x t e r n a l c a mer a r ead the positi on of L E D s on you r h e a d, Mic ro s o f t HMDs i ncl ude sensor s that ar e bu i l t d i r ec t ly in t o t h e h e a ds e t . This i s called “i nside- out“ tr a c k i n g a s o ppo s e d t o “o u t s ide -in ,” and enables 6 degr ees o f f r ee do m . T h e s e in t e r n a l s e n s o r s detect how the wear er i s m ov i n g a n d re g is t e r s t h a t t o match thei r in- VR positi on. T h i s r e m ove s t h e n e e d fo r a de dic ated VR r oom: user s can w a l k a r ou n d n o r ma lly. Th e s e h e a ds ets w i l l be tether ed, but w i l l n o t r equ ire a s h ig h powe re d c omputer s as do the Vi ve o r Ri f t .

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

M i cr o sof t a ls o pa r t n e re d wit h the Chinese company 3 G l a sses. T h eir h e a ds e t , du bbe d Blubur S1, w i l l be a hi ghere n d syst e m , fe a t u r in g 2880 x 1440 p ixels, and a r efr esh r ate o f 1 2 0 H z . Pr i c e d a t $449, t h e B u lbu r S1 featur es 24 i nfr ar ed s u r f a ce L E D s t o e n a ble pre c is e p ositi onal tr acking w ith th e su ppl i e d e x t e r n a l c a me r a a s par t of the 3Wand Suite . 3 G l a sses a l so h a s it s own s t o re c al l ed VR Show, i ncl udi ng s e ve r a l V R t i t le s a n d s u ppo r t fo r SteamVR via a pl ugi n. M i cr o sof t a n d In t e l a re a ls o c o lla bor ati ng ON “Project Evo,” an i n i t i a t i ve to me r g e t h e ir dive r s e PC offer i ngs to deliver s y st e m s t h a t will s u ppo r t f a r-fie ld speech communi cations. Us er s w i l l be able t o s pe a k t o t h e ir c onnected PCs fr om acr oss a r oo m . Al so in c lu de d will be imp r ovements to Window s H el l o bi om e t r ic ide n t ific a t io n (whi ch cur r ently i ncl udes fi n g e r pr i n t , i r is , a n d f a c ia l re c o g n it ion) ; secur i ty i ntel l i gence fro m bo t h M ic ro s o f t a n d In t e l; g a ming innovati ons l i ke 4K d isp l ay s w i t h H ig h D y n a mic Ra n ge , Wide Color Gamut (W C G ) , sp a tia l a u dio, a n d t h e a bility to br oadcast games an d e - spo r t s ; improve d mixe d-re a li ty exper i ences thr ough affor da bl e PC s a n d h e a d-mo u n t e d displ ays ( HMDs) .

Oculus Rift

Oc u l u s’ $ 2 B a c qu is it io n by Fa c e bo o k r ocketed it to star dom, an d q u i c k ly b e c a me t h e f a c e o f h ig h-end VR . T he Oculus Rift w a s t h e fi r st h e a ds e t t o prov ide devel oper s w i th softw ar e/ c o n t en t c r e a t io n k it s (SD Ks ), a n d active suppor t of thi r dp a r t y co n t e n t de ve lo pme n t by s h ar i ng technical upgr ades an d be st pr a c t ic e s , a n d in s o me cases, even financing d e vel opm e n t . R i f t c o m e s bu n dle d wit h a po s it ional tr acking sensor and X b ox co n t r o lle r. Oc u lu s To u c h is being offer ed for an ad di t i on a l $ 200, a n d f u n c t io n s a s bo th a gamepad and motion s e n sor. I t t r a c k s h a n d move me n t s to al l ow user s to mor e ful ly i n t er a ct wit h t h e ir e nv iro n me nt, w hich pr ovides mor e i nt u i t i ve c o n t ro l t h a n a s t a n da rd gamepad. F acebook and Oc u l u s a r e wo r k in g t o ma ke t h e Ocul us Stor e the l eadi ng d e st i n a t i on fo r V R c o n t e n t , a n d the l eader i n mar keting vi r t u a l ex pe r ie n c e s . Oc u lu s de moed the HMD in select re t a i l l o c a t i on s fo r a bo u t 9 mo n t hs i n 2016. I n F ebr uar y 2 0 1 7 , B est B uy a n n o u n c e d t h e y would r emove hal f of their Oc u l u s de m o s t a t io n s . T h e st r on g vis ibilit y o f t h e Oc u lus br and and the deep fi n a n c i a l r e s o u rc e s o f Fa c e bo o k pr ovided O culus w i th an o pe n i n g a dv a n t a g e in t h e ma r keting, pr oduction and d ist r i bu t i on of Rif t .

contributions@vrarproject.com

VR/AR Project

Ocu l u s ’ $ 2 B a cq u i s i t i o n by F a ce b oo k ro cke t e d i t to s t a rd o m , a nd q u i ck ly b e came t h e f a ce of highe n d V R.

New Amsterdam Media

S e e here for a de t a ile d s pe c c o mpar i son between VI VE and Oc u l u s.

Oculus Santa Cruz

B e yon d t h e Rif t , Oc u lu s h a s a n n o unced its self- contained Santa Cruz H M D. Simu lt a n e o u s Localization and Mapping (S L A M ) t ec h n o lo g y e n a ble s t h e h eadset to use four bui l ti n c a m er a s t o s e n s e t h e e nv iro nment, and mi r r or that m ove m e n t i n t h e v ir t u a l wo r ld. Accor di ng to F acebook C TO M i ke Sc h ro e pfe r : “Th is h e adset doesn’t r equi r e a c e l l p h on e . I t do e s n ’ t re qu ire a PC . No speci al har dw ar e in yo u r r oo m . You ju s t g r a b t h e u n it , p ut it on your head [and] yo u ’r e i n st a n t ly in V R.”

PlayStation VR PSVR h a s so l d ove r 915, 000 u n it s b etween O ctober 2016 an d F ebr u a r y 2017, ma k in g it t h e best- sel l i ng hi gh- end H M D t o d a t e . H av in g be e n o n t h e mar ket for r oughly 4- 5 m o n t h s, t h ese nu mbe r s a re surprisingly high. As a r efer ence p o i n t , t h e o r ig in a l iPh o n e s o ld 1. 4 mi l l i on uni ts its fir st 3 m on t h s on t h e ma r ke t , a n d it is consider ed one of the m ost su c c e ssf u l t e c h pro du c t s o f al l ti me . Sony’s i nter nal go a l w a s t o s e ll o n e millio n u n it s in i ts fir st si x months, by m i d - Apr i l, a n d is o n t r a c k t o sur pass that for ecast. O ve r a l l , t h i s s a le s fig u re is a ve r y positi ve sign for the e n t i r e i n du st r y.

PSVR h a s s o l d ove r 9 1 5 , 0 0 0 units b e t we e n Oc to b e r 2 0 1 6 a n d F e br uar y 2 0 1 7 , m a k i n g it the b e s t - s e l l i n g high-e nd H M D t o date .

T h e su cc e ss o f PSV R is du e t o t h e pr evi ous instal l base of P S4 ( 5 3 . 4 m illio n PS4s wo r ldwide) , w ho coul d buy a VR s y st e m fo r $ 4 00, in s t e a d o f bu y in g an expensive computer p lu s a $ 6 0 0 - $ 800 Oc u lu s Rif t o r HT C Vive . PSVR comes with a 5. 7 in c h OLE D di splay panel, 1920×1080 resol u t i o n , a re f re s h r a t e o f u p t o 120hz, and a 100° field o f v i e w. Th e LE D s a ro u n d t h e PlayStation VR headset, on th e D u a l S h oc k 4 wire le s s c o n t ro l l er, and the Pl ayStation M ove m ot i o n c o n t ro lle r a re t r a c ked by the PlayStation C a m er a , pr ovidin g pre c is e c a pt u re of natur al movement in vi r t u a l sp a ce .

Qualcomm Snapdragon 835 VR DK

Qu a l co m m re c e n t ly in t ro du c e d its development ki t for i ts Snapdragon 835 mo bile pla t fo r m . T his dev ki t w i l l gi ve d e vel ope r s ac c e s s t o a n H M D that i s bui l t w ith the S n a pdr a g o n 8 35 mo bile pla t fo r m.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

“W i t h t h i s n e w V RD K , we ’re providi ng vir tual r eal i ty appl i cation developer s w i th ad v a n ce d t oo ls a n d t e c h n o lo g ie s to accel er ate a new gener ati on of VR ga m es, 3 6 0 - de g r ee VR v ide o s a n d a v a r ie t y of inter active educati on, enter pr i se , heal thcar e an d e n t e r t a i n me n t a pplic a t io n s ,” said Cr i stiano Amon, executi ve vice pr esi d en t, Qu a l co m m Te c h n o lo g ie s , In c . , a n d pr esi dent, Q CT. “We see gr eat potenti al for th e e x c i t i n g n e w e x pe r ie n c e s ma d e possibl e by tr uly mobile , untether ed vir tu al re a l i t y t h a t ’s a lway s c o n n e c t e d t o the inter net, and we’r e exci ted to help m ob i l e V R de vel ope r s mo re e f fic ie n t ly de li ver compelling and high-qual i ty exper i ences on up c o m i n g S n a pdr a g o n 835 V R-c a pabl e pr oducts.” T h e H M D d e v k it c o n s is t s o f fo u r m egapi xel ( 2560x1440) WQ HD AMO LED di sp l ay (two m eg a pixe ls pe r e ye ), s ix de gr ees of fr eedom, and eye tr acking. F or a m or e d e t a i l e d l oo k in t o t h e Qu a lc o mm Snapdr agon 835 VR DK, click here.

Snapdragon 835 VR Dev Kit Qualcomm

StarVR

T h e StarVR H M D is a pro du c t o f Star br eeze and Acer. T hey r ecently par tner ed w i th IM AX t o o f fe r a lo c a t io n ba s e d e n t er tainment zone . What differ entiates thi s hea d set i s t h a t i t h a s a 210 (h o r izo n t a l) x 130 ( ver tical) degr ees FOV compar ed to m ost o t h e r H M D s t h a t a re ro u g h ly 110 x 100. F ur ther, i t featur es 5K r esolution vi a d u al 5 .5 ” qu a d H D dis play, 6 de g re e s of fr eedom via outsi de-in tr acking, and r eti n al tr a c k i n g fo r fove a t e d re n de r in g . Thi s techni que i mpr oves r ender i ng per for man ce , by r e du ci n g t h e ima g e qu a lit y in the per i pher al vi si on. T he image r esol uti on , or s p e c i fi c a l ly t he a mo u n t o f de t a il, v ar i es acr oss the image accor di ng to one or m or e “fi xa t i on po in t s .” A fix a t io n po in t i ndi cates the highest r esol uti on r egi on of th e i m a g e a n d c o r re s po n ds t o t h e c e nter of the eye’s r eti na, the fovea. T hus, foveated re n d e r i n g w i ll re du c e s t h e re n de r ing wor kload.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

AR/MR HEAD MOUNTED DISPLAYS (HMDS) 11 A R g rea t e s t s t r e ngth i s i ts i nher ent d iffer ence fr om V R . Wher eas VR blo c k s o u t t he wo r l d a r ound it and is gener ally an isolated exper ienc e , A R ad d s d i g i t al over lays to r eal-wor ld exp er ience . Â I t i s a m e diu m t h a t le n ds it s e lf enti r ely to multi - tasking. A R H M D s e mpowe r u s e r s t o accomplish tasks in the v i r t u a l wor ld, wit h o u t be in g f ul ly separ ated fr om the physi c a l wo r ld. Th is t e c h n o lo g y has huge potenti al to t r a n sfor m t h e u t ilit y o f e ve r yday life . Soon, people w i l l be a ble t o do a ny t h in g t h ey do on a computer or sm a r t ph on e wit h o u t n e e din g to check their devi ce . User s of AR c a n a c c e s s e ma ils , d ocuments andÂ websites di r ec t ly f ro m t h e ir fie ld o f v ision, and visit hi stor ical l a n d m a r k s t o s e e wh a t t h e y lo o ked l i ke i n the past. HoloLens Microsoft

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

DAQRI

DAQRI t a r g et s A R a s a t o o l fo r cor por ati ons to use in fac t or i es, o pe r a t in g ro o ms , a n d ma ny other i ndustr i es. T hey c u r r en t ly h ave t h re e ma in pro ducts: the DAQ R I Smar t H e l m et , t h e DAQRI Sma r t G la s s e s, and the DAQ R I Smar t H UD. T h e DAQRI Sma r t He lme t is t h e ir pr emier AR pr oduct for th e wor k i n g wo r ld. It prov ide s data vi sual i zati on, gui ded wo r k i n st r u c t io n s , t e a m a s s is t a n c e , and mor e di r ectly onto th e l e n s, h e l p in g in c re a s e wo r ke r safety and effi ciency. T he S m a r t H el m et is powe re d by In tel â&#x20AC;&#x2122;s 6th gener ation m7 p r oc e sso r. I t als o fe a t u re s a h ig h s peed, w ide- angled tr acking c a m er a a n d a n In t e l Re a ls e n s e LR2 00 depth sensor. DAQ R I al so i n t eg r a t e d a t h e r ma l c a me r a that offer s per sistent p a ssi ve t h e r ma l mo n it o r in g o f in dustr i al equi pment. T h e DAQ RI Sma r t G la s s e s c o n t a in ver y si mi l ar featur es to t h e DAQRI Sma r t He lme t , bu t in a much smal l er and l i g h t we i g h t de s ig n . Th e g la s s e s fe a tur e a w i r el ess compute p a c k t h a t d i st r ibu t e s t h e we ig h t o f the headset. L a st ly, t h e DAQRI Sma r t HUD is an AR heads up displ ay fo r a u t om o bile s . Th e t e c h n o lo g y proj ect digi tal i nfor mati on o n t o t h e veh ic le â&#x20AC;&#x2122;s win ds h ie ld, a n d is cur r ently used i n over 1 5 0 , 0 0 0 ca r s .

contributions@vrarproject.com

VR/AR Project

Smart Helmet Daqri

New Amsterdam Media

Intel’s Project Alloy

Intel’s Project Alloy is a s e lf -c o n t a in e d headset that uses I ntel’s R e a l S e n se de pt h s e n s o r s t o a n a ly ze the envi r onment and p r ov i d e m ot i o n t r a c k in g s imila r t o Micr osoft’s “insi de-out” s y st em . Th e h e a ds e t fe a t u re s s imilar 1080p di splays and 90 fp s a n d a b a tt e r y t h a t In t e l s t a t e s wi l l last for “hour s”. Un l i ke t h e V ive a n d Rif t , In t e l’s h eadset does not r equir e b e i n g t et h e r e d t o a c o mpu t e r. A ls o, I ntel’s R ealSense depth s e n sor s a dd a n e w e le me n t t o v ir tual r eality exper i ences th a t t h e V i ve a n d Rif t la c k , ma k in g it a cl oser to a mi xed re a l i t y h ea d set . Th e s e n s o r s c a n c reate a 3D r epr esentati on o f t h e wo r l d in f ro n t o f it . Fo r e x ampl e , if a user i s about to r u n i n t o a c o f fe e t a ble , t h e h eadset cr eates a vi r tual re pr e sen t a t i on o f t h e t a ble , pre venti ng a shar p edge to th e sh i n s. F u r t h e r, t h is t e c h n o lo g y al l ow s the objects to be i nc o r po r a t e d in t o t h e V R e x pe r ie n ce . I n the VR exper i ence , th e co f fe e t a ble o r c h a ir c a n be s imulated to an obj ect of s i m i l a r si ze a n d s pa c e . L a st ly, t h e R e a lSe n s e t e c h n o lo g y a l so identi fi es both other p e o pl e a n d a u s e r’s own bo dy t o i ncor por ate i nto the e x p e r i en ce . Th is h e lps ma ke t h e exper i ence even mor e i m m er si ve . I n t e l pla n s t o pa r t n e r wi th headset manufactur es to i m pl em e n t t h e t e c h n o lo g y in t o f utur e har dw ar e . Headsets usi n g t h e t ec h n o lo g y a re e x pe c t e d to be ar ound $1,000.

Magic Leap

Magic Leap i s o n e o f t h e mo re myster i ous companies in th e i n du st r y. Wit h n e a r ly $1. 4B i n funding and a $4.5B va l u a t i on , M ag ic Le a p is c o n s ide red an industr y l eader — ye t t h ey h ave n o pro du c t o n t h e mar ketplace . Al though m o st o f t h ei r de ve lo pme n t a l s e c rets ar e under w r aps, the c o m p a ny i s wo r k in g o n a t e c h n o logy that beams photons d ir ec t ly i n t o a we a re r’s re t in a . Magi c Leap devel oped a s i l i c o n l i g h t - fie ld c h ip t h a t C E O Rony Abovitz descr i bes as a “ t h r ee d ime n s io n a l wave … c omponent that has ver y s m a l l st r u ct u re s in it , a n d t h e y ma nage the flow of photons th a t u l t i m a t e ly c re a t e a dig it a l lig h t field signal .”

Th e s e n so r s ca n cre a t e a 3D re p re s e n t a tio n of t h e wo r ld in f ro n t of i t . Fo r e x a m p l e , if a u s e r i s a b o ut to r u n i n t o a c o ffe e t a bl e , t h e h e ad s e t cre a t e s a vir tual re p re s e n t atio n of t h e t able , p re ve n t i n g a e d ge t o t h e s hins .

To h el p g en e r a t e bu zz fo r t h e ir new technol ogy w i th i m m er si ve co n t e n t , Ma g ic Le a p h a s joined for ces w ith WETA Wor k sh op, Pe t e r Ja c k s o n ’s Ne w Zealand- based special e f fe c t s wo r k sh o p.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

Meta 2

T h e Meta 2 de ve lo pme n t k it is av a il able now for $949 and uses a si m i l a r t e c h n o lo g y t o t h e Hol oLens that pr oj ects d ig i t a l i m a g es o n t o t h e g la s s h e a dwear. T he Meta 2’s fr ont faci n g d e pt h s e n s o r s h ave a 270-degr ee field of vision that e n a bl es t h e dig it a l pro je c t e d ima ges to appear as i f they ar e pa r t o f th e re a l wo r ld. Th e s e sensor s also tr ack hand m ove m e n t s, allowin g u s e r s t o in t er act w i th the pr oj ected i m a g e s. W i t h a 9 0 ° fie ld o f v ie w, t h e Meta 2 has near ly doubl e th a t of t h e H o lo Le n s , h owe ve r, t he Meta r equi r es a PC c o n n e c t i o n t o o pe r a t e . Th e c o m pany pl ans to cr eate un t e t h e r e d h e a ds e t s by t h e e n d o f next year.

Meta 2 Meta

Microsoft HoloLens

Microsoft HoloLens is a s e lf -c o n t a in ed AR headset power ed by W i n d ow s 10. Th e de v ic e is o n e of the fi r st to featur e an “ i n si d e - ou t t r a c k in g s y s t e m”, m eani ng that it does not d e pe n d o n ex t e r n a l s e n s o r s t o tr ack the user and the s u r r ou n di n g wo r ld. Us in g a nu mbe r of camer as and depth s e n sor s, t h e de v ic e pro je c t s ima g es onto the lenses that ap pe a r a s a p a r t o f t h e re a l wo r ld. T he headset is power ed by M i cr o sof t ’s c u s t o m bu ild Ho lo gr aphic Pr ocessing U ni t, an d l a st s a bo u t 2-3 h o u r s o f a c ti ve use . T he HoloLens d e vel opm e n t k it is pr ic e d a ro u n d $ 3,000.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

ODG Ea r l i er t h i s ye a r, awa rd-win n in g Osterhout Design Group (OD G ) a n n ou n c e d t wo n e w model s to thei r R -ser i es s m a r t g l a sse s: t h e R8 a n d R9. Th e sleek glasses featur e Qu a l co m m ’s n e w Sn a pdr a g o n 835 8-cor e pr ocessor. T he $1 , 0 0 0 R8 is a ime d a t c o n s u mer s and featur es 720p re sol u t i o n , du a l 1080p s t e re o c a mer as, and a 40- degr ee fi el d o f v i ew. Th e $1, 800 R9 is tar geted mor e tow ar ds d e vel ope r s a n d fe a t u re s 1080p resol uti on, a 13MP fr ont fac i n g ca m er a (4K a t 60 FPS t o 1080p at 120 FPS) , and a 5 0 - d e g r ee fi e ld o f v ie w. Th e g la s s es ar e contr olled usi ng a s m a r t p h on e o r B lu e t o o t h a c c e s s o r i es. Vrvana Totem Vr vana Totem de ve lo pe d a wire less headset that can s e a m l essly d is play bo t h Mixe d Re al i ty and VR , w hich they c a l l t h e “ M at r ix Mo de ”. Th e h e adset featur es a 1440p d ispl ay, a h i g h e r re s o lu t io n t h a n both the O cul us and the H TC ’s o f fe r i ng , a n d t h e re f r a c t ive aspher i cal lens combo p r odu ce s a fie ld o f v ie w o f 120-degr ees. Vr vana Totem has not r e l ea se d t h e pro du c t fo r de vel oper s yet, and ther e is no wor d o n p r ic in g . Zappar Zappar i s wo r k in g o n a pro du c t s imilar to Google Car dboar d, bu t fo r AR. Us in g yo u r c e llph o n e , the car dboar d headset p r ov i d e s a n a f fo rda ble way t o e x pe r i ence Augmented R eality. T h e AR e x pe r ie n c e is powe re d by Zappar ’s smar tphone ap p a n d u ses t h e ph o n e ’s c a me r a to augment obj ects i nto a u se r ’s su r r o u n din g s . Za ppa r de ve loped mar ker s to put on s u r f a ce s a r ou n d a ro o m t o h e lp t h e app under stand depth b e t t e r. Z a p pa r h o pe s t o s e ll t h is bundled exper i ence for as little as $30.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

SPATIAL AUDIO

Sp a t ia l a u d i o i s e s s e n t i a l fo r i mm er sive exp er iences. Cr eating be lie v a ble 3 D a ud i o c a n b e mor e d ifficult than cr eating the visual env iro n m e n t . T h e tr ue s t fo r m of 3D aud io can b e cap tur ed thr ough ei t h er b in a u r a l o r a mb i s o n i c recor d ing. An a m b i so n ic mic ro ph o n e is essenti ally an ar r ay o f seve r a l sma ll mic ro ph o n e s p ointing in differ ent d ir ec t i o n s, m a k in g u p 360°, wh ile a binaur al micr ophone ha s t wo m i c ro ph o n e s o n e a c h s ide , simul ati ng a human hea d w i t h t wo e a r s . To m a t c h t h e e nv iro n me n t a l objects w i th their c o r r esp o n d i ng a u dit o r y c u e s in t h e edi t bay, an obj ectb a sed a u d i o t e c h n iqu e mu s t be used, and i s the next e vol u t i o n i n au dio mix in g a n d re c or di ng. Obj ec t - ba se d a u dio h o lds t h e s e audi o obj ects as i ndi v i d u a l a ss e t s , a n d a llows e a ch asset to contain m e t a da t a de s c r ibin g wh e re a n d h ow it shoul d play i n a t h r e e - d i m en s io n a l s pa c e . Pre v iously, sur r ound sound m i xi n g w a s l imit e d by re c o rdin g a nd then mixing audio i nt o c e r t a i n c h a n n e ls , be it 5. 1, 7.1, etc . Essential ly, o b j ec t - ba se d a u dio s imu la t e s s o u n d s i n a 3D envir onment e x a c t ly t h e way t h e a r t is t in t e n de d.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

Obj ec t - ba se d a u dio e n a ble s s o und engineer s to give i ndi v i d u a l obje c t s t h e ir own dis c re t e audi o “mi x” w ith spati al i nfor m a t i on , a n d e ve n e n a ble s n e w hei ght channel s to make s o u n d s se e m ove rh e a d. A s a n e x a m pl e , a voi ce hear d above a pe r so n c a n c u e t h e m t o lo o k u p, in w hi ch case the spati al p o si t i o n i n g o f t h e vo ic e ’s s o u rc e t r avels fr om above to the fron t of t h e pe r s o n . Lis t e n e r s c a n p er ceive sound el ements i n t h e f r o n t , be h in d, f ro m a bove or bel ow and be tr uly i m m er se d i n a ny v ir t u a l e nv iro n ment, w ith a stunni ng l evel o f r ea l i t y t h a t c re a t e s a t r u e s e n s e of pr esence . In V R, i t i s cr it ic a l t h a t s o n ic pe r spective of an object i n s p a ce i s pr e ser ve d, o r t h e illu s io n of VR cr umbles. Accur ate s o n i c c u es are e s s e n t ia l s o t h a t user s know w her e to e x p l or e a n d fo c u s , a n d c a n fo llow the stor yline . Ul t i m a t ely, w h e n e a c h s o n ic s o u rc e i s encoded into its ow n d isc r e t e ch a n n e l wit h in in t h e s o undtr ack, the exper i ence b e c o m e s m ore in t e r a c t ive a n d in fini tely mor e bel i evable .

Dolby

Dolby Atmos fo r mo bile de v ic e s t a kes the r enow ned “ci nema s o u n d ,” a n d pu t s it in bu ilt -in s pe a ker s and headphones. T he e f fe c t s a r e st u n n in g , re n de r in g a u di o that seems to fl ow al l a r o u n d you , in c lu din g ove rh e a d. Dol by’s obj ect based At m os so l u t i o n is ve r y c o mpa r a ble to DT S:X. It p r ov i d e s c r is pe r dia lo g u e , a mor e envel opi ng field of s o u n d , g r e a t e r s u bt le t y a n d nu a n ce , maxi mi zed l oudness w i t h ou t d i st or t io n , a n d c o n s is t e n t pl ayback volume for a w i d e v a r i et y o f c o n t e n t . D ol by At m os h a s t h e po t e n t ia l t o r evol uti oni ze the sound o f mu si c , g a m e s , a n d t e le v is io n s h ow s, and is enabled in the l at est L en ovo a n d A ma zo n mo bile devi ces. I t functions thr u any pa i r o f h ea dph o n e s o r, t o a s lightly l esser extent, builti n st e r e o sp e a ke r s .

DTS

D T S h a s de ve lo pe d a n o bje c t -ba s ed audio solution called DTS:X, a n d c o mple me n t a r y D TS Headphone:X technology, w h i c h si mu l at e s u r ro u n d s o u n d f rom 11 discr ete sour ces. H e a d ph on es e qu ippe d wit h t h is technology ar e like “su r r o u n d so u n d t o g o,” a n d pre s ent the most i mmer sive an d a c c u r a t e s o n ic s pa c e po s s ible . DT S:X is bui l t atop MDA (M u l t i D i m en s io n a l A u dio ), a n o pen pl atfor m for objectb a sed a u di o t h a t D TS h a s ma de av ailable to content maker s l i c e n se fee - f re e . contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

Fraunhofer

Fr aunhofer’s Cingo offer s quite a different solution, making use o f h e a d - t r a ck in g t o a dv a n c e t h e e nter tainment exper i ence o n h ea d ph on e s . C in g o is av a ila ble fr om F r aunhofer as a p r odu ct - r e a d y s o f t wa re imple me n tation for mobi l e device m a nu f a ct u r e r s , c h ip s e t ve n do r s a n d pr ovi der s of mul timedi a ser vices. C i n g o s u ppo rt s re n de rin g of 3D audi o content w ith for m a t s l i k e D TS: X o r A t mo s , wh ic h add a height di mension to t h e so u n d ima g e , s u c h a s 9. 1, 11 .1. Ci ngo also uti l i zes a s p e c i a l equ a liza t io n fe a t u re t o c o m pensate for the lack of b a ss u su a l ly e n c o u n t e re d wit h s mar tphones and tabl ets, as we l l a s l o u d n e s s o pt imiza t io n t o deliver a natur al and clear s o u n d e ven in n o is y o f e nv iro n ments. Both featur es ar e i m p l em e n t e d t o pre ve n t u n ple a s a n t fr equenci es, distor tions, an d cl i ppi n g , pa r t ic u la r ly a dv a n t a g eous for content that is ve r y sof t o r o f h ig h dy n a mic r a n g e . C i n g o i s en able d o n Go o g le ’s Ne xus and Pixel; LG’s 360 V R; Al ca t e l / T C L C o mmu n ic a t io n ’s VI SI O N VR ; and Samsung G e a r V R de v ic e s .

Dysonics

D yson i c s h a s a u n iqu e V R s o lu t ion called Rondo360. T he R o n d o M i c , a n u lt r a -h ig h fide lit y 360° mic ar r ay, captur es s o u n d env i r on me n t s wit h in c re dible accur acy and enables l i ve br o a d c a st in g in a n a t ive 360° binaur al for mat. R ondo360 i nt eg r a t es w it h a ll ma jo r DAWs a n d wor kflow s to yield 3D au di o. M o r e ove r, re g a rdle s s o f yo ur r ecor ding setup, you c a n l ever a g e t h e Ro n do 360’s in t u itive spati al mixing tool s an d r e a l - t i m e mo t io n -t r a c ke d 36 0° pr eview ing w i thout d isr u p t i n g you r po s t pro du c t io n wor kflow. D yson i c s p r ov ide s 360 v ide o player s w ith nati ve suppor t fo r t h ei r r u n -t ime a u dio e n g in e fo r ster eo, 5.1, Ambi soni cs, an d M ot i o n - Tr a c ke d B in a u r a l (M TB) r ender i ng, along w i th l i ve st r ea m i n g s u ppo r t . Th e y a ls o provi de an SDK for custom i nt eg r a t i o n i n t o iOS, A n dro id, OSX , & Wi ndow s apps.

OSSIC

OSSIC X h ea dph o n e s o f fe r h ig h -q uality and i ndi vidual lytai l o r e d 3 D a u dio. Th e y c a libr a t e t o an indivi dual’s head by c a l c u l a t i n g t he ir po s it io n in s pa c e , head si ze , and ear shape . It a l so c o m e s f u lly in t e g r a t e d wit h head tr acking, a mul tid r i ve r a r r ay (wh ic h dire c t s s o u n d to the cor r ect por tion o f t h e e a r ) , a bu ilt -in mic ro ph o n e , U SB + AU X connection,

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

an d a 1 0 / h r. B a t t e r y Life . OSSIC X suppor ts ster eo, 5.1, 7 .1 , a n d be yo n d. Th e la t e s t it e r a ti on of the headphones i s av a i l a bl e fo r pre o rde r fo r $299, and w ill r etai l at $499 th e r e a f t er.

VisiSonics

RealSpace3D is V is iSo n ic ’s 3D a u dio solution that lever ages th e i r p r o pr i et a r y a lg o r it h ms a n d har dw ar e , w hi ch i s being i nc o r po r a t e d in g a min g e n g in e s . The R eal Space 3D Gami ng En g i n e en a bl es g a me a n d V R de s ig ner s to cr eate extr emely i m m er si ve a u dio, wh ile be in g e f fic ient enough to wor k w i th m i n i m a l C P U n e e ds . Fu r t h e r, t h e RealSpace 3D After Effects En g i n e a l l ow s u s e r s t o mix a 3D audi o scene offline via a V S T p l u g - i n arc h it e c t u re . A n a t ive libr ar y and U ni ty pl ugi n ar e a l so av a i la ble .

Sennheiser

S e n n h ei se r ’s A M B E O V R Mic is a n ambisonic micr ophone th a t a l l ow s yo u t o c a pt u re t h re e di mensions of sound from a si n g l e po in t . Th is re s u lt s in a fully spher i cal sound to c o m pl em e n t imme r s ive c o n t e nt. Sennhei ser has also d e vel ope d a c o mpre h e n s ive pro du ct package i ncl udi ng: A-B e n co de r so f t wa re t h a t wo r k s a s a pl ugi n, w hich seamlessly i nt eg r a t es w it h a ny po s t pro du c t ion. Ac co r di n g t o Se n n h e is e r, Th e AMBEO VR Mic “deliver s A- for m a t , a r aw 4-c h a n n e l o u t pu t that has to be conver ted i nt o a n e w se t o f 4 c h a n n e ls , t h e Ambi soni cs B- for mat. T hi s i s do n e by t h e s pe c ific a lly de s ig ned Sennheiser AMBEO A- B for m a t c o nve r t e r plu g in , which i s avai l abl e as fr ee d ow n l oa d fo r V ST, AU, a n d A A X for mat for your pr efer r ed D i g i t a l A u d i o Wo r k s t a t io n fo r bo t h PC and Mac . B-for mat i s a W, X , Y, Z re pre s e n t a t io n o f t h e sound field ar ound the m i c r o ph on e . W be in g t h e s u m o f a ll 4 capsules, w her eas X, Y an d Z a r e t h re e v ir t u a l bi-dire c t io nal micr ophone patter ns repr e sen t i n g f ro n t / ba c k , le f t / r ig h t and up/dow n. T hus, any d ir e c t i o n f r o m t h e mic ro ph o n e c an be auditi oned by the l i st en er du r i n g play ba c k o f A mbis o nics B .”

Amebo VR Mic Sennheiser

THX

T h e THX AAA chip is a n A SIC c h ip that’s bei ng made i n p a r t n er sh i p w it h Tr ia d Se mic o n du c tor, a company w ho also wo r ked w i t h Va lve t o c re a t e c o n s t ituents of the HT C Vi ve’s L i g h t h ou se t r a c k in g t e c h n o lo g y. THX cl aims that T HX AAA o f fe r s “ t h e wo r ld’s lowe s t le ve ls of distor tion and noi se ,” al o n g w i t h “ i n c re dibly low powe r c onsumption.”

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

HAPTICS, ACCESSORIES, AND CONTROL SYSTEMS 13 Ha pt ic t e c hno l o g y r e c r e a te s t he sense of touch by ap p lying for ces, vibr a t io n s, o r mo t i o n s to a u s er.

HAPTIC SUITS

We a r e a l l f a milia r wit h h a pt ic fe edback fr om vibr ating c a l l s a n d S MS a le r t s . A dv a n c e d applicati on of hapti c fe e dba ck u sin g e le c t ro -ma g n e t ic sti mulation ( EMS) or o t h e r t e c h n o lo g y s u c h a s s o u n d w aves, can possibly m i m i c t h e fe e lin g o f dif fe re n t o bje cts or substances for use r s i n V R.

TeslaSuit

T h e Tesl a S u i t is a c o mple t e f u ll bo d y sui t w ith 68 haptic p o i n t s a n d 1 4 in s ide -o u t mo t io n captur e sensor s as we l l a s 1 0 “ clima t e po in t s ” t h a t s imul ate temper atur es from 1 0 - 4 0 ºC (50-104 º F). It is c ompatibl e w ith U nity an d Un r ea l E n g in e fo r in t e g r a t in g API , as wel l as w ith W i n dow s OS, A n dro id, iOS, XB OX and PlayStation.

AxonVR HaptX

AxonVR HaptX is n o t a s u it bu t r a t her mor e of a wor king tec h n o l og i c a l c o n c e pt t h a t c u r re n tly r esembl es a hefty b ox. T h e sys t e m u t ilize s h a pt ic t exti l es that si mul ate l i fel i ke t ou ch , a llowin g o n e t o feel textur e , shape , m o t i o n , v i b r a t io n , a n d t e mpe r a t u re . T he effect i s cr eated by h u n d r e ds o f s ma ll e le c t ro n ic actuator s that apply p r essu r e w h ile c o n s is t e n t ly ma n ipulating temper atur e . A f u l l bo dy h a pt ic s u it is c u r re n t ly i n development and tar g et ed t ow a rds e n t e r pr is e c u s t o m er s.

TeslaSuit

HardlightVR

HardlightVR by Nu lls pa c e V R is a n upper- body vest w h i c h u se s 16 h a pt ic pa ds t o e lectr o- stimul ate the fe e l o f i m pa ct a n d a llows fo r pro g r amming of 16 families o f v a r i a bl e a n d c o mbin a ble e f fe c ts. I t al so integr ates i nsi de - ou t t r ac k in g t o e n a ble a pa r ti al avatar in vir tual s p a ce , i n st ea d o f o n ly s imu la t in g h ands fl oating i n space w i t h h a n d h eld c o n t ro lle r s . Nu lls pa ce is acti vely wor king w i t h AA A d e ve lo pe r s t o ma ke t h e suit compati ble w i th p o pu l a r V R g a me s . HardlightVR NullspaceVR

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

Rez Infinite Synesthesia Suit

T h e Rez Infinite Synesthesia Suit is n o t an actual suit but r ather a c ol l e c t i o n of s t r a ps t ig h t e n e d a ro und the body that utilizes 2 6 a c t u a t o r s t o de live r e le c t r ic s t i mulation. T he system is i n e a r ly st a g e s o f de ve lo pme n t , bu t is cur r ently compati bl e w i t h P S V R.

OMNIDIRECTIONAL TREADMILLS Om n i di r ec t i o n a l t re a dmills (OD T s) offer a sol uti on to th e l a c k of spa c e fo r playe r movement in vi r tual wor lds; th e y e n a bl e move me n t wit h in a small space to achieve l i m i t l e ss a n d c o n t inu o u s move me nt i n VR . Al though it is s t i l l t o o e a r ly t o t e ll h ow e x a c t ly VR systems w ill wor k in th e f u t u r e t o e n a ble f u ll r a n g e o f moti on ( either thr ough m or e so ph i stic a t e d OD Ts , s o me s y stem r elying on galvani c ve st i bu l a r st i mu la t io n , o r ju s t a n a c cur ately desi gned open s p a c e t h a t mimic s t h e v ir t u a l o n e ), ther e ar e sever al e a r ly sol u t i o n s a lre a dy o n t h e ma r ket.

Virtuix Omni

T h e Vir tuix Omni is t h e mo s t we ll-know n O D T w i t h a h ig h -pro file Kic kstar ter l au n c h i n 2 013. V ir t u ix pa r t n e re d ear ly wit h mu l t i p l e h ig h pro file g a me devel oper s to i n t eg r a t e t h e Omn i in t o e x is ti ng games i n c l u di n g C ou n t e r s t r ike , G TA , Fal l out, and Mi n ec r a f t . T h e O m n i ut ilize s a low f r ic t io n sur face to “w a l k ” ( sl i p) in a ny dire c t io n , we ar i ng special s h o e s w h i l e a c irc u la r r in g a ro u n d t h e w ai st keeps th e u se r i n o n e pla c e . A lt h o u g h t he exper i ence fe e l s m or e l i ke s lippin g o n a s u rf a c e , the O mni tr i cks th e b o dy i n t o t h in k in g it is wa lk in g . Vir tui x tar gets VR ar ca d e s a n d ot h e r in t e re s t e d e n t e r pr i se customer s. T he c om pa ny r e c e n t ly a n n o u n c e d a for mal par tner ship w i th HT C t o f u r t he r de ve lo p c o n t e n t that takes advantage of th e syst e m .

Virtuix Omni

Kat Walk VR

Kat Walk VR a ims fo r a le s s t ether ed and r estr i cti ve e x p e r i en ce wit h a n in de pe n de n t s uppor t str uctur e and no r i n g s o r c o l u mn s . It is a s pe c ia l h igh-fr i cti on, concave floor th a t , w i t h a c o n s t a n t fo rc e o f ro l ling fr i cti on, makes you fe e l l i ke you ’re wa lk in g o n re a l g r ound, r ather than “slip w a l k i n g ” l i ke t h e Omn i. K a t Wa lk is ai mi ng thei r pr oduct at e n t e r pr i se c u s t o me r s . contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

Vue VR

T h e Vue VR t r e a dmill is ve r y s imila r i n design to Kat Walk VR . Vu e c a n be bo u g h t fo r $699 by in t er ested ear ly adopter s.

EXERCISE AND FITNESS V R a n d AR bo t h h ave ple n t y o f po tenti al in r evol uti oni zi ng e xe r c i se a n d fit n e s s u s in g imme r s ion and gamificati on. T he fi t n ess i n d u st r y will c e r t a in ly s pu r development in thi s ar ea as p o pu l a r fit n e s s c o mple xe s lo o k to lur e member s w ith c o m p e l l i n g n e w e x pe r ie n c e s .

Gogi Play

T h e Goji Play ($119)Â de s ig n e d by Blue Goji w as or i ginally c r ea t e d for sma r t ph o n e /t a ble t Â iOS and Andr oid systems to b e u se d i n t an de m wit h a n e llipt ic al , stepper, or stati onar y b ike . I t c o n si st s o f t wo s t r a ps wit h buttons that w r ap ar ound th e h a n d l eba r s , a n d t h e u s e r r u n s the companion app w ith a l i br a r y o f ga me s t h a t u pg r a de t h e gener i c car dio r outi ne i nt o a n i m m er s ive v is u a l t r a in in g e xper i ence .

Icaros Icaros GmbH

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

Icaros GmbH

T h e Icaros i s a c o mple t e V R e xe rc i se machine that alr eady e xi st s i n ove r 200 h ig h -e n d fit n e s s center s i n Tokyo, London an d seve r a l o t h e r c it ie s , wit h a cost of appr oxi mately $1 0 , 0 0 0 pe r s y s t e m. A lt h o u g h c u s tomer s r epor t a l ack of c on t e n t for ge t t in g t h ro u g h e xe rc i ses, the machine show s p r o m i se i n p rov idin g g y m me mber s w ith an al ter native wor ko u t r o u t in e , wit h t h e pro mise of mor e content to c om e . A c o n su me r ve r s io n fo r h o me use i s being developed to be r e l ea se d by t h e be g in n in g o f next year, w i th a tar get MS R P o f a r o u n d $2, 000.

VirZOOM

VirZOOM ( $ 3 99) is a c o mple t e h a r dw ar e kit consi sting of a sp i n n i n g bi ke wit h bu t t o n s o n t h e handl ebar s, plus games an d sof t w a r e , a lt h o u g h it re qu ire s a separ ately sold HMD. I t ut i l i ze s i n t e g r a t e d wire le s s s e n s o r s to tr ack user movements an d sen so r s me a s u r in g pe da lin g s p eed and r egi ster i ng the m ove m e n t i n t h e v ir t u a l wo r ld. St eer i ng wor ks by leaning o n e’s bo dy t o t h e le f t a n d r ig h t . Action buttons, tr i gger s an d d- p a d o n t h e h a n dle ba r s o ffer var i ous inputs for ga m e s t h a t ca n be de ve lo pe d fo r t he system. Vir ZO O M has al r ea d y c r e a te d mu lt iple e x pe r ie n ces w her e user s can r i de a Pe g a su s, ope r a t e a t a n k o r r a c e a car, w hile playing mini ga m es t o c o lle c t it e ms a n d a c h ie ve hi gher scor es.

GALVANIC VESTIBULAR STIMULATION (GVS) G V S i s t h e p ro c e s s o f s t imu la t in g ner ves i n the ear that m a i n t a i n ba l a n c e , t o in du c e a n a t u r al feel i ng of movement. G V S m ay e l i min a t e mo t io n s ic k n e s s in VR w hi l e also enabl i ng th e fee l i n g o f mo t io n . GV S c a n t r ic k a subj ect into thinking th e y ’r e m ov in g wh e n t h e y a re n ot, and may effectively d e l i ver m o t i o n fe e dba c k fo r a u s e r in VR . S a m su n g ’s Entrim 4D h e a dph o n e s ar e the fi r st pr oduct to take a d v a n t a g e o f GV S. Th e y a re capabl e of deliver i ng th e se n sa t i on o f 30 dis t in c t pa t t er ns of movement. T he he a d ph on es we re a n n o u n c e d a t SXSW 2016 but ar e sti l l in e a r ly de vel opme n t .

ADVANCED MOVEMENT TRACKING Alt h o u g h t h e h a n dh e ld c o n t ro ller s used w i th the Vive an d R i f t m ay be mo re n a t u r a l t han the pr evi ously used contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

tr a di t i on a l g a me pa ds , t h e y a re n o t the best w ay to contr ol i nt er a ct i o n i n v ir t u a l re a lit y. A c c u r ate hand and eye tr acki ng (a s we l l a s voic e c o n t ro l) a re t h e n ext steps tow ar ds natur al c o n t r ol i n V R . Mu lt iple c o mpa n ie s including F acebook and H TC a r e wor k in g o n in t e g r a t in g t h is technol ogy i n the next gen er a t i on of h e a ds e t s .

Leap Motion

Leap MotionÂ h a s pio n e e re d t h e development of accur ate ha n d t r a c k i n g a n d c o n t ro l fo r V R s y stems, most notably open s o u r c i n g i t s h a rdwa re fo r o t h e r de veloper s w i th the O SVR ki t . Over 2 0 0 k de ve lo pe r s a re a lready wor king w ith Leap M o t i o n . T h e h a rdwa re it s e lf is e s s e nti ally a smal l i nside- out m o t i o n t r a c k in g c a me r a t h a t c a n be attached to a tether ed o r m o bi l e h e a ds e t a n d t r a c k s h a nds at up to 200 fr ames p e r sec o n d , g iv in g u s e r s a 150Â° fiel d of vi ew w i th r oughly 8 c u b i c fe e t o f in t e r a c t ive 3D s p ace . T he technol ogy i s s t i l l d e vel opin g a n d is a ls o a pplic a ble i n augmented r eal i ty s y st em s t h a t u t ilize a ny t y pe o f g e stur e based input. D evel opm e n ts in a dv a n c e d in s ide -out tr acking benefit not o n ly V R bu t a ls o a ll t y pe s o f A R systems and as VR and AR sy n ch r on ize in t o a s y n e r g e t ic m ixed r eality, so w ill the s y st em we u se t o c o n t ro l it (h in t : it w i l l not be handhel d) .

contributions@vrarproject.com

VR/AR Project

Leap Motion

New Amsterdam Media

CAMERAS AND CAPTURE SYSTEMS 14 Th e re a r e fo u r ty p e s o f c a p tur e solutions at p lay: the custom r ig, t h e m a nufac tur e d s o l u t i o n , the p r op r ietar y custom solution, and t he D IY k it . Pa r a l l el s wit h “t r a dit io n a l” c in ematogr aphy r emain i n t a c t i n V R: t h e re is n o o n e c a m er a package to sui t al l c o n d i t i on s. Sh o o t in g c o n dit io ns, lighti ng, subj ect and st y l e a l l co me in t o play wh e n c h oosing the best sol uti on. S o m e o f t h e s e s o lu t io n s a re sold or r ented. For an e x c e l l e n t ove r v ie w o f wh a t is cur r ently avai l abl e click here.

Custom Rigs

Th i s i s w he re 360/V R c a pt u re began: innovator s took o f f - t h e- sh e lf c a me r a s a n d c o mponents to assemble h a r n esse s a n d 360 c a pt u re r ig s. I n the ear ly days, thi s w a s d o n e wit h G o Pro s a n d 3D pr i nted pl asti c har nesses t h a t wou l d prov ide t h e c o r re c t geometr y to cr eate the de si r e d c a pt u re e f fe c t (180, 270, full 360, or ster eoscopic ver si on s of t h e s a me ). Th e sophisti cation has vastly i n c r e a se d t o 16 8K Re d c a mer as har nessed together. S o m e of t h e s e s o lu t io n s a re built to be sol d or r ented; som e a r e ma de in -h o u s e by producti on compani es. Most popular camera rigs: 1. RED 2. SONY ALPHA 3. GoPro (Google Jump) 4. Nikon/Cannon DSLR

Camera Rig RED

5. Black Magic 6. ZCam

The Manufactured Solution

A s t h e me diu m c o n t inu e s t o evolve we’r e seeing a v a r i et y of c o mpa n ie s ma k in g self- contained 180° and 3 6 0 ° c a m e r a s . Th e s e a re e n g ineer ed fr om the gr ound u p, t o employ ba s e c o mpo n e nts to cr eate a unique c a m er a so lu t io n . Th e down s ide i s the r equi r ement of c o n si d e r a ble e n g in e e r in g a n d manufactur i ng exper ti se . Th e c a m er a s , dis c u s s e d be low ar e mor e consumer to “ pr o - su m er” g r a de , a n d a re me ant to be sol d at scale . contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

In so m e c a se s , t h e s e c a me r a s o lu t ions ar e al l - i n-one— fr om c a p t u r e t o st it c h in g t o o u t pu t . O n the hi gh end of the s p e c t r u m i n th is c a t e g o r y, t h e re is the J aunt O ne or Nokia Ozo, w h i ch a re ma nu f a c t u re d u n i ts but r equir e exter nal s o l u t i on s for o pe r a t io n s a n d c o n t e n t assembly. ( F or i nstance , fo r J a u n t O n e , a s s e mbly is c lo u d based stitchi ng.) H e r e i s a q u i c k bre a k down o f s e ve r a l manufactur ed solutions: 1. 360 Fly a. 4K b. Full 360 horizontal, 240 degrees ver tical FOV c. 64GB internal memor y d. Single lens, no seams = no stitching e. Can shoot in 360 and standard 16:9 f. Dust and shockproof g. Action cam mount accessor y $400 2. Samsung Gear 360 a. 2 F2.0 180-degree fish eye lenses, b. Have the option of disabling one lens for a 180-degree shot instead c. 25.9 Megapixels, 1920 HD d. Cer tified IP53 for protection against harmful deposits of dust and against water sprayed at any angle up to 60º from the ver tical e. Compatible only with latest Samsung Galaxy phones f. Requires external memor y g. $350

Gear 360 Camera Samsung

3. Ricoh Theta S a. 2 megapixels, 1080p HD, 30 FPS b. Internal memor y 8GB c. 2 F2.0 180-degree fish eye lenses d. Candy Bar design emphasizes por tability e. Special features: noise reduction, DR compensation, HDR Rendering f. 25 min. max recording time for one video g. Live streaming capability h. High compatibility with social networks i. IOS and Android suppor t j. $325 k. New Ricoh Theta SC i. maxes out at 5 min. record for a shot and no live streaming capabilites ii. $300

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

4. LG 360 a. Dual 13 Megapixel lenses b. 2K c. Requires external memor y d. $200 5. Vuze a. $800 b. Comes with mini tripod and Homido HMD c. 8 lenses d. Proprietar y stitching technique e. Vuze Studio 6. Insta360 Pro a. 6 200° fisheye lenses b. 4K @ 30fps (real time stitching) or 8K @ 30fps (post-production stitching) c. 4K @ 25fps (real time stitch) or 6K @ 30fps (post stitch) for 3D stereoscopic d. Suppor ts H.264 and H.265 video codecs e. 2x microphones w/ external microphone suppor t f. Optimized stabilization capabilities for action shots g. Suppor ts livestreaming to FB, YouTube, Periscope h. $3,499

LG360 LG

7. Z camera S1 a. 4 190° fisheye lenses b. 4K @ 60fps or 6K @ 30fps c. 2 hour recording time d. 4x directional mono microphones e. Includes stitching software f. $2,499 8. Live Planet a. 16x 180° lenses b. 4K @ 30fps in 3D stereoscopic c. Suppor ts H.264 and H.265 video codecs d. $9,995

Orah 4i Orah

9. Orah 4i a. 4 fish eye lenses b. Built in stabilization and horizon correction c. 30 fps 4k d. Live processing unit AKA “Stitching box” enables live stream broadcast with push of a button e. 4 HDR microphones; ambisonic 3D sound capture f. $2,595 pre-order, $3,595 MSRP contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

g. All metal body—thermally efficient for long streaming sessions h. Compact design—optimal footprint to reduce parallax i. Great Low light capabilities 10. 360 Designs Mini Eye 4 a. 4 185° lenses b. 3K @ 60fps or 6K @30fps c. Manual stitching d. $11,699-$16,999 11. Jaunt a. b. c.

One 24 130° lenses w/ global shutter 8K-by-4K @ 30fps in 3D stereoscopic Available for Rent from Radiant Images at approximately $1000 a day

12. Facebook Surround 360 a. New versions (announced at F8 this year) are the “x6” and “x24” with 6 and 24 lenses respectively b. Full resolution; 6 degrees of freedom c. Original version was an open source build - no final product or par ts sold by FB d. Facebook plans on licensing the new camera designs to a “select group of companies”

Jaunt One Jaunt

Facebook Surround x24 (left), Facebook Surround x6 (right) Facebook

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

13. Nokia Ozo a. 195° lenses with 100° of crossover between each, using a global shutter b. 2K-by-2K @ 30fps in 3D stereoscopic c. 8 microphones capturing spatial audio d. $45,000

Proprietary Solutions

T h e se a r e ca me r a s o lu t io n s t h a t ar e typically made by or fo r p r o du ct i o n e n t it ie s , a n d a re h eld as par t of the “secr et s a u c e ” fo r t ha t e n t it ie s c o n t e n t c reati on. A few 3D scene c a p t u r e sol u t io n s a re wo r t h me n t ioning her e . 8 i i s a N e w Ze a la n d c o mpa ny t h at uti l i zes multi ple DSLR c a m er a s a n d a lig h t s t a g e t o c a pt ur e tr uly vol umetr ic 3D vi de o. V i e we r s c a n wa lk a ro u n d and obser ve the model o f t h e p e r son , wh ic h is e n h a n c e d by editi ng to cr eate an i nc r e di bly r e a lis t ic a n ima t io n . 8i recognizes the potential th e i r t e c h n o lo g y mu s t c re a t e imm er sive VR exper i ences th a t a d v a n ce e mo t io n a l c o n n e c t io n and empathy. Two ot h e r co mpa n ie s wo r k in g o n the similar technology ar e Ti m eS l i c e Films a n d Ske t c h f a b.  C a m e r a co m pa ny Ly t ro h a s be e n wor king on a pr opr i etar y l i g h t fi e l d c a m e r a r ig n a me d t h e Lytr o I mmer ge . Numer ous s e n sor s c a p t u re t h e e n t ire lig h t fi el d in volumetr i c space (a r ou n d o n e me t e r, a c c o rdin g t o the company) , allow ing

contributions@vrarproject.com

VR/AR Project

Immerge Lytro

New Amsterdam Media

th e v i ewer t o e x a min e in t h e ir e nvir onment in that ar ea w h er e t h e c a me r a is s h o o t in g . A h igh qual i ty 360° image is th e n a sse m ble d, wh ic h c a n be v ie wed in VR .  Un l i ke mu l t i -c a me r a s e t u ps wh ic h sti tch together mul tipl e i m a g es i n a c irc u la r fo r ma t io n , t h e reby assumi ng the viewer ’s p e r spe c t i ve t o be c o mple t e ly h or i zontal, Lytr o’s setup p r ov i d e s st ere o s c o pic de pt h a n d posi tional eye tr acking so th a t t h e v i e we r is f re e t o le a n a nd r ol l their head i nside th e sp a ce . I n a ddit io n t o t h e r ig i tself, I mmer ge comes w i t h a se r ve r box t o s t o re t h e massi ve amount of data c a pt u r e d by t h e c a me r a . C o mpa t ibi l i ty w ith Nuke , Adobe P r e m i er e , a n d Fin a l C u t Pro t h ro ugh pl ug- i ns along w ith i nt e g r a t ed play ba c k a bilit y wit h the major HMD’s w i l l m a ke L y t r o I mme r g e t h e fir s t e n d-to-end sol uti on for live V R fi l m m a k i ng .

DIY

T h i s c a t e g or y is a n e x a mple o f c amer a sol uti ons that can b e a sse m bl ed ba s e d o n pla n s . These r ange fr om the 3D p r i n t ed pl a stic h a r n e s s e s f ro m 360R i ze to the br ushed al u m i nu m o f Go Pro Omn i a n d be yond.

360Rize

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

SPOTLIGHT ON CHINA

201 6 w a s p e r c e i ve d as “Ye a r One” of V ir tual R eality in China. Hu n d re ds o f V R c o mp a ni e s c r op p ed up to catch this w ave . Ac co r di n g t o C a n a ly s , a ro u n d 300, 000 VR headsets shi ppi ng to C h i n a i n 2016, ma k in g it t h e wor ld’s second lar gest m a r ke t , w i t h a s h a re o f a ro u n d 15% . Potenti al user s ( w ho have h e a r d abo u t V R a n d in t e re s ted in i t) ar e estimated to b e 2 8 6 m illio n , a c c o rdin g t o a Chi na VR user behavior s t u dy by B a o fe n g M o jin g , Na t io n a l I nsti tute of Adver tising, an d S o c i a l B et a . Ac co r di n g t o 2016 H 1C h in a V ir tual R eal i ty I ndustr y R e sea r ch R e po r t c o n du c t e d by iMedi a R esear ch, the scal e o f C h i n ese VR ma r ke t is e s t ima t e d to over 55 billion R MB (8 b i l l i o n U S D ). C h i n e se G ove r n me n t is ve r y s u ppo r ti ve of the devel opment o f t h e V R i n du s t r y. Th e C h in a E le ctr onics Standar di zati on Inst i t u t e r el ea s e d a wh it e pa pe r o f VR i ndustr y devel opment i n A pr i l 2 0 1 4, wh ic h a ime d t o promote the standar ds for V R/ A R i n C hin a . It c ove re d 5 key aspects: the cur r ent d e vel opm e n t (bo t h do me s t ic a n d i nter nati onal) , featur es an d ch a r a ct er is t ic s o f V R t e c h n o lo gy, the applicati on of VR i n ke y a r ea s (in c lu din g milit a r y, enter tainment, medi cine , i ndu st r y, a n d e du c a t io n ), a n d t h e possible impr ovements i n th e f u t u r e a nd s u g g e s t io n s o n g over nment pol i cies.

GOVERNMENT SUPPORT

In M ay 2 0 1 6 , Xin h u a Ne ws A g e n cy, China’s offici al pr ess ag e n cy, l a u n c h e d a V R/ A R c h a n n e l on i ts website . C h i n a ’s 1 3 t h Five Ye a r Pla n defi niti vely states the gove r n m en t ’s pla n s t o in du s t r i alize immer sive and i nt er a ct i ve m e dia : “Ne w in fo r ma tion technol ogy w ill be s u p po r t e d by t h e g ove r n me n t as a str ategic emer ging i ndu st r y, w h i le t h e in du s t r ia liza t io n of Vi r tual R eality and Int e r a ct i ve Film t e c h n o lo g ie s a s wel l as other emer ging i ndu st r i es w i ll be re c o g n ize d a s n ew sour ces of economic gr ow t h .” I t co n t inu e s , “We will s uppor t the development o f n e x t g en e r a t io n in fo r ma t io n technol ogy, new - ener gy ve h i c l es, bi ot e c h n o lo g y, g re e n a n d l ow -car bon technol ogy, hi g h - en d e qu ipme n t a n d ma t e r ials, and digi tal cr eative i ndu st r i es. I n fo s t e r in g n e w a re a s of economi c gr ow th, we contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

w i l l sp u r i n n ov a t io n a n d in du s t r ia l appl i cation i n emer ging, c u t t i n g - edg e fie lds s u c h a s a dv anced semi - conductivi ty, ro bo t i c s, a d dit ive ma nu f a c t u r in g , i ntel l i gent systems, next gen er a t i on av ia t io n e qu ipme n t , compr ehensi ve ser vice s y st em s for s pa c e t e c h n o lo g ie s , smar t tr anspor tati on, p r ec i si on m e dic in e , s y s t e ms fo r high-effi ciency ener gy s t or a g e a n d dis t r ibu t e d e n e r g y, s m ar t mater i als, effici ent e n er g y c o n se r v a t io n , e nv iro n me ntal pr otecti on, vi r tual re a l i t y, a n d i nt e r a c t ive mov ie s a n d tel evision. “

LEADING COMPANIES

T h e l ea d i n g f a c t o r in C h in a ’s bo o ming VR i ndustr y has been th a t t h e I n t er n e t Tit a n s — “B AT” (Baidu, Alibaba, Tencent) . All t h r ee a r e e n t e r in g t h e ma r ke t , announci ng their plans, an d i nve st i n g in s t a r t u ps .

Tencent

In 2 0 1 5 , Te n c e n t be c a me t h e fir s t in B AT to decl ar e its i n t e r e st i n V R in du s t r y, t h e ir pla n ini tial ly center ed on just ga m es. Ten ce n t is a ls o in t ro du c in g thei r ow n HMD. T hey ar e i nt er e st e d i n bo t h mo bile a n d PC based devices, planni ng to su p po r t apps , g a me s , mov ie s , and live- str eamed event an d d e vel opin g t h e ir own V R g a m es. Tencent VR al so is p la n n i n g t o impo r t t h e SD K f ro m Q Q and WeChat, w hich ar e t h e l a r g e s t s o c ia l pla t fo r ms in Chi na, suggesting a big p lay i n soc i a l V R. Alipay AR Red Envelope Alibaba

Alibaba

Al i ba b a i s i nve s t in g in V R/ A R s t ar tups, i ncl udi ng Magi c L ea p, I n fi n i t y A R, Lu mu s a n d WayR ay. As China’s biggest e - c o m m e r c e c o mpa ny, A liba ba i s focusi ng heavily on d e vel opi n g VR-re la t e d s h o ppin g exper i ences. I n Mar ch 2 0 1 6 , A l i ba b a e s t a blis h e d it s V R l ab— GnomeMagic Lab. T h e fi r st pr o je c t o f Gn o me Ma g ic l ab i s to build thei r ow n 3 D pr o du c t inve n t o r ie s a n d cr eate the bi ggest 3D m e r c h a n d i se da t a ba s e . Th e g o a l is to help pr omote VR s h oppi n g ex pe r ie n c e s , a n d s e t u p VR stor es. In J u ly 2 0 1 6, A liba ba in t ro du c ed Taobao BU Y + plan at Ta oba o Ma ke r Fe s t iv a l. C u s t o mer s can br ow se items i n a v i r t u a l ma ll, in c lu din g Ta r g et, Macy’s and Costco. Ali p ay r e c e n tly la u n c h e d A R Re d E nvel ope , a fr esh take on a cu l t u r a l t r adit io n : A lipay u s e r s hide vi r tual hongbao, or re d e nve l ope s , a t a ny lo c a t io n . Th e n, they upl oad a photo of th e sp o t t o t he a pp a lo n g wit h a c lue . T he app automati cal ly s t or e s t h e l oc a t io n , fo r pe o ple t o hunt. U ser s can tr ack a hon g ba o by me s s a g in g t h e pe r s o n w ho hi d the tr easur e and

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

ask i n g fo r c l ue s . W h e n t h e e nve lo p e i s l ocated, the player s c a n s t h e sp o t , a n d t h e h o n g ba o wi l l appear onscr een. O ne tap de po si t s t h e mo n e y s t o re d in the hongbao i nto the trea su r e- h u n t e r’s A lipay a c c o u n t . Ali b a b a a r e a ls o de ve lo pin g V R b ased payment ser vices un de r A n t Fin a n c ia l, a n d will pro d uce VR content under Ali b a b a P i ct ure s .

Baidu

In D e c e m be r 2015, B a idu la u n c h e d a VR video str eami ng c h a n n e l u n de r it s o n lin e s t re a mi ng segment. I n May 2 0 1 6 , B a i d u la u n c h e d B a idu V R+ , a community to shar e an d e x c h a n g e in fo r ma t io n a bo u t VR . I n September 2016, th e C h i n a ’s la r g e s t s e a rc h e n g in e l aunched Chi na’s fir st V R web b r ow s e r. It o f fe r s pa n o r a mic view ing, online video s t r ea m i n g a n d n av ig a t io n . In 2 0 1 7 , B aidu Ve n t u re s inve s t ed in Hol ogr aphic- tech s t a r t u p 8 i , w h ic h is it s fir s t inve s t m ent i n VR /AR ar ea. i Q i yi , a B a i d u s u bs idia r y v ide o s t re aming company, l aunched i ts V R p l a t for m in M ay 2016. It in t ends to bui l d the wor l d’s l ar g est C h i n es e -la n g u a g e V R s e r vice . I t i s wor king w i th m o r e t h a n 3 0 0 pa r t n e r s t o pro du c e content and har dw ar e . It p l a n s t o c r e a t e a n e w a pp s u it e , cal l ed i VR +, consi sting of i V R Pa n or a m a C in e ma a n d iV R G ame R oom, w hich makes i ts m ov i es a n d g a me s c o mpa t ible w i th any HMD. I t i s also d e vel opi n g i ts own a ll-in -o n e V R d evice , w hi ch w i l l debut i n 2 0 1 7 .

JingDong

In S ept em be r 2016, JD. c o m a n n o u nced i ts VR /AR str ategy, e n co m pa ssi n g V R s h o ppin g e x pe r i ence and 3D models. JD w i l l foc u s o n A R e x pe r ie n c es i n home decor ati on an d r e n ov a t io n , a s we ll a s lo g istics management. T hey have r ec e n t ly a n n o u n c e d a V R/A R e- commer ce industr i al al l i a n c e . J i n g D o n g a ls o la u n c h e d V R.jd.com and established a V R soc i a l n e t wo r k in g c o mmu n it y.

Baofeng Technology

B a ofen g i s o n e o f t h e e a r lie s t V R headset businesses i n C h i n a . I n 2 0 14, B a o fe n g h a d a lready r eleased its fi r stgen er a t i on pro du c t . By D e c e mber 2016, i t r el eased its fi f t h - g e n er a t io n pro du c t . Or ig in a lly, Baofeng focused on VR H M D s a n d Ca rdbo a rd, bu t C E O Xin Feng has indicated th a t a l l - i n - on e , s t a n da lo n e V R h eadsets w ill be i n the s p o t l i g h t fo r 2017. In t e r ms o f c o ntent, the huge database contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

o f i t s S t or m Playe r o f fe r s v a r io u s resour ces, al so i mpor ti ng c o n t en t f r om Yo u k u , Tu do u , a n d iQiyi . B a o fen g for me d a pa r t n e r s h ip wit h Sohu New s to pr oduce V R n e w s. E a r lie r t h is ye a r, Ba o fe n g established Baofeng New Cu l t u r e t o e x pa n d it s bu s in e s s in VR tour i sm.

LeTV

L e TV h a s r ele a s e d t h e ir Le V R CO O L1 VR Helmet 3D H M D. I t a l so re le a s e d it s own V R mobile app to pr ovide c o n t en t t h a t is c o mpa t ible wit h its VR HMD. Content is th e c e n t e r o f t h e Le TV ’s V R pla n . I ts divi si ons—Le Vision P ict u r e s, L e S po r t s a n d Le Mu s ic — w i l l cover the VR content fo r fi l m , c o n ce r t s , s po r t s , n e ws , t r avel, education, games and o t h e r a spe c t s. Th e c o mpa ny c o n t inues to seek cooper ati on w i t h o t h e r co mpa n ie s t o e x plo re VR content. I n ter ms of V R edu ca t i on t h e y in ke d a pa r t n e r shi p w ith New O r i ental Edu ca t i on & Te c h n o lo g y Gro u p; fo r VR New s they par tner ed w i t h C a i xi n M e dia ; fo r V R H o me D ecor ati on they par tner ed wi t h Tu ba t u .

NetEase

N e t E a se pr o du c e d it s fir s t V R g a me Tw ilight Pi oneer s for th e G o o g l e D ay dre a m Pla t fo r m. It exper i mented w ith AR tec h n o l og y in it s n e w h it mo bile game O nmyoj i , w hich l au n c h ed i n 2016, qu ic k ly be c o ming the No.1 top gr ossing ga m e o n t h e C h in e s e A pp St o re .

ELECTRONICS COMPANIES HTC

H TC ’s m a i n bu s in e s s is s ma r t ph o n es and tablets. I n 2016, i t e x p a n de d i n to t h e V R h a rdwa re mar ket. HT C Vi ve , a hi ghe n d V R h ea d s e t de ve lo pe d by HTC and Valve Cor por ati on, w a s r e l ea se d in A pr il 2016 a n d g a ined an 18% mar ket shar e o f t ot a l V R he a ds e t s in C h in a in 2 016. Vi ve Tr acker, w hi ch ad ds m o t i o n t o t h ird-pa r t y a c c e s sor i es, debuted at CES 2 0 1 7 . O n M arc h 15t h 2017, H TC announced i ts pl ans to s e l l a $ 9 1 m i llio n ph o n e ma nu f a c t ur i ng factor y in Shanghai to su ppo r t i ts V R bu s in e s s .

Xiaomi

X i a om i r el ea se d it s M i V R Play h e a dset and Mi VR app. T heir o b j e c t i ve i s t o pro du c e h ig h -e n d, affor dable headsets for c o n su m er s. Th e c o mpa ny h a s s e t up an “Explor er ’s Lab” fo cu si n g o n re s e a rc h in a r t ific ia l i ntel l i gence and vir tual re a l i t y. M obi l e V R is c u r re n t ly Xia omi’s focus, but it is also e x p l or i n g t h e po s s ibilit y o f a s t andalone , all-in- one VR hea d set . contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

Mi VR Play Headset Xiaomi

Huawei

In A pr i l 2 016, H u awe i in t ro du ced its VR headset— H u awei V R— wh ic h wo r k s wit h it s P9, P9 Plus, and Mate 8 sm a r t p h on e s . In Nove mbe r 2016, Huawei i ntr oduced two n ew sm ar t ph o n e s , M a t e 9 a n d Mate 9 Pr o, w hi ch ar e th e fi r st G oo g le D ay dre a m re a dy smar tphones in China. As fo r co n t e n t , H u awe i is c o o pe r ati ng w ith Baofeng w hich i s c o n t r i bu t in g e x is t in g film, TV, and gaming r esour ces. H u awei i s a l so pa r t n e r in g wit h Youku, w hich w ill connect i ts V R c h a n n e l wit h t h e H u awe i V R headset. Huawei i s also ad dr e ssi n g t h e bo t t le n e c k in bro adband: the cur r ent 4G net wor k i s n o t a de qu a t e t o s u ppo r t VR /AR at scal e . Huawei p r opo sed t h e c o n c e pt o f 4. 5G networ k, w hile wor ki ng tow a r d s a 5G n e t wo r k . H u awe i al so announced that it w i l l co o pe r a t e wit h In s t a 360 t o pr oduce a clip- on 360° s m a r t p h on e c a me r a .

STARTUPS DeePoon VR

D ee Po o n V R is a le a din g C h in e s e VR company that cr eates b o t h m o bi l e a n d PC -ba s e d V R har dw ar e . I t took 17% m a r ket sh a r e o f h e a ds e t s s h ipme nts i n Chi na in 2016. At C E S 2 0 1 7 , D e e Po o n V R in t ro du c ed i ts next- gen systems: a PC h el m et c a lle d E 3, a n d a la ser positi oni ng sol uti on c a l l e d E - Po l ar is . Th e n , in Ma rc h 2017, DeePoon VR and D el l a n n ou nc e d a s t r a t e g ic c o o per ati on. DeePoon w ill c o n n ec t i t s n e w E 3 pro du c t wit h Del l ’s devices to br i ng V R e x pe r i en c e s t o c u s t o me r s wi th lower expenses and th e r e fo r e i m p rove t h e wh o le V R e cosystem.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

ANTVR

Est a bl i sh e d i n 2014, A NTV R is fo cusi ng on devel opi ng VR he a d set s for PC a n d mo bile , a s well as VR camer as. T he AN TV R Al l - in -On e k it wa s f u n ded vi a Kickstar ter. T he c o m p a ny h a s a ls o de s ig n e d V R HMD kits for Lenovo and H i se n se , a n d re le a s e d t h e ir A NTV R Camer a i n 2016.

VR/AR THEME PARKS Th e Vo id

In F ebr u a r y 2 016, Sh a n da G ro u p a nnounced that it i s one of th e pr i n c i p a l inve s t o r s in Th e Vo id, the most hi gh- pr ofile VR th e m e p a r k pro je c t . Sh a n da G ro u p is playi ng an i mpor tant ro l e i n Th e Vo id’s s t r a t e g ie s , pro d ucts, and management o f C h i n ese ma r ke t a n d h a s pla n s to br i ng T he Void to C h i n a . Th ey are s t ill lo o k in g fo r a Chinese company that is c o m p e t en t a nd e x pe r ie n c e d a s t h e ir par tner i n thi s pr oject.

Pa n go lin VR P ark

H E I V R i s r e sea rc h in g a n d de ve lo pin g their Pangolin VR theme p a r k p r o g r a m. H E I V R h a s pa r t n er ed w ith Car de Happy Wor l d i n J i a n g s u prov in c e a n d in t e nds to build a 2700m2 VR th e m e p a r k . Th e t h e me pa r k will in clude space explor ati on, e -sp o r t s, sci en c e wo r ld, a n d Pa n g olin enter tai nment.

Pro je ct A lic e

P r oj ec t Al i c e is B e ijin g -ba s e d Noitom’s commer cial B 2 B sol u t i o n in t e n de d fo r V R e nter tainment zones and th e m e p a r k s. Pro je c t A lic e c o mes as an end- to- end s o l u t i on for t h e me pa r k a n d a rc a de oper ator s including an o p t i c a l syst e m t o t r a c k pe o ple a n d objects, i ner tial motion c a p t u r e su i t s, mo t io n g love s , a n d head- mounted di splays (H M D s) . P r o je c t A lic e c a n be e x per i enced in a gr adi ent of c o n fi g u r a t i o n s , f ro m t h e f u ll V R bo dysuit to a simpl e HMD w i t h co n t r o l le r s .

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

INVESTMENTS AR/VR Company

ZeroKEY Inc.

2mee

Sketchfab

Occipital

Avegant

6 7

Jaunt NullSpace VR

Focus Investment ($M) Funding round Date Investors (Firms) AR; Consumer electronics; VR $1.20 Seed Mar-15 Content, Android, iOS, AR 0.15 Seed Apr-15 Angels Den Balderton Capital, Borealis, Real Time, 3D, FirstMark Capital, Partech Software, FinTech 7 Series A Jun-15 Ventures, Techstars Intel Capital, Draper Frontier, Foundry Group, Grishin Real Time, iOS, Robotics, k9 Ventures, Shea Software, Mobile 13 Series B Jul-15 Ventures 500 Startups, Capital Community Angel Investors, CKM Capital Partners, CrunchFund, DN Capital, Hangzhou Liaison Interactive Information Technology, Intel Capital, Kaiwu Capital, Kickstarter, Michigan AR; Video; Mobile 24 Series B Aug-15 Angel Fund, NHN Investment China Media Capital, Evolution Media Partners, GV, Highland Capital Partners, InnoSpring Seed Fund, Redpoint, Sky, The Madison Square Garden Hardware, Media Company; Axel Spring; Disney; and EntertainParticipant Media; ProSiebenment, Software 65 Series C Sep-15 Sat.1 Media VR

0.55

10 Envelop VR

VR; 3D Technology; Software 13.5 VR; AI; Travel; Mobile 0.1 VR; Enterprise Software; B2B 5.5

11 GeoCV

VR; SaaS; 3D Tech; Real Estate --

Sturfree

contributions@vrarproject.com

Series A

Oct-15 Undisclosed Advancit Capital, BDMI, Boost VC, Dolby Family Ventures, End Cue, Founders Fund, Freelands Ventures, Frontier Tech Ventures, Horizons Ventures, Inevitable Ventures, Kortschak Investments, L.P., Right Click Oct-15 Capital

Convertible

Jan-16

Series A Seed

VR/AR Project

Investors (People)

Jeremy Stoppelman, John S. Hendricks, Kevin Wall, Scott Nolan, Walter Kortschak

Madrona Venture Group; GV; Jan-16 KPCB Edge Moscow Seed Fund; Starta Accelerator: Starta Capital; Jan-16 Universe Ventures

New Amsterdam Media

AR/VR Company

12 MagePrints 13 Spincle 14 SpaceView 15 Immersal

Focus Investment ($M) Funding round Date Investors (Firms) AR; Photography; Non Equity Social Media -Assistance Jan-16 TechCode Accelerator US VR; Photography; Mobile -Seed Jan-16 Right Side Capital Management; Portland Seed Fund AR; B2B; Mobile 0.1 Debt Feb-16 (Seed & Debt)

18 Digital Generals

AR; Software 0.11 VR; E-Commerce; Mobile 0.25 AR; Collaboration; Virtualization 0.5 VR; Info Tech; Content 0.5

19 Fusion Tech Inc.

EdTech; Health Care; 3D Tech; VR 0.524

Debt

Feb-16

20 KAT

VR; Internet

Angel

Unity Ventures; Xinyuan InvestFeb-16 ment

VR; AR; Autonomous Vehicles 1

Series A

Ohio TechAngels Fund; TechGROWTH Ohio; Tri-State Feb-16 Invesetment Group

VR; Internet

Venture

Feb-16

Series A

Radical Ventures; Aligo Innovation; gr9k Technologies; Radical Feb-16 Investments LP; TandemLaunch

16 SpaceoutVR 17 18Angles

21 Yost Labs 22 Janus VR

23 wrnch 24 Sphero

25 WEVR 26 MindMaze

27 Magic Leap 28 Get Jenny

1.58

AR; Computer Vision; Image Recognition; Software 1.8 Robotics, AR, Gaming 3.12

SaaS; entertainment; social; VR 25 VR; Medical; Motion Capture 100

Feb-16 Presence Capital

Seed

Feb-16 Redwood Technology Ventures

Seed

Feb-16 Alliance of Angels

Seed

Feb-16

Debt

Venture

Wearables; AR; HMDs; Consumer Electronics; Software 793.5

Series C

AI; VR

Seed

Feb-16 Nestholma

30 Enyx Studios 31 Oriense

AR; Wearables; Embedded Systems

contributions@vrarproject.com

0.05

Venture

Heatherm Huang

Feb-16 Mercato Partners (Series E) AME Cloud Ventures; Boldstart Ventures; Digital Garage; Dragoneer Investment Group; Evolution Media Partners; HTC Ross LevinCorp; Orange Digital Ventures; sohn; Scott Feb-16 Samsung Ventures McNealy Feb-16 Hinduja Group Alibaba; Fidelity Investments; Google; JP Morgan; Morgan Stanley; Qualcomm Ventures; T. Rowe Price; Warner Bros.; Feb-16 Wellington Management

VR; Media Entertainment; News -VR; Service; Creative Agency 0.025

29 HoneyVR

Seed

Investors (People)

Equity Crowdfunding Feb-16 Grant

Innovation Fund Northeast Mar-16 Ohio

Grant

(Seed): RSV Venture Partners; Mar-16 TMT Investments

VR/AR Project

New Amsterdam Media

AR/VR Company

32 Cre8VR 33 Look 34 Freeform Labs 35 OSSIC 36 Infinite360

Focus Investment ($M) Funding round Date VR; Digital Media; Social Media 0.1 Seed Mar-16 VR; AI; Apps VR; Content; Gaming VR; Gaming; Audio VR; Publishing; News

0.272

Investors (Firms)

Mar-16 Viralety Ventures

0.75

Seed Non Equity Assistance

Seed

Mar-16

1.12

Venture

Mar-16 Gate Ventures Plc

Series A

Mar-16 Salesforce Ventures

Venture

Mar-16

Series A

Draper Fisher Jurvetson (DFJ); BDMI; End Cue; Greylock Partners; The Venture Reality Fund; Mar-16 Vayner/RSE

E-Commerce; 3D Technology; AR; visual search 3

37 Augment 38 Archiat Interactive VR; Video Games 3.16

Investors (People) Jason Trindade

Mar-16 Starbreeze

39 Visionary VR

VR; Internet; Software

40 Penrose Studios

Film Production; AR 8.5

Seed

41 FOVE

HMDs

Series A

Sway Ventures; 8 Angel; Suffolk Equity Partners; Translink Mar-16 Capital Colopl VR Fund; 2020 Ventures; Foxconn Technology Group; Mar-16 Samsung Ventures Taizo Son

42 VOKE

Digital Entertainment; VR; 3D 12.5

Series A

Intel Capital; A+E Networks; Mar-16 Nautilus Venture Partners

43 Blippar

AR; computer vision; AI; visual search; mobile

Series D

Mar-16 Khazanah Nasional

Wearables; Machine Learning; AR 2.55

Series A

Lux Capital; ARCH Venture Mar-16 Partners; Felicis Ventures

Angel

Apr-16 QVentures

Seed

Apr-16

Seed

Apr-16

44 Aira Tech Corp 45 Metadrift 46 Jahshaka LLC 47 Photonomie

48 Aero Glass 49 Cluster

0.07

VR; Publishing 0.1 VR; Photography; Mobile 0.227 AR; VR; Aerospace

Edith Yeung; James Riney; Kelsey Morgan; Tim McLaughlin; Victor Young

0.25

Seed

Apr-16 OXO Labs Ltd

Seed

Apr-16 Skyland Ventures; East Ventures

50 inVRsion

VR; Internet; Apps 0.46 VR; B2B; Real Time 0.68

Seed

Apr-16

51 Edorble

VR; 3D Technology; Education 0.75

Seed

Apr-16

contributions@vrarproject.com

VR/AR Project

Larry Brock; Scott Belsky

Akos Maroy; Mortoff;Szabolcs Szakacsits

New Amsterdam Media

AR/VR Company

52 SpaceVR 53 Lucid VR 54 VRstudios

55 DataMesh 56 Marxent 57 Merge Labs 58 Gazillion, Inc. 59 Apollo Box Midas Touch 60 Games 61 River Studios

Focus VR; AI; Space Travel

AR; Consulting; Big Data; Industrial Automation 3.09 AR; VR; Retail; Software 10 VR; HMDs; mobile 10.11 VR; Games; Mobile 11

Venture

Apr-16 Stony Lonesome Group

Series A

Apr-16 IDG Capital Partners Detroit Venture Partners; Beck Apr-16 Besecker; Stage 1 Ventures

Series B

Series A

E-Commerce; AR --

Angel

Apr-16

VR; Console Games

Venture

Apr-16 TEEC Angel Fund

Debt

Frontier Tech Ventures; TranApr-16 scendVR

Series A

Apr-16 Qualcomm Incorporated

0.125

Seed

May-16

0.525

Debt

May-16

Venture

May-16

Grant

Korea Tech Incubator Program for Startup; National IT Industry Promotion Agency - NIPA; May-16 SparkLabs (Angel)

Series A

Rana Capital Partners; Salem May-16 Partners

Series A

645 Venture; A+E Networks; Social Starts; The Tornante May-16 Company

VR; Marketing; Digital Entertainment --

66 DoubleMe

AR; VR; Virtual World; 3D

64 PRSONAS, Inc.

Venture

Investors (Firms) Shanda Group; Skywood CapiApr-16 tal; Skywood Ventures Lab360 Hardware Incubator; S2 Capital; TEEC Angel Fund; Apr-16 Wistron Corporation

Seed

67 Lucid Sight

VR; Gaming; Advertising 3.5

68 Littlstar

VR; Video Streaming; Advertising 5.24

Venture

Date

Apr-16 Tech Coast Angels Oak Investment Partners; Apr-16 ROTH Capital Partners

65 Aemass

63 Pair Inc.

1.25

Photography; AR; VR; Wearables 2.1 VR; Computer; Gaming 3

VR; Cloud Computing; Visual Search; Mobile AR; Mobile; E-Commerce AR; Computer Vision; Software VR; Digital Media; 3D

62 Ximmerse

Investment ($M) Funding round

69 Eonite Perception 3D Tech; AR; VR 5.25

Seed

Signia Venture Partners; Rising Tide; Presence Capital; Lightspeed Venture Partners;CLI May-16 Ventures

70 roOomy Interior Design contributions@vrarproject.com

Series B

May-16 Antimeridian; Bertoia

3D; AR; Retail;

VR/AR Project

Investors (People)

Greg Richardson; Greg Castle; Andrew Gault; Andreas Bechtolsheim; Amr Awadallah; Nader Shenouda

New Amsterdam Media

AR/VR Company

71 Zeality Inc

72 Sessio Software 73 VisualPathy 74 Primitive 75 Against Gravity

Focus VR; 360-degree video; social

Investment ($M) Funding round

May-16 Frontier Tech Ventures

Grant

Tekes; OP Financial Group Jun-16 (Debt)

Angel

Jun-16

VR; Graphic Design; Developer Platform 0.7

Seed

Jun-16 Boost VC; Draper Associates

AR; VR; Software 0.815

Debt

Jun-16

AR; Collaboration; Enterprise 0.055 VR; Content Delivery Network 0.5

1.3

Seed

Colopl VR Fund; CRCM Venture Capital; Mission and Market; Social Starts; Sparkland Jun-16 Capital Adways Inc; Colopl VR Fund; Incubate Fund; KLab Ventures; Jun-16 Slogan; Viling Venture Partners Anorak Ventures; Danhua Capital; Presence Capital; Y Jun-16 Combinator

1.75

Seed

Jun-16

1.85

Seed

Jun-16 Frontier Tech Ventures

4.54

Venture

Jun-16 Innovacom; MAIF; Seed4Soft

Series A

Comcast Ventures; LDV PartJun-16 ners; PHI Group; Quebec

VR; animation

77 Dverse

VR; Interior Design; Training 1.04

79 Tixsee 80 TimeLooper

81 Cozy Cloud Felix & Paul Stu82 dios

Investors (Firms)

Seed

76 Limitless

78 Loom.ai

Date

Computer vision VR; Ticketing; Mobile Apps VR; entertainment; tourism Web Hosting; Cloud Computing; AR; Open Source

83 Starbreeze

VR; Digital Media; Content 6.8 VR; Digital Entertainment 9

84 Within

VR; Software; Apps

85 uSens

AR; VR; Machine Learning; Computer Vision; AI 20

contributions@vrarproject.com

12.6

Seed Seed

Venture

Series A

Jun-16 Acer; SmileGate Andreessen Horowitz; Marker; 21st Century Fox; Annapurna Picture; Freelands Ventures; Legendary Entertainment; Live Nation Entertainment; Vice Media; WME Entertainment; Jun-16 Tribeca Enterprises

Series A

Jun-16 Fosun Kinzon Capital

VR/AR Project

Investors (People) Jason Khalipa; Paraag Marathe Maria Mera; Mikko Mikkola; Tito Toivola; Tomas Westerholm; Vitalii Vasylenko

Jay Rifkin; Masi Oka; Nick Rau Shogo Kawada Joe Kraus; Zach Coelius

New Amsterdam Media

AR/VR Company

86 Meta 87 CryWorks 88 Virtuka 89 PerfectaMe 90 Feel VR 91 Umbo, Inc.

92 Dermandar 93 AmbiensVR 94 Paracosm 95 ScopeAR 96 Radd3

97 Kite & Lightning 98 Woofbert 99 Vuzix

100 Unity 101 AVR 102 Banian Labs 103 SpectraVR 104 Second Studio

Focus

AR; hardware; HMDs

Investment ($M) Funding round

VR; Media Entertainment; Film -VR; Photo Sharing; Video -AR; Retail; Fashion; E-Commerce 0.02 VR; Gaming; Simulation 0.04 AR; Lifestyle; Electronics 0.175

VR; Photo Sharing; Real Estate 0.35

Series B

Date

Investors (Firms) Banyan Capital; Bold Capital Partners; Comcast Ventures; Danhua Capital; GQY; Horizons Ventures; Lenovo; Jun-16 Presence Capital; Tencent

Venture

451 Media Group; 500 Mobile Jun-16 Collective; WI Harper Group Brithright Israel Excel Ventures Jun-16 Program

Debt

Jul-16

Seed

Jul-16

Seed

Jul-16

Seed

Investors (People)

IoT Hub; Roman Kravchenko

Angel

Jul-16

BeryTech Fund

VR 0.89 AR; 3D; Robotics; Real Estate 1.25 AR; software; training 2

Seed

Jul-16

Venture

Jul-16

Seed

Jul-16

LVenture Group Osage University Partners; Atlas Venture (Seed Lead) New Stack Ventures; Presence Capital; Susa Ventures

AR; Software

2.2

Series A

Jul-16

VR; Cinema

2.5

Seed

Jul-16

VR; Education

2.54

Debt

Jul-16

HTC Corp; Presence Capital Raine Ventures; Boost VC; Comcast Ventures; Courtside Ventures; Outpost Capital; Social Capital

(Angel): Daniel Neuwirth; Georges Harik; Hala Labaki; Jan Erik Solem; Pierre Elzouki

Jason Rosenthal

AR; Consumer Electronics; Eyewear 6.6

Post IPO Equity Jul-16

3D technology; software; gaming; AR; VR 181

Series C

Jul-16

Intel Capital (Series A) DFJ Growth; China Investment Corporation; FREES FUND; iGlobe Partners; Sequoia Capital; Thrive Capital; WestSummit Capital Max Levchin

Angel

Jul-16

Hacker Unit

Seed

Jul-16

Seed

Jul-16

0.04

Seed

Aug-16 Rotolab

VR; AR; Gaming AR; Internet; Travel; Apps 3D Technology; VR VR; Education; Enterprise

contributions@vrarproject.com

VR/AR Project

Frontier Tech Ventures

New Amsterdam Media

AR/VR Company

105 Qbit Technologies 106 ConstructVR

Focus

Investment ($M) Funding round

VR; Serious Games; Virtual Assistant 0.085 VR; Mobile; Software 0.12

Date

Investors (Firms)

Seed

Aug-16

Seed

Aug-16 Y Combinator

Investors (People)

AR; Semiconductor; Biotechnology; Optical Communication 0.2 Digital Entertainment; VR 1

Seed Aug-16 Hanhai Investment Inc. Equity Crowdfunding Aug-16

VR; Video Game 1.25

Venture

Aug-16 IrisAngels; Presence Capital

VR; Architecture; Virtualization 1.5

Seed

GREE Ventures; CyberAgent Ventures; Frontier Tech VenAug-16 tures; Y Combinator

AR; VR; 3D Tech 1.7

Venture

Aug-16 KTH Holding

1.8

Seed

113 TheWaveVR

Digital Entertainment; Concerts; VR 2.5

Seed

114 Navisens

AR; Indoor Positioning; Mobile 2.6

Seed

VR; gaming

Series A

Aug-16 Colopl VR Fund; Lux Capital KPCB Edge; Frontier Tech Ventures; Immersion Ventures; Luma Launch; Outpost Capital; Presence Capital; RRE Ventures; Joe Kraus; Aug-16 Seedcamp Mike Fischer Resolute.vc; Amicus Capital; Arba Seed Investment Group; Aug-16 KEC Ventures Gokul Rajarm Qualcomm Ventures; Capital Factory; Colopl VR Fund; HTC Aug-16 Corp; The VR Fund

Digital Entertainment; VR 5

Series A

107 Aurora AR 108 Legion M 109 Superstar Games

110 InsiteVR

111 ManoMotion 112 Pinscreen

115 Owlchemy Labs Steel Wool Stu116 dios 117 Sliver.tv

360-degree video; gaming; VR 6.2

118 Virtuix

VR; Consumer Electronic

contributions@vrarproject.com

7.7

Aug-16 HTC COLOPL; DCM Ventures; GREE; Sierra Ventures; SparkSeed Aug-16 land Capital; The VR Fund (Seed): 2020 Ventures; Houston Angel Network; Maveron; MSR Capital; QueensBridge Venture Partners; Radical Investments LP; Scentan Ventures; Scount Venture; SeedInvest; StartCaps Venture; Tekton Venture; Walden Woods Equity CrowdHoldings; Western Technology funding Aug-16 Investment

VR/AR Project

Hong Yuan; Lucy Lv

Greg Castle; Ralph Gootee; Tracy Young Jakob Tolleryd; Lars Molinder; Lars Thunell

Bradley C. Harrison; Mark Cuban

New Amsterdam Media

AR/VR Company

119 NextVR

Focus

Investment ($M) Funding round

broadcasting; wntertainment; VR 80 VR; Media Enter-

120 Boundless Reality tainment; Film

122 Matterport

VR; E-Commerce; Food & Beverage -VR; Digital Media; 3D Tech --

123 Qoobex

VR; Broadcasting; Social Network; 3D Technology --

124 3DLOOK

AR; 3D Tech; Celebrity

121 Coolhobo

125 Alphachannel Drone League 126 Racing 127 Briskeye

0.03

AR; Social Network; Marketing 0.1 Drones; Racing; VR 1 VR; Video; Broadcasting 1.28

130 InstaVR

Content; Developer Tools; VR 1.5 AR; Printing; Mobile 1.65 VR; SaaS; mobile apps 2

131 Polyarc

VR; gaming

128 TimefireVR 129 LifePrint

132 LiveLike 133 Vertebrae 134 ROOBO 135 Osso VR 136 Embodied Labs

137 TinyPCB

3.5

broadcasting; sports; VR 5.01 Advertising Platforms; VR 10 Robotics; AI; VR; IoT 100

Date

Series B

Investors (Firms) CITIC Group Corporation; Comcast Ventures; Dick Clark Productions; Formation 8; RSE Ventures; The Madison Square Garden Company; Time WarAug-16 ner Investments

Seed

Aug-16

Seed

Aug-16 Chinaccelerator; SOSV

Venture

Aug-16 CBRE Group; Luminari Capital

Angel

Aug-16

Sep-16

Seed

Capital Innovators; Newark Sep-16 Venture Partners

Seed

Sep-16 Sky; Courtside Ventures

Venture

Sep-16

Private Equity

Cavalry Fund; Hudson Bay Sep-16 Capital Management

Debt

Sep-16

Series A

Sep-16 The VR Fund Colopl VR Fund; UCCVR; Sep-16 Vulcan Capital CAA Ventures; Courtside Ventures; Dentsu Ventures; Evolution Media Partners; Residence Ventures; Techstars Sep-16 Ventures

Series A

Sep-16

Venture

Sep-16 Iflytek Co Ltd

Seed

Sep-16 Presence Capital

VR; Healthcare

0.025

Seed

Oct-16 Creative Startups

Debt

Oct-16

Wearables; Semiconductor; Medical Device; Robotics; VR 0.25

contributions@vrarproject.com

VR/AR Project

Peter Guber

William Bao Bean

Igor Eremin; Vadim Rogovskiy (Seed)

Debt

Seed

Investors (People)

Craig Vachon Francisco Pereira

David Stern

New Amsterdam Media

AR/VR Company

138 Hyperverse 139 moBack

Focus VR; Gaming VR; AR; Mobile Apps

Investment ($M) Funding round

Date

Investors (Firms)

0.4

Debt

Oct-16 Starta Accelerator (Seed)

0.5

Seed

Oct-16

0.564

Undisclosed

Oct-16

0.891

Undisclosed

Oct-16

Seed

Brightstone Venture Capital; Frontier Tech Ventures; GREE Oct-16 VR Capital; HTC Corp

Venture

Oct-16

John Ostrem

141 Univrses

AR; VR; Web Development; Gaming VR; AI; Virtual Assistant

142 VRChat

VR; social

First Contact 143 Entertainment

VR; Digital Entertainment; Gaming 5

144 Iris VR, Inc.

VR; 3D Tech; Architecture

Series A

145 InContext

Retail; software; VR

15.2

Series F

Series B

Emergence Capital Partners; Indicator Venture; Valar VenOct-16 tures Beringea; Intel Capital; Hyde Park Venture Partners; PlymOct-16 outh Ventures Horizons Ventures; Evolution Media Partners; LDV Partners; Shanghai Media Group; Youku Oct-16 Global Media Fund

100

Venture

Oct-16

Seed

Oct-16 B Dash Ventures; Speee

Angel

Nov-16

0.311

Seed

Nov-16

0.335

Angel

Nov-16

140 iGOTCHA

146 Baobab Studios 147 Kernel Co. 148 VRIZE

8th Wa AR; Hardware; Software VR; Developer Platform

1.2

Investors (People)

Bryan Johnson

Musical Instru-

Taction Enterpris- ments; VR; Con149 es sumer Electronic 0.095 VR; Information Technology; Medical

150 PsycApps 151 Virtual SugerySIM VR

152 Hashplay Inc. 153 Bublar

VR; Internet; Mobile

0.5

Venture

Nov-16 Metatron Global

AR; VR; Gaming

0.557

Undisclosed

Nov-16

Seed

Nov-16

Seed

Nov-16 Belmore Capital SV Tech Ventures; Sand Hill Angels; SIG China; StanfordStartX Fund; Youku Global Nov-16 Media Fund

155 NGCodec

VR; Developer APIs; Data Visualization 1 VR; Cloud; AR; Wireless 1

156 Tactical Haptics

Hardware; Gaming; VR 1.47

154 AppOnboard, Inc.

contributions@vrarproject.com

Seed VR/AR Project

Alexander El Gammal; Klaus-Peter Stegen

New Amsterdam Media

AR/VR Company

157 Fieldbit

Focus AR; Enterprise Software

Investment ($M) Funding round 2.21

Nov-16

Seed

Betaworks; Greylock Partners; Norwest Venture Partners; SV Angel; The Venture Reality Fund; Third Kind Venture Nov-16 Capital

159 nDreams 160 Soul Machines

AI; E-Learning; VR 7.5

Series A

161 obEN

AI; IoT; VR 7.7 AR: AI; Enterprise Software 18

Series A

162 Infinity 163 Glo Inc 164 INKHUNTER 165 Rain 166 VRee 167 WTFast 168 Xanadu Heights

VR; Content; Digital Entertainment -AR; Computer Vision; Apps -AR; VR; 3D Tech; Computer Vision -VR; Information Technology --

Investors (Firms)

Venture

AR; VR; Computer Vision; Mobile 2.4 VR; Internet; Publishing 2.49

158 8th Wall

Date

Venture

Nov-16 Mercia Technologies Horizon Ventures; Iconiq Nov-16 Capital CrestValue Capital; Cybernaut Westlake Partners; Dream Maker Entertainment; E3 Capital Partners; Idealab; Inmaginator; NewDo Venture; Shenzhen Leaguer Venture Nov-16 Capital; Third Wave Digital

Debt Debt

Nov-16

Seed

Nov-16 BOM Capital; LUMO Labs

VR; Gaming; SaaS; Content Delivery Network --

Series A

Nov-16 VA Angels

VR; Video Games --

Seed

Nov-16 Redcliffe Capital

169 GlassUp 170 Reality Reflection VR; Gaming -171 Kabaq AR; Food Menu 0.18 172 Taqtile

AR; VR; Enterprise applications 0.5

173 Neurable

AR; VR;Human Computer Interaction 2

Drifter Entertain174 ment VR; gaming 360-degree video;

175 Visbit VR contributions@vrarproject.com

Gordon Cheng

Post IPO Equity Nov-16 Alibaba; Sun Corporation

Nov-16 MindWorks Ventures Kayon Partners; Entrepreneurs Nov-16 Roundtable Accelerator (Seed)

AR; Consumer Electronics; Sports; Hardware 0.27

Investors (People)

Seed

Equity CrowdEuropean Union, UniCredit funding Nov-16 Group Venture

Federico Palumbo, Francesco Giartosio

Nov-16 Stonebridge Capital Dec-16 The Glimpse Group

Debt

Seed

2.25

Seed

3.2

Seed VR/AR Project

Dec-16 Lighter Capital Loup Ventures; NXT Ventures; Dec-16 PJC Anorak Ventures; Initialized Capital; Pathbreaker Ventures; Presence Capital; Signia VenDec-16 ture Partners; The VR Fund Amino Capital; COLOPL; Ever Sunny; Presence Capital; Dec-16 ZhenFund

Brian Shin;Robert Winter

New Amsterdam Media

AR/VR Company

Focus

Investment ($M) Funding round

Date

Investors (Firms)

176 STRIVR Labs

AR; VR; Enterprise 5

177 AxonVR 178 Zero Latency

AR; VR; Hardware 5.8

Seed

VR; gaming

Venture

179 Atheer

AR; hardware; HMDs

Series B

180 High Fidelity

VR; social

Venture

181 Survios

Hardware; Gaming; VR 50

Series C

AR; Wearables; Consumer Electronics 58 AR; Audio; Virtualization --

Series A Non Equity Assistance

Dec-16 Carthona Capital Signatures Capital; Streamlined Ventures; FundersClub; Dec-16 RONAholdings; Shanda Group Breyer Capital, IDG Capital Dec-16 Partners Lux Capital; MGM; Danhua Capital; Dentsu Ventures; Felicis Ventures; Shanda Holdings; Dec-16 Shasta Ventures; WiL 21st Century Fox; Shenzhen O-film Technology; Vanfund Urban Investment & DevelopDec-16 ment Skydeck | Berkeley; Keiretsu Dec-16 Forum Northwest

Seed

Dec-16 LUMO Labs

Seed

Dec-16 Startup Wise Guys

0.15

Seed

Dec-16

Angel

Dec-16 Asimov Ventures

Oster Design 182 Group

183 Owlized Pillow’s Willow VR 184 Studio VR; Gaming

VR; 3D Tech;

185 Wolfprint 3D Gaming Virtual Xperience 186 Inc. VR; Real Estate

Series A

Dec-16 Signia Venture Partners Dawn Patrol Ventures;Keeler Dec-16 Investment Group

187 Obsess

VR; E-Commerce, Retail 0.02

188 Pandora Reality

VR; AR; Gamification; Enterprise 0.1

Seed

Jan-17 BlueCap, ETohum

VR; Photography; Social Media; Mobile; Computer Vision 0.2

Private Equity

Jan-17

AR, SaaS, mobile 0.5

Seed

Jan-17 Turn8, Prime Venture Partners

191 Remoria VR

VR; Software Engineering; Information Technology 0.74

Seed

Invitalia Ventures; LVenture Jan-17 Group

192 Mimesys

VR; AR; Collaboration; Enterprise 1.07

Angel

193 SkyLights

VR; Digital Entertainment; Air Transportation

Seed

189 FOV 190 Pixelbug

contributions@vrarproject.com

1.3

VR/AR Project

Investors (People)

Jan-17 LRM Bragiel Brothers; Comcast Ventures; David Gurle; Plug and Play; Presence Capital; Y Jan-17 Combinator

Dick Costolo; Jon Snoddy; Paul Matthaeus

Ali Rıza Babaoğlan, Hüseyin Emre Yılmaz

Koen Desmedt, Frank Bekkers

New Amsterdam Media

194 Spaces

Computer, Software, VR 6.5

Venture

195 Digilens

AR; Product Design; Manufacturing 22

Series B

Investors Investors (Firms) (People) Boost VC, Canyon Creek Capital, Colopl VR Fund, Comcast Ventures, CRCM Venture Capital, GREE, Sinovation Ventures, The Venture Reality Fund, Jan-17 Youku Global Media Fund Kai Huang Alsop Louie Partners, Bold Capital Partners, COntinental AG, Dolby Family Ventures, Foxconn Technology Group, Nautilus Venture Partners, Jan-17 Panasonic, Sony

196 Lumus

AR; Wearables; Consumer Electronics 30

Series C

Crystal-Optech; HTC; Quanta Computer; Shanda Group, Jan-17 Alibaba Group

AR; VR; Media and Entertainment; Video Games --

Seed

Jan-17

AR/VR Company

197 Binary Bubbles 198 VR Connection

Focus

Investment ($M) Funding round

Date

Jan-17

Seed

CRCM Venture Capital, IDG Capital Partners, Lux Capital, Shenzhen Valley Venture, SV Tech Ventures, Youku Global Jan-17 Media Fund

200 Skullfish Studios 201 Flexound Systems VR; AI; Music 0.97 202 Virtualistics VR, AR, software 3

Seed

Jan-17 GameFounders

Venture

Jan-17

Seed

Jan-17

203 Lithodomos VR

0.68

Seed

Jan-17

Seed

Feb-17

Debt

Feb-17 Social Finance

Series A

Feb-17 WPP Ventures

199 Arraiy

204 vGolf Agility Technolo205 gies Corp. 206 SubVRsive 207 NUVIZ 208 uBUTLER

Machine Learning; AR; VR 3.86 VR; Graphic Design 0.03

VR, Software, Architecture Mixed Reality; Golf

VR; Homeland Security; Communication Hardware 0.32 VR; Marketing; Advertising 4 AR; Consumer Electronics, Automotive, IoT --

Husqvarna, Pankl Racing Systems, Pierer Industrie, WP Feb-17 Suspension

Virtual Goods, VR --

Seed

Feb-17

209 AccuPS

3D Technology, VR0.1

Seed

Feb-17

210 8i

3D Technology, Computer, Software, VR 27

Series B

Time Warner Investments, Baidu Ventures, Hearst Ventures, Verizon Ventures, One Ventures, Seen & Speed Feb-17 Ventures

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

AR/VR Company

AR; Software; VR 5

Venture

Investors (Firms) Acequia Capital (Acecap), Anorak Ventures, Betaworks, First Round, Maveron, Sequoia Capital, The Venture Reality Feb-17 Fund, Vulcan

Venture

Feb-17

213 Wemersive

Education; VR VR; 3D Engines; Mobile

214 Schell Games

AR; VR; Connected Toys, Interactive Experiences

211 Against Gravity Immersive VR 212 Education

Dreamscape Immer-

215 sive

216 Lytro 217 LiveEdu.tv 218 Walkthrough 219 Imaginate

Focus

Gaming; VR

Investment ($M) Funding round

1.06

11.36

Photography, Hardware, Software, Film, Video, VR 60 VR; Education; Video Residential, Real Estate, VR 0.33 AR, Software, VR 0.5

Total Funded

Date

Venture

SBIR; National Institute of Health 21st Century Fox, Bold Capital Partners, IMAX, MGM, Warner Feb-17 Bros, Westfield Corporation Allen & Company, Blue Pool Capital, Danhua Capital, EDBI, Foxconn Technology Group, GSV Capital, Huayi Brothers Feb-17 Media Group Angel Round Capital, EuropeanPioneers, Y Combinator

Angel

Jan-17

Seed

Feb-17 SRI Capital

Series A

Series D

Investors (People)

Steven Spielberg

Barry Sternlicht

$2,344.59

R e s e a r c h co m pi l e d by Alice Shen and Michael Hoydich

*THIS DOES NOT I NC LU D E E VE RY I NVE ST M E NT I N T H E S PAC E , A N D D O E S N OT R E P R E S E N T T H E TOTA L FU N D I N G TO DAT E .

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

NOTABLE PROJECTS

Th e fo llow i ng l i s t fe a tur e s l ead ing V R p r ojects as well as som e n o t a ble r e c e n t fe s t i v al e ntr ies, all w ith links.

ALLUMETTE

STUDIO: PENROSE STUDIOS DIR. EUGENE CHUNG “ Si x d eg r e es of freedom i s hug e . I t adds so m u c h t o w ha t w e ca l l ‘ pres ence ,’ wh ich i n m an y wa ys i s t he hol y g ra i l of vir tu al r ea l i t y. H ow do w e m ov e a nd th in k in t h i s m ed i u m ? A l l um et t e i s a bout th in kin g n a t i v el y i n v i r tua l rea l i ty. ” — E u g en e C hung A l l u m et t e br in g s v ie we r s in t o a magi cal c i t y i n t he c lo u ds — D r. Se u ss meets C i r qu e d e So le il. We win d t h r ough its st r e e t , fo llowin g a lit t le g ir l named A l l u m et t e a n d h e r mo t h e r. Th e n avigati ondr i ven st or y u s e s t h e powe r of r oomsca l e V R be a u t if u lly. A llu me t t e el evates t h e p l a t fo r m wit h o n e o f t h e fir st tr uly o r i g i n a l c in e ma t ic V R e x pe r ie n c es. To see a 2 D c lip, c lic k here Av a i l a bl e f ro m: V ive po r t , St e a m , O culus S t or e

CLOUDS OVER SIDRA

STUDIO: WITHIN D I R E C TO R S : G A B O A R O R A , CHRIS MILK “ B y l ev e rag i ng brea k throug h techn o lo gie s su c h as v i r t ua l rea l i ty, w e ca n cre ate so l i d a r i t y w i th t hos e w ho a re n o r mally ex cl u d e d a nd ov er l ooked, a m pl i fyin g th e ir v o i c es an d expl a i ni ng thei r s i t ua tio n s .” — G a bo A ro r a

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

T h e g r o u n dbre a k in g C lo u ds Over Si dr a w a s on e of t h e fir s t pro je c t s t o prove VR ’s p ower a s a n “u lt ima t e e mpa t hy m achine ,” an d a p o t en t ia l a g e n t o f s o c ia l change . S i d r a br i n g s v ie we r s in t o t h e wor l d of a 1 2 - yea r o l d g ir l n a me d Sidr a , who guides th e a u di en ce t h ro u g h h e r t e mpo r a r y home: th e Za a t a r i R e f u g e e C a mp in Jo rdan. S i d r a h e l p e d UNIC E F do u ble it s donati on r a t e ; a f t e r o n e s c re e n in g , t h e U N’s H u m a n i t a r i a n Ple dg in g C o n fe rence i n K u w a i t r a i se d $3. 8 billio n — n e a r ly tw i ce th e pr o j ec t ed re ve nu e s . To see a 2 D c lip, c lic k here Av a i l a bl e f r om: W it h in a pp, Yo u Tu b e 360

DEAR ANGELICA

S T U D I O : O C U L U S S TO RY S T U D I O D I R E C TO R : S A S C H K A U N S E L D “ I t sh ou l d fe el l i ke a l uci d drea m i n th at way t h a t t h i n g s appea r a nd di s a ppea r i n a n e ar ly e ffor t l e ss way. We ca n us e tha t to basically s tr i n g u p t h es e na r ra ti v e m om ent s . ” — Sa sc h k a Uns el d 2 0 1 7 ’s D ea r A n g e lic a is another b r ea k t h r o u g h : a n e x pe r ie n c e c re a t ed fr om i nsi de V R sp a c e , r a t h e r t h a n o u t s ide of it. An g e l i c a w a s a u t h o re d in Oc u lu s ’ Q uill, a too l t h a t a l l ows de ve lo pe r s a n d ar tists to c r e a t e e x pe r ie n c e s wh ile t h e y a re movi ng th r o u g h t h e ve r y s pa c e t h a t t h e audience w i l l oc c u py. D ea r An g e l i c a o c c u r s in t h e wor l d of a yo u n g wo m a n n a me d Je s s ic a — a boundless, ab st r a ct wo r ld, fille d wit h pa in ter ly 3D s h a pe s. M u si c a n d A n g e lic a ’s vo ic e ( Geena D av i s) fi l l o u t t h e wo r ld o f t h e gr i eving d a u g h t e r, a s we wa lk t h ro u g h h e r m emor i es o f h e r m o m — wit n e s s in g f r a g me n ts of the s t or y, a n d o f A n g e lic a ’s me mo r ie s , in a stor y p a i n t ed i n sp a c e a n d s o u n d. To see a 2 D c lip, c lic k here Av a i l a bl e f r om: Oc u lu s St o re contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

GNOMES AND GOBLINS STUDIO: WEVR D I R E C TO R : J O N FAV R E A U

“ D o n ’t b e a f ra i d to expl ore a nd l ook an d tr y d if fer en t t h i n g s a nd l ook ev er yw here . Eve n if y o u d o i t t h roug h once , do i t a g a i n.” — J on Fav r e a u A m a g i c a l c o mbin a t io n o f V R a n d game p lay, G n om e s a n d Go blin s s e t s player s in th e m i ddl e of a n in t e r a c t ive e n chanted fo r est , t e e m in g wit h fire f lie s , t iny cottages, an d r ope b r idg e s . Yo u e x plo re t h e vi l l age , an d bu i l d p l ay f u l re la t io n s h ips wit h cr eatur es al o n g t h e w ay. Gn o me s s u s s yo u out, w al k up t o you , ma in t a in e ye c o n t a c t , and toss yo u m a g i ca l o bje c t s . Yo u c a n r in g a bell to aw a ken m o r e fire f lie s , play fe t c h , or open w i n dow s. To see a 2 D c lip, c lic k here P r ev i e w av a i la ble f ro m: V ive po r t , Steam

HENRY

S T U D I O : O C U L U S S TO RY S T U D I O D I R E C TO R : R O M E R O L O P E Z DA U “ W h e n w e set out to m a ke Henr y, i t was a s te p i n t o t h e unk now n w or l d of m a kin g an e m ot i o n a l VR m ov i e . ” — Ra m i ro L opez Dau Hen r y t r a n s po r t s t h e v ie we r into the m a g i ca l h o m e s o f a n a do r a ble h e dgehog na m e d H en r y — a s t h e lo n e g u e s t at his b i r t h day p a r t y. Oc u lu s ’ po s it io nal and e ye t r a ck i n g e n a ble H e n r y t o lo ok at you b e see c h i n g ly … t ill yo u fo r g e t t h at you’r e l o o k i n g a t a n illu s io n , n o t a liv in g br eathi ng c r ea t u r e . I t wo n a n E mmy in 2016. To se e a 2 D c lip, c lic k here Av a i l a bl e f r om: Oc u lu s St o re

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

THE LAST GOODBYE

S TUD I O : H E RE BE D RAGONS D I R E TORS : GA BO A RORA , A RI PA LI T Z In J u ly o f 2016, Ho lo c a u s t s ur vivor P in c h a s G u tt e r t o u re d t h e Majdanek C on ce n t r a t i on C a mp in wh a t h e vowed wo u l d b e h is fin a l v is it . B y ma r r yi ng a s t er e o v i d e o c a pt u re o f Pin c h a s w ithin a p h o t or e a l i st i c ro o m s c a le e x pe r ie nce , T he L a st G o o dbye re a c h e s pro fo u n d l evels o f i m m er si o n in s e r v ic e o f t h e fir st ever V R t e st i m ony t h a t will be a rc h ived and p r eser ve d. T h e impo r t a n c e o f lis t eni ng to P in c h a s’ st o r y is mo re impo r t a n t now than e ver a n d t h i s is a ls o a be a u t if u l t estament to l ove , co m pa s s io n a n d t h e h u ma n spi r i t. To see a 2 D c lip c lic k here N o t yet av a i l a ble

LIFE OF US

S TUD I O : WITHIN D I R E C TORS : A A RON MILK

KOBLIN,

CHR I S

“ For som e r e a s on, hum a ns ha v e this fu n n y t h i n g ab ou t w here w e ca m e from — i t always ha s far m o r e em ot i ona l w ei g ht t ha n wh e re we ar e .” — C hr i s M i l k L i ke D e a r A n g e lic a , Life o f Us premier ed a few we e k s a g o a t Su n da n c e . Th e sevenm i nu t e pi ec e c o n s is t s o f s c e n e s f ro m acr oss ti m e a n d sp a c e , c o mpr is in g wh at studio W i t h i n r e fer s t o a s “t h e c o mple te stor y o f l i fe on E a r t h .” Life o f Us is a highly i nt er a ct i ve , s o c ia l V R e x pe r ie n c e that toes th e l i n e be t we e n g a me a n d n a r r ative . U p to fou r p e o ple c a n e x pe r ie n c e evol uti on tog et h e r, e vo lv in g f ro m s in gle-cel l ed o r g a n i sm s, fi sh , din o s a u r s , a pe s , a n d mor e . To see a 2 D c lip, c lic k here N o t av a i l a bl e ye t

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

THE MARTIAN VR EXPERIENCE

S TUD I O : THE VIRTUAL REALITY COMPANY D I R E C TOR: ROB E RT STROMBE RG (B AS E D ON RID LE Y SC OTT’ S FE AT U R E F IL M ) “ A n ew m ed i u m g i v es oppor tuni ti es to n ew a r t i st s , a n d w e a re cul ti va t i ng th at. We d on ’t se e v i r tua l rea l i ty a s a m ar ke tin g t ool — w e se e i t a s a new wa y to v isu alize e n t e r t a i n m en t . ” — Ted S chi l ow i tz , FOX I n n ovat i o n L a b T h e M a r t i a n V R E x pe r ie n c e wa s c r eated as a su ppl em e n t t o 20t h C e n t u r y F ox’s T he M a r t i a n . T h e v ie we r s e e s M a r s thr ough s t r a n d e d E a r t h lin g Ma r k Wh a t n e y ’s eyes… an d mu st fi gu re o u t h ow t o g e t back to Ea r t h . A s you s o r t yo u r fo o d, pla nt a flag, an d ope r a t e a rove r, c r a n e , a n d rocket, the i nt er a ct i o n s de e pe n t h e e x pe r ie nce , and hei g h t e n t h e s e n s e o f pre s e n c e . To see a 2 D c lip, c lic k here Av a i l a bl e f r om: Oc u lu s St o re , St e a m

NOTES ON BLINDNESS: INTO DARKNESS D I R E C TORS : PE TE R MID D LE TON, J AMES S PI N N E Y

T h i s i m m e r sive e x pe r ie n c e do c ument’s Jo h n H u l l ’s j o u r n e y wit h blin dn e s s. After l osi n g h i s sig h t in 1983, H u ll began re c o r di n g a n a u dio -dia r y. Th is dia r y i s the b a si s of t h e e x pe r ie n c e , a n d po nder s a wo r l d b e yon d s ig h t , t o e x plo re t h e i nter ior wo r l d o f bl i n dn e s s . To see a 2 D c lip, c lic k here Av a i l a bl e f r om: G o o g le Play, Oc u lus Stor e , W i t h i n a pp contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

NE URO S P E C UL AT I V E A F RO F EM INISM STUD I O : H YPHEN -L A BS D I R E C TORS : A SHLE Y B AC C US-C LAR K, C A RM E N AG UILA R Y W E D GE , E C E TAN K A L , N I TZA N B A RTOV N e u r oS pe c u la t ive A f ro Fe min is m i s an e m pa t h i c e x plo r a t io n in t o t h e e x pe r i ence of wo m en of co lo r. It t a ke s pla c e in a futur i stic c o sm e t i c s sa lo n (a n e u ro c o s me tics lab) , w h en h u m a n it y h a s t r a n s c e n de d its bodily l i m i t a t i on s. To see a 2 D c lip, c lic k h e re Av a i l a bl e f r om: Oc u lu s St o re

PEARL

S TUD I O : GOOGLE SPOTLIGHT STORIES D I R E C TOR: PATRIC K OSB ORNE “[V R i s] a v is u a l me diu m, t h a t ’s how we w a n t t o c o mmu n ic a t e … Yo u ’re t r ying to tou ch p e o pl e wit h yo u r a r t wo r k . — Pa t r i ck O s bo r n e Pea r l t a ke s pla c e in o n e c a r ove r twenty ye a r s. T h e v i e wer is in t h e pa s s e n g e r s e at, as the l an d sca p e c h a n g e s a n d c h a r a c t e r s gr ow. W i t h ou t d i alo g u e , it t e lls t h e stor y of a f a t h e r a n d a da u g h t e r a s s h e g r ow s up. D i st r i bu t e d i n bo t h 360 a n d 2D, Pear l is nom i n a t e d fo r a n A c a de my Awa rd for Best An i m a t e d S h o r t To see a 2 D c lip, c lic k here Av a i l a bl e f r om: V ive po r t , Yo u Tu be G o o g l e Pl ay

contributions@vrarproject.com

360,

VR/AR Project

New Amsterdam Media

THE PEOPLE’S HOUSE

S TUD I O : F E LIX A ND PAUL D I R E C TORS : FE LIX LA JE UNE SSE, PAU L R A PH A Ë L Th e Peop l e’s Hous e t a ke s t h e a u dience on a h i st or i c v i s it o f Pre s ide n t Ba r a c k O bama an d F i r st L ady M ic h e lle Oba ma ’s Whi te H o u se . T h e Oba ma s g u ide t h e a udience al o n g a j o u r ne y in s ide t h e We s t Wing and P r i v a t e R e si de n c e s . Th e y t a lk a bo ut thei r ti m e t h e r e , a n d t h e bu ildin g ’s h is t or y. To see a 2 D c lip c lic k here Av a i l a bl e f r om: Oc u lu s St o re

THE PROTECTORS: WALK IN THE RANGER’S SHOES S TUD I O : HERE BE DRAGONS D I R E C TORS : K ATHRYN IM RA A N I S MA IL

BIGELOW,

Th e P r o t ect or s c h ro n ic le s a day in the l i fe o f t h e r a n g e r s in G a r a mba Na t io nal Par k. T h e se r a n g er s a re o f t e n t h e la s t line of d e fen se i n a r a c e a g a in s t t h e poacher s i nt en t on sl a u g h t e r in g e le ph a n t s for their i vo r y t u sk s. Th e r a n g e r s f a c e constant d a n g e r a n d e ve n de a t h , a t t h e s er vi ce of th e se sen t i en t , n o ble c re a t u re s . Av a i l a bl e f r om: Yo u Tu be 360, W it h in app To see a 2 D c lip c lic k here

RAINBOW CROW S TUD I O : BAOBAB STUDIOS D I R E C TOR: ERIC DA RNE LL

In t h i s e x pe r ie n c e , u s e r s s t e p inside a hea r t - w a r m i n g a n d mu s ic a l la n d . T he c a r e f r ee fo r e s t a n ima ls ima g in e s pr i ng w ill l ast for e ver. Howe ve r, win t e r c o m es and th e a n i m a l s s o o n re a lize t h a t t h ei r l i ves ar e i n da n g er. W h a t t h e y n e e d is a her o; w h a t t h e y n ee d is Ra i nbow C row . N ot yet av a i la ble To see a 2 D c lip c lic k here contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

TILT BRUSH

S TUD I O : GOOGLE D I R E C TORS : PATRIC K D R E W S K I L LMA N

H AC K E TT AND

“ W h e n I l e f t D i s ney i n 2012, I t old th e m it was b eca us e I k now t here’ s s ome th in g new c om i n g … “The g oa l [ of Ti l t B r ush is] to a n i m a t e n o t o n pa per but i n s pa ce , ” h e said. — Leg e n d a r y D i s ney a ni m a t or G l en Ke an e G o o g l e ’s T i l t Br u s h may be V R’s “k iller app” th u s f a r — a llowin g u s e r s t o c reate and i nt e r a ct w i t h t h e ir a r t in a c t u a l 3D space . I t rec e i ve d a P ro t o Awa rd fo r Be s t Gr aphical Us er I n t e r f a c e , a n d a Un it y Awa rd for Best V R E x pe r i en ce . Tilt B r u s h ma r ke d o ne of the fi r st st e ps i n ma k in g n a t ive a n ima t ion tool s m a i n st r e a m an d a ls o re c e n t ly in c o r por ated mu l t i - u ser, co lla bo r a t ive a bilit ie s . I t paved th e w ay fo r o t h e r s s u c h a s Oc u lu s ’ Medium an d Qu i l l . To see a 2 D c lip, c lic k here Av a i l a bl e f r om: St e a m, Oc u lu s St o re

TREE

S TUD I O S : NEW REALITY COMPANY, HERE BE DRAGONS D I R E C TORS : M ILIC A ZE C , WIN SLOW P O RTE R Ar m s be c o me br a n c h e s , a n d t h e body b e c o m e s t h e t r u n k . Fro m s e e d t o ful l est fo r m , t h e u se r a s s u me s t h e t re e ’s r ol e i n th e m a j est i c r a in fo re s t , a n d e x per i ences i ts f a t e fi r st ha n d. To see a 2 D c lip c lic k here N o t yet av a i l a ble

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

100

TREEHU GGE R : WAWO N A

S TUD I O : MARSHMALLOW LASER FEAST D I R E C TORS : B A RNA B Y STE E L, E RSI N HAN ER S I N , ROB I N M C NIC H OLA S This is an inter active installation that combines ar t , sc i en ce , da t a , e nv iro n me n t a lism and tec h n o l og y i n a n imme r s ive e nv ir onment. T h e u se r i s t r a n s po r t e d in t o a t re e’s secr et i nn er wo r l d. Th e lo n g e r s o me o n e hugs the tree , t h e de e pe r t h e y dr if t in t o “t r eetime ,” a d i m en si on t h a t lie s be yo n d t h e limit of o u r se n se s. To see a 2 D c lip c lic k here N o t yet av a i l a ble

THE WALK VR

S TUD I O : CREATEVR D I R E C TOR: ROB SILVA “ O n c e con su m er s ha v e ha d a VR ex p e r i e n ce b a s ed a round a s peci fi c pro je ct o r b ran d — t hey a re i nv es t ed — they’ve h ad a per so n a l ex per i ence a nd t hei r rel atio n sh ips wi t h t h e m at er i a l i s forev er cha ng ed.” — D a v i d S t er n, O w ner/ F ounder, C reate VR A ( r el a t i ve) o ldie bu t g o o die , Th e Walk VR wa s c r e a t e d a s a ma r ke t in g e x t ensi on of S o ny ’s J o e G ordo n Le v it t fe a t u re T he Wal k, by C r ea t e A dve r t is in g . In it , t h e u s er r el i ves P h i l i ppe Pe t i t ’s t e r r if y in g wa lk be t ween the two Wor l d Tr a de Towe r s . W h e n C r e a te V R o r ig in a lly de moed T he Wa l k , t h ey adde d h a pt ic e le me n ts: a cut ho se o n t h e f lo o r t o s imu la t e t he w ir e , an d a f a n t o s imu la t e win d. A s a r esul t, at l e a st 3 0 % of v ie we r s ju s t c o u ld not br i ng th e m sel ves t o wa lk t h e wa lk . To see w hy, wa t c h t h e c l i p be low. To se e a 2 D c lip, c lic k here Av a i l a bl e f r om: Go o g le Play, St e a m

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

101

OTHER GREAT TITLES ADR 1 F T, T h r e e On e Ze ro | here Af t e r S o l i t a r y, E mble ma t ic Gro u p/Fr ontl i ne PBS | here D re a m s o f “ O ” , C irqu e de So le il | F el i x and Paul Studi os | here Ha l c yo n , S yFy, Se c re t Lo c a t io n | here M r. Ro b o t V R , H e re Be D r a g o n s | here M y B ro t h e r ’s Ke e pe r, PBS D ig it a l Studios | here Ol d F r i e n d , We v r | here P ro j ec t S y r i a , E mble ma t ic Gro u p | here Sl e e py H o l l ow Expe ri e nce , Se c re t Location | here E ve: Va l ky r i e, C C P Ga me s | here

FESTIVAL SELECTIONS TRIBECA 2017 Alteration

S t u di o: Okio-Studio D i r e c t or : J ér ô me B la n qu e t

APEX

S t u di o: Wevr D i r e c t or : Ar ja n v a n Me e r t e n

Arden’s Wake

S t u di o: Penrose Studios D i r e c t or : E u ge n e C h u n g

Auto

P r oj ec t C r e a to r : St e ve n Sc h a rdt

Bebylon Battle Royale

S t u di o: Kite and Lightning D i r e c t or s: C or y St r a s s bu r g e r, Ik r ima Elhassan

Becoming Homeless: A Human Experience S t u di o: Vir tual Human Interaction Lab, Stanford University

Broken Night

D i r e c t or s: Alo n B e n a r i, Ta l Zu ba ls k y, Alex Vlack

Draw Me Close

D i r e c t or : J o r da n Ta n n a h ill

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

102

Extravaganza

D i r e c t or : E t ha n Sh a f t e l

Hallelujah

S t u di os: Within, Lytro D i r e c t or s: Za c h Ric h t e r, B o bby Ha l vor son, Eames Kol ar

The Island of the Colorblind D i r e c t or : S a n n e de Wilde

The Other Dakar

S t u di o: Electric South D i r e c t or : S elly Ra by K a n e

Talking With Ghosts

S t u di o: Oculus Stor y Studio D i r e c t or s: Ri c C a r r a s qu illo, Ro ma n Mur adov, Sophia FosterD i m i n o, M a r ia Yi

Testimony

S t u di o: Kaleidoscope VR D i r e c t or : Z o h a r Kfir

Unrest

S t u di os: S h e l l a Films , E x Nih ilo, Lit tle By Li ttl e Films D i r e c t or s: J e n n ife r Bre a , A ma u r y La Bur the

SUNDANCE 2017

Asteroids!

S t u di o: Baobab Studios D i r e c t or : E r i c D a r n e ll

Chasing Coral: The VR Experience Studios: View Into The Blue , Seaview 360 D i r e c t or : J ef f Or lows k i

Chocolate

S t u di o: Gentleman Hands D i r e c t or : Tyl er Hu rd

Hue

S t u di os: Marr y the Moon, Fr a me s t o r e Ventur es D i r e c t or : N i c o le M c D o n a ld

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

103

Melting Ice

S t u di o: Condition One D i r e c t or : D a nf u n g D e n n is

Mindshow

S t u di o: Mindshow D i r e c t or s: G il Ba ro n , Jo n n ie Ro s s , Adam Levin

Miyubi

S t u di o: Felix and Paul D i r ec t or : F e lix La je u n e s s e , Pa u l Ra phaël

Orbital Vanitas

S t u di o: BADFAITH D i r e c t or : S h a u n Gla dwe ll

Out of Exile: Daniel’s Story S t u di o: Emblematic Group D i r e c t or : N on ny D e La Pe ñ a

The Sky is a Gap D i r e c t or : R a ch e l Ro s s in

Through You

S t u di o: Oculus Stor y Studio D i r e c t or s: S a s c h k a Un s e lf

Zero Days VR

S t u di o: Scatter D i r e c t or s: Ya s min E laya t , E lie Za n a nir i

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

104

INTERVIEW: GRANT ANDERSON, JAUNT (2016) 17 *INTERVIEW CONDUCTED BY SETH SHAPIRO

JANUARY 2016.

G r a nt And er son is H ead of S tud io for J aunt V R , one of the lead ing p r ovid er s of p r e mium cinem atic V R exp er iences. J aunt l e a p t to p r om inence w ith its And r oid an d i O S ap p s, and w ith V R p ar tner s includ ing Pa ul McCar tney, the N or th Face , Disneylan d and ABC N ew s. T he com p any r ecent ly c o m p leted a $ 65 m illion S er ies C w ith i nve stor s includ ing Disney, Evolution Medi a Pa r tner s, Par ticip ant Med ia; and Chin a M e d ia Cap ital. P r ior to joining J aunt, G r an t wo r ked w ith com p anies includ ing S ony P i c tur es Disney, S tan L ee Med ia and Ap p le . Q. What brought you to into VR, and to Jaunt? Grant: I came here was because Jaunt was a frontrunner. They had their hands in the whole pipeline, from beginning to end. But what really drew me to VR from the film industr y is this is really truly the first new medium we’ve had in dozens of even hundreds of years since film, basically. Maybe video games. So, it’s a chance to be truly innovative and I think that’s what excites people. We’re creating, and building, and extending what is the language of cinematic VR. When do you get to do that, come in on the ground floor of a new medium? So that’s what I think is drawing most people here. Q. How would you describe Jaunt? Grant:. Well, Jaunt star ted about two years ago, primarily, as a technology company. I mean our founders, our CEO and CTO, were serial Silicon Valley entrepreneurs, so they star ted many different tech companies. And one of them was on vacation and said, “Wouldn’t this be great if we could share this with the world?” They had the idea of putting this camera together, a video camera system that could capture a full 360 ,image which led to VR and that research. And they created a series of cameras, eventually leading to our first production camera, called the GP16. It’s basically a collection of GoPros in a housing, but heavily modified along with our own software algorithm, created to stitch ever ything together in the cloud. That’s our cloud-based component, which is now called Jaunt Cloud Ser vices.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

105

So, they had, really, one of the first perfectly built VR cameras. And they star ted shooting content. And it turns out the content was really good. And so they got a lot of attention for it. And they quickly realized, of course, that content is king and really wanted to be in the content business. And so, Jaunt morphed into becoming a content and a distribution company. It’s the camera acquisition system on the frontend, the Jaunt cloud ser vices and a suite of software tools and platform in the middle, and then a distribution app on the end. And we’re hardware agnostic, both on the front end and the back end. Meaning, yes, we build cameras but we’re not really a hardware company. We build cameras as a means to an end. We’re also using [Nokia’s] cameras, and we use others. We’re really looking for the best tools for the job to create the best content for our platform. So, in the end, that’s what Jaunt really is, it’s a content and a distribution company. Q. What distinguishes great VR or transformational VR content from the stuff… that just isn’t interesting? Grant: I come from the 3D filmmaking industr y as of late, having been Executive Director of Sony’s 3D Center and doing 3D films. And that was the problem back then. There was a lot of junk in the early days and that’s true with VR. Ever ybody jumping in because it’s the latest, greatest thing. There’s a lot of hype out there, and a lot of creative potential, as we talked about. But there’s a lot of junk that’s coming out too. I mean, the good VR content is like any content, first and foremost, stor y drives it. Is it a good story, are you engaged? Nothing else is going to save that. Number two, there needs to be a high production value. Some of this stuff is incredibly difficult to look at, especially stereo production of which the best VR is in stereo and full 360. A lot of people are doing mono work. That’s okay, but I don’t think that’s the full experience because what you really want is a vir tual reality. And what everybody talks about is presence, the feeling that you’re actually there, or in the piece. And that’s what’s so transformative with vir tual reality. And so you really want those elements. So you need a good stor y. You need high production value. It needs to be comfor table to look at. You don’t want to be making your audience sick. There’s a lot of issues with the camera motions and movement and people tr ying to use their 2D sensibilities within this new cinematic language, which doesn’t always work. It’s a lot of trial and error but all those things have to come together to really feel present in the material. Q: Are there particular projects that stand out for you, either from Jaunt or others? Grant: The North Face: Climb was one of our most popular pieces, where you’re actually out in Moab, Utah, in Yosemite, and you’re on these huge mountain cliffs. We had the camera on a drone, which was the first time we ever did that and didn’t know how it would work but it actually worked out pretty well. It’s a beautiful, amazing experience that’s experiential, but also takes you on a little journey. Another really cool piece that we just did, which raced up the char ts was the Disney Small contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

106

World Experience, which we did during the holidays, where you’re actually on the Small World ride with Mickey and Minnie and going through that. So it just shows you the power of VR, where you can be in that front row seat at your favorite spor ting event or concer t. You can be on the ride of Disneyland, even if you may not be able to make it, so that’s really amazing. In the CG realm, one of my favorites of last year, which I’m a board member of the Advanced Imaging Society, and Chairman of their VR Committee. And, last night, we had the Creative Ar ts Awards, where we recognized the best 3D films and, now, high dynamic range and ultra HD. And, also, VR was our first award last night for that. And we handed one out for Sony and Create: LA, The Walk VR Experience. That was really one of my favorites of last year. It’s CG based, where you’re on the corner of the Twin Towers, walking out on that tightrope, just like in the film, looking down, and they put an actual rope under your feet. And it’s motion tracking, so as you’re walking out on the rope, you’re actually moving around in the full environment. And it is absolutely terrifying. It’s really an amazing piece. Q. Absolutely. One of the things that Create mentioned to me was that as many as 3 out of 10 people that try The Walk VR are like, “No, I can’t do this!” Even though they know it’s not real, that nothing about it can hurt them. That’s got amazing implications – for phobias, traumas, etc. Grant: Yeah. I was one of those people. My legs were shaking and you keep telling yourself, “You’re on the ground. You’re on the ground. You’re on the ground.” But your brain is going, “No way!” Q. As a filmmaker and as a 3D guy, where does VR fit, in terms of creating narrative? Grant: So, I always say, I think the killer app of vir tual reality is going to be at the intersection of cinema, gaming and interactive theater. So I think all those three elements to me, in my gut, seem to be like the magic combination. At Jaunt, right now, we’re focusing on cinematic vir tual reality – meaning, we’re filming the real world. We’re creating narratives in the real world, using the new languages of cinematic VR, whatever that is and whatever we’re building that out to be as a community. But, we also have limitations with what we can do with that right now. It used to be that doing CG was ver y difficult and filming something, “Just go film it,” was a lot easier. Well, now, it’s almost the opposite. Filming in vir tual reality is more difficult than doing things in CG because we have cer tain limitations. Unless you move the camera, you can’t move about the space. To do that, it’s going to take either physically scanning the entire space and turning it into a cg simulation, or light field technology, which we and other companies are working on, for sure, but that’s several years out. And we’ll still be limited in volume with which you can move around. So, on the flip side, you have the CG real-time graphics engines, in which you have full autonomy to move around and manipulate objects and control that. So, it’s ver y freeing but it’s also this synthetic world. So, we’re tr ying to bridge the gap where you have elements of contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

107

both, where you’re in a CG or film/cinematic VR environment. But, yet, we have interactivity on top, allowing you to control the path through the film, manipulate objects within reality of all those different kinds of things. But I do think you need some interaction, ultimately. Not in ever ything but on many projects, because you do feel so present in the material that you feel like you need agency in that world. Q. So for a filmmaker, the viewer traditionally has a predetermined, fixed position that’s dictated by the camera’s position, and by editing. In VR, the viewer can be the driver of that position. How does that change the process of crafting story? Grant: Well, two things there. One is the idea of your narrative – versus a narrative that a viewer can control. That’s one issue. I don’t think we’re not quite sure where VR is going to be on that. But I think it’s going to be a combination of the same three things: cinema, gaming, and interactive theater. This is something that the gaming community has been working on for around 20 years. How do you tell a compelling narrative that’s satisfying, but that the viewer has control over? It’s difficult. Really, really difficult. But I think we’re going to have to have some of that in VR. And, in terms of the window into the world, yeah, we don’t have a window anymore and that’s why it’s a windowless world, right? You can look anywhere you want. And so that’s where the challenge comes in production, where, how do we draw the viewers’ attention to what we’re tr ying to get them to look at? And that’s basically using the tools of motion, light, and sound. You can force the viewers’ perspective – but that’s been proven to not be ver y satisfying and also can make people sick. You could put arrows hanging in the air but that’s a little weird and obtrusive. So, you have to either use those cues of motion, light, and sound to guide the viewer or wait until the viewer looks – and use gaze detection and to let the stor y unfold once they finally do look there. One of the things I should mention is a lot of people think that now that we’re filming in 360 that we need to have action unfolding on the camera at all times like, “Isn’t this great? Look, people are running around the camera, or there’s multiple people performing around the camera.” That’s getting old really quick; it’s the equivalent of the 3D world and 3D filmmaking where somebody’s always poking you in the eye with something coming out the screen. So, I’m like, “Let’s just stop that right now.” Because you don’t want to feel like you’re in a tennis match when you’re watching something in VR. You really, physically, get tired ver y quickly. People, even in VR, still only look in a 30 to 60-degree arc, like in real life. Generally, you’re scanning side to side to your head, tiny bits. And ever y once in awhile, if you hear something, you look at something behind you, you’ll do that. So we need to take that into account in filmmaking. And, right now, a lot of it’s ver y gimmicky.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

108

Q: Do you see an inflection point that brings the medium to a critical mass? Or is one not required? Grant: You know what? It’s really undetermined. I mean, anybody that would tell you differently is a liar. Nobody knows what’s going to happen. And there’s all kinds of forecasts and all that kind of stuff, which are ver y lively, but we just don’t know. I mean, consumers are fickle so we’ll see if they take to it. But this is the year of VR. This is the launch pad, basically. This is year one. Because, all the three major headsets shipped, but those are going to be ver y niche. We know they’re not going to sell millions and millions of units. They’re targeted towards gamers. They’re expensive, if you’ve seen the prices that were announced, more expensive than we thought, including having to buy a powerful computer to actually run it. And so, it’s going to be targeted towards gamers, which itself is a niche. So it’s a niche of a niche. So there’s not going to be huge numbers. And so this is the same kind of feeling I had in 3D. There’s so much money and so much hype going into it, and then ever ybody disillusioned. Like, the trough of disillusionment, the hype cur ve and then the trough of disillusionment, and then you rise out of the valley of despair, all of that stuff. You can kind of see that here, hopefully it’s not. I think it’s different. It’s just so immersive and so compelling and you’re not aware of your real world. You really are transpor ted somewhere else. But, it’s going to be a niche and I think the press will see and say, “Look, all of them shipped and the numbers are small and VR is over.” There’s going to be that, for sure, probably. But, I think it will then definitely pick up. These things are going to get less expensive. They’re going to get smaller, lighter, more comfor table. And then, for us, we really view the biggest platform is going to be the cellphone. Because, ever yone has one in their pockets. This is all the technology that allowed VR to reemerge anyways. And people, companies, Apple, Samsung, others will star t building more and more specific VR tech into phones, to enable better experience, including head tracking and motion tracking and all that kind of stuff. And, it’s ver y inexpensive. There’s all kinds of little goggles on the market for 10 bucks you can get off Amazon. You can drop your phone in plastic headsets, and a million different Chinese companies, ver y inexpensive, and allows you to have a better experience. I mean Google Cardboard is great but you’re limited in the amount of experiences you can have because you have to hold it up to your face because your arms get tired. But there’s going to be a million different devices, and millions of different configurations. And the price points. Google Cardboard is essentially free. I mean they’re given out ever ywhere. So, we really do see the cellphone as one of the main entr y points, if not one of the main drivers, of the whole industr y.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

109

INTERVIEW: GABO ARORA UNITED NATIONS & LIGHTSHED (2017) 19 *INTERVIEW CONDUCTED BY SETH SHAPIRO

APRIL 2017

G ab o Ar or a is a aw ar d -w inning film m aker and Found er & Cr eative Dir ector of L i g htS hed , a vir tual r eality and social i mp a ct star t-up. H is wor k focuses on new te c hnologies that p r om ote social causes and m ake d ecision-m aking p r ocesses mo re inclusive . Am ong these p r ojects i s a vir al vid eo cam p aign for the r ecent c l i m ate change sum m it in Par is, and var i ous collab or ations w ith inter net i n f l uencer s to p r om ote accountab ility on g lo b a l h u mani t ar i an a i d a s sistance . H e has also d ir ected and pr o du c e d a s e r i e s o f p i o n e e r ing, and w id ely acclaim ed , vir tual re a lit y do c u me ntar i e s fo r t he U nited N ations focusing on vulner able popu la t io n s i n c r i s i s . Clo u d s Ove r S i d r a , Wave s o f G r ace and My Mother ’s Wing have all p re m iere d a t maj o r fi l m fe stivals ar ound the wor ld , includ ing Ca n n es, Su n d a nc e a nd Tr i b e c a, featur ed at the Wor ld Econom ic Fo r u m in Davo s , s c r e e ne d at t he White H ouse , and have exhib ited at t h e M u se um o f M o d e r n A r t’s inaugur al p r ogr am on im m er sive st o r y t ellin g . Background M y b a ck g r o u n d is a s a de ve lo pment economist. I studied emer ging mar kets at Jo h n s H opk i n s Un ive r s it y, po lit ic a l economy w i th F r ancis Fukuyama, w ho is my m e n t o r a n d th e pro fe s s o r wh o r a n the i nter national development pr ogr am w he n I w a s a t H o pk in s . B e fo r e t h a t , I s t u die d ph ilo s o phy and film at NY U. I alw ays had aspir ati ons t o b e a fi l m m a ke r, wr it e r, a r t is t . A n d then 9/11 happened, and I j ust felt that I nee de d m o r e life e x pe r ie n c e . I wa nted to get a lot mor e connected to w hat w as ha ppe n i n g o n t h e pla n e t . B a si ca l ly, I di d ju s t t h a t . I wa s f u ll-t ime at the U N by 2009. I wor ked i n the fi el d i n H a i t i , w a s i n C o lo mbia fo r a c o u ple of year s. I l i ved for a year in Bol i via, Tanzania, Z a m bi a . I ’ ve be e n in In dia wit h t h e Wor l d Bank. contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

110

B y 2 0 1 1 I wa s ba c k in Ne w Yo r k, and w as i n the office of the U nited Nati on s S e c r e t a r y G e n e r a l, Se c re t a r y -Ge n er al Ban Ki-moon, as a Seni or Policy Advisor. I wo u l d h el p wr it e re po r t s , bu t my goal then woul d be to get people to car e ab ou t th e i ssu e: i n f lu e n c e r s , de c is io n -ma ker s, gener al publ i c , ci vil soci ety. I l ooked for w ays to c o nv i n c e t h e m wit h mo re t h a n just number s. I’d a l so h ave t o e n g a g e wit h c e le br i ties, w r i te op- eds, and tr y to use tr adi ti on al fo r m s of g e t tin g o u r me s s a g e o u t . B y 2 0 1 2 - 1 3 , I wa s s e e in g blo g g e r s and technol ogy succeed at thi s. F or insta n ce , H u m a n s o f N e w Yo r k is a blo g g e r o n F acebook. He singlehandedly has mor e F aceb ook fo l l ower s t h a n a ll o f t h e UN g roups and soci al media teams combined… m or e e n g a g e m e n t , fo r s u re , be c a u s e pe o ple ar e passi onate about the stor i es that he tel l s. I fel t t h a t we n e e de d t o e mbr a c e n ew technology. We needed to embr ace someth i n g th a t wo u l d be re le v a n t t o h ow p eople wer e consuming i nfor mati on and ma ki n g d e c i si on s. I n 2013, I did a v ir a l v ide o campai gn for the cl i mate change summi t, wh i ch ha s a l m ost 2 millio n v ie ws . It ’s c a lled “Keep the O il i n the Gr ound.” T hen I cal l ed up H u m a n s o f Ne w Yo r k , a n d pro duced his U N tour —a l ot of stor ytel l i ng would b e th a t h e wo u l d g e n e r a lly do o n t h e str eets of New Yor k, w hat that would look like i n Ir a q , i n t h e C o n g o. T h a t w a s a f u n da me n t a l e x pe r ie n c e for me , a bi g success. I t led to almost a mi l l i on m o r e fol l ower s fo r h im a n d o u r is sues i n seven weeks. J ust, agai n, pr oving out th e use o f n e w t ec h n o lo g ie s . At t h a t sa m e t ime , I h e a rd a bo u t VR . T his w as pr obably O ctober 2014. I ’d n ever tr i e d i t , c o u l d o n ly u n de r s t a n d it a bstr actly. But my i ntuiti on w as that if i t w as ha l f as am a z i n g a s p e o ple we re s ay in g it coul d be , then it would be somethi ng that wou l d b e a m a z i n g for g e t t in g pe o ple t o under stand our issues at the U N. Ar o u n d t h e s a me t ime , I wa s wo r king w i th the new ly-elected Nor wegi an p r i m e m i n i st er. A l o t o f t h e UN’s wo r k is i n Afr i ca, and she had never been to Afr i ca. So, I di d a t h r ee -c o u n t r y t o u r t o g e t her to under stand the main issues. And I tr i ed to d o som e t h i n g t h a t wo u ld g ive h e r the most authentic , most r eal w ay to under s tan d th e i ssu e s sh a pin g pe o ple ’s live s t her e . W h en I fi r st we n t t o A f r ic a , t o Na mi bia, in 1999, I had al l these mi sconcepti on s. I t too k a w h i l e fo r me liv in g in a s h a n ty tow n to even star t to have some under stand i n g o f w h a t pe o ple a re like . A n d I wa n t ed to get the pr i me mi nister to under stand m or e q u i ck ly. S o, w h en I h e a rd o f v ir t u a l re a lit y, I thought: i f we wer e abl e to give people access to o t h e r pe o ple ’s live s , may be t h a t woul d make al l the differ ence .

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

111

W h en I h e a r d a bo u t V R, I did wh a t I usually do, w hi ch i s to star t tal king to ever yb od y, w h er e ver i t to o k me , a s k in g pe o ple “how does this wor k?” T hen we wor ked on th e ON E c a m p a i g n wit h B o n o a n d U2. At the launch par ty, Bono and I spent 40 mi nu tes tal k i n g a bo u t t h e bo ro u g h o f Qu eens... because that’s w her e al l the I r i sh ar e , of c o u r se , a n d th e Ra mo n e s . A n d a t t he same par ty, I met someone fr om Samsung w h o wo r ked i n V R. CLOUDS OVER SIDRA Even t u a l ly, we pit c h e d t h e E dg e o n the idea of doing VR in a r efugee camp, an d h i s e ye s w i d e n ed, a n d t h o u g h t it wa s a gr eat idea. And he intr oduced me to Chr i s M i l k, w h o h a d be e n wa n t in g t o do s o mething l i ke thi s for a long ti me . T h e n C h r i s g ave me a de mo, h is a nd Beck’s Sound and Vision pr oj ect, that j ust bl ew my m i n d. A n d I k n e w r ig h t away t h at we had to do somethi ng together. And I sa i d , “ I have n o m o n e y. Bu t if yo u g ive me your camer a, I ’l l fi gur e i t out. I ’ll go dow n th er e . It’s g o i n g t o be g re a t .” An d t h a t w a s e n d o f Nove mbe r. Two weeks later, I ’m in a r efugee camp film i n g C l o u d s O ver Sidr a wit h B a r r y Po usman. And since then Bar r y and I have wor ked tog et h e r o n My M o t h e r’s Win g , Ground Beneath Her, and other pr oj ects. An d t h a t ’s h ow it h a ppe n e d. We g ot i t r eady for Davos, and then Sundance i n 2 0 1 5 , an d t h e r esp o n s e wa s in c re dible . But some peopl e didn’t get it, because th er e we r en ’ t t h a t ma ny h e a ds e t s a t t h e ti me . We had Samsung come fr om Kor ea w i th 1 5 0 h ea d set s t o D avo s a day be fo re the event. And some people just w atched i t as a st r et ch ed- ou t ima g e a n d didn ’ t like it. B u t we di d it , by h o o k o r by c ro ok. And then it began to have a l i fe of its ow n , b e c a u se i t w a s g re a t , a n d pe o ple r eacted to i t. T hat’s w hat got me i nto VR . Ch r i s an d Pa t r i c k Millin g Smit h o f H e re Be Dr agons have been ear ly i nvestor s i n getti n g th i s st u f f m a de , a n d ve r y s u ppo r t ing of all the wor k. T h e n I r em e mbe r a t Su n da n c e , I was l i ke , “T hanks a lot, Chr is. T hi s w as amazing.” An d he sa i d, “ We ll, we ’ ve g o t t o do mor e . We’ve got to do mor e… ther e’s no shor tage o f g r e a t st o r i e s t o be t o ld, s o yo u should keep doi ng i t. You’r e gr eat.” And, you kn ow, he’s a r ea l st ro n g me n t o r o f min e , and close fr i end now. WAVES OF GRACE Af t e r S i dr a , I me n t io n e d t o C h r is t hat I w anted to do somethi ng on Li ber i a, and th e Eb o l a c r i si s. He s a id, “t h a t ’s a ma zing. Let’s do i t.” I did Waves of Gr ace ther e , wh i l e Eb o l a w a s r a g in g . We t r ie d t o g ive people an under standing of w hat’s happe n i n g th e r e , e sp e c ia lly t h ro u g h t h e e ye s of a sur vivor. And then I thought “w hat next ? ”

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

112

MY MOTHER’S WING M y M ot h e r ’s W in g , o u r G a za film, wor ld pr emier ed i n Tel Aviv. I w as feel i ng that th i s tec h n o l og y i s ve r y powe rf u l t o g ive peopl e a sense of w hat it’s l i ke in places th at th e y co u l dn ’ t u s u a lly g o t o. A n d I thought that ther e is no pl ace people have l ess un de r st a n di ng o f t h a n G a za . Pe o ple don’t have access to Gaza, because you have to b e a p a r t of t h e UN, o r yo u h ave to be a jour nal i st, and even then, it’s a ver y l on g, ar d u ou s pr o c e s s t o g e t in t o Ga za . We we r e ve r y h e av ily wa t c h e d wh i l e we wor ked on the pr oj ect. T he shoot wasn ’t tec h n i ca l ly du r in g a wa r, bu t it wa s over the summer that i t happened, ther e wer e c o n st a n t ly sk ir mis h e s , lit t le bo mb i ngs, things alw ays happeni ng at night an d i n d if fer e n t pl a c e s . Th e re wa s a lo t o f tensi on. F i l m i n g t h a t wa s pro ba bly o n e o f t he most diffi cul t things, but also one of the m ost i nsp i r i n g , b e c a u s e yo u re a lize h ow b eauti ful the people ar e , how mi sunder stood th ey ar e , a n d h ow mo s t pe o ple a re ju s t l i ke you and I , just tr ying to get thr ough the d ay, get t h ei r k i d s t o s c h o o l, e a t s o me t hing, and l i ve their l i ves. It w a s a n i n st a n c e wh e re my UN c redenti als coul d be used able to be used for g ood . An d I a p pl i e d fo r t h e pe r mit . I lo bb i ed. I di d ever ythi ng I could. And just to be abl e to do a n d fi l m a n d pro du c e a ny t h ing out of ther e is a mi r acle , because you don’t see th a t mu c h st uf f , wh e t h e r it ’s in V R or not. M y M ot h e r ’s Win g wa s s h own o n t he str eets of I sr ael. I t w as amazi ng to give peop l e an u n de r st a ndin g t h a t t h e y wo u ld never have , because the r egular media do esn ’t gi ve t h a t k i n d o f pe r s pe c t ive . W h a t w a s eve n mo re t o u c h in g was that it has been at Tel Aviv. I t’s been at th e Je r u sa l em F i l m Fe s t iv a l. To h ave it wor ld pr emier e in Tel Aviv at Tel Aviv U niver si ty w a s a st on i sh in g . THE LAST GOODBYE T h e L a st G o o dbye is my e ig h t h pr oject i n VR . I t’s in r oom scal e , and w i l l pr em i er e at Tr i b e c a i n la t e A pr il. Th e La s t Goodbye i s the fi r st ar chived testi mony fr o m a gen oc i d e su r v ivo r fo r t h e Sh o a h Foundati on for the futur e , w hich i s incr edi ble . At e a ch st e p, I’ ve be e n lu c k y. B u t I think we get suppor ted because we’ve b een abl e t o sh ow impa c t wit h it — wit h our face to face fundr aiser s at U NI CEF dou bl i n g d o n a t i o n s. An d we ma ke s u re o u r proj ects ar e ar ti stically cr eative and technol ogi cal ly i nn ov a t i ve . S t o r y t e llin g -wis e , we t r y to do somethi ng differ ent each ti me , and we tr y to m a ke a d i ffe re n c e wit h t h e m, you know ? I t’s been a w ild r i de . I fee l i m m en se ly g r a t e f u l, a n d s o mew hat vi ndi cated, that people w ant mor e than j u st s e x a n d v i o l en c e f ro m t h is me diu m— thi ngs like Notes on Blindness, and the stu f f c o m i n g ou t of Ja me s Ge o r g e ’s s t u dio Scatter. I t’s pr oving the point that peopl e w an t c o n t en t t h a t ma t t e r s .

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

113

I t h i n k we n e e d t o c o n t inu e t o fig u r e out a w ay to suppor t th a t . I ’ m ver y t o u c h e d t h a t Oc u lu s VR for Good w as i nspi r ed by C l ou d s Ove r Sidr a a n d bu ilt a VR for Good pr ogr am ar ou n d t h a t mo de l. An d I h ope th a t we ma ke it a c c e s si bl e so that peopl e can c o n t i nu e t o te ll t h e s e s t o r ie s , a n d that ther e’s a space for th e m . I h o pe n o o n e fo r g e t s t h is sor t of moment, and I hope w h a t we ’ ve be e n a ble t o a chi eve w i th a l ot of the wo r k t h a t I ’ ve be e n a pa r t o f wit h so many amazing people . . . b e c a u se of c o u r s e it ’s n o t ju s t me . I t’s a l ot of people w h o g et b e h in d a lo t o f t h e s e t h i ngs. Hopeful ly it l eaves i ts m a r k o n h ow t h is in du s t r y will continue to evol ve , and p e o pl e wo n ’ t fo r g e t t h a t we n e e d to thi nk about it i n mor e th a n j u st t h e way we t h o u g h t a bo ut other content. I’m h o pe f u l , be c a u s e it ju s t s e e ms l i ke each time I go to a fe st i v a l , t h er e a re in c re dibly me a ni ngful , awesome stor i es. I j u st h o pe t h a t it ’s n o t ju s t t h is ear ly i ndi e per i od, and th e n i t g e t s a ll in t o c o mic bo o k s or w hatever it is. I hope th a t t h i s t y pe o f s t u f f is fo s t e re d. I think it w ill, because th e wo r l d ’s be c o min g a mo re u r gent, di fficult, fr actur ed p la c e , a n d p e o ple a re e x pe c t in g mor e fr om not only thei r s t or yt el l i n g , bu t f ro m t h e ir t e c h nol ogy. I ’m excited and hon or e d t o be be in g a pa r t o f t h a t.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

114

INTERVIEW: WILLOW BAY,

USC ANNENBERG SCHOOL FOR COMMUNICATIONS AND JOURNALISM (2017) 20 *INTERVIEW CONDUCTED BY PERISA BROWN

APRIL 2017

W i l low Bay, a veter an television jour nalist , d i gi t al new s ed itor and author, is the Dean o f t he Annenb er g S chool of J our nalism at the U niver sity of S outher n Califor nia. Bay al s o ser ves as S enior S tr ategic Ad visor for the H uffington Post and is a sp ecial cor r es p o n d ent for Bloom b er g Television. B as ed in L os Angeles, Bay has r ep or te d fr om num er ous sp ecial events for B lo o m ber g T V, i n c l ud i ng t he Milken I nstitute G lob al Confer ence and th e C lin t o n G l o b a l I ni t i a ti ve . Bay also hosted “ Wom en to Watch,” B lo o m ber g T V ’s p r i me ti me p r ogr am that p r ofiled the next gener atio n o f wo m e n l e a d e r s i n var i ous ind ustr ies. S he r egular ly inter views to p bu sin es s a nd tho ught l e ad er s includ ing for m er P r esid ent Bill Clin t o n , Pe p s i C hai r ma n a nd CEO I nd r a N ooyi, Tesla Motor s CEO El o n M u sk , V i c e P r e s i d e nt A l Gor e , I AC Chair m an Bar r y Diller, and fo r m er B e a r S t e ar n s CEO A l a n S chw ar tz . Fro m 2 0 0 7 to 2 0 1 4 , B ay w a s a S enior Ed itor at the H uffington Post, m a n a g in g e d i t o r i al c o nte nt a nd gr ow th initiatives for the on-line ne w s a n d c o mme n t ar y s i t e . Wor king closely w ith found er s Ar ianna Hu f fin g t o n a nd K e n L e r e r, s he m anaged the launch of over a d ozen ne w s, bu sin e s s a nd l i fe s ty l e c ontent ver ticals includ ing 8 m etr o p ages a c ro ss t he c o u n t r y. B ay i nter viewed investor War r en Buffet, Face boo k ’s S h e r y l S a nd b e r g a nd H is H oliness the Dalai L am a, for the si t e ’s fir st g l o b al l i ve - s t r e ami ng event. W illow is c u r r e ntly a V i c e C hair of the Boar d of Tr ustees of the L os An g ele s C ounty M u s e u m o f A r t and ser ves on the Boar d of Tr ustee s fo r t h e C e nte r fo r E a r ly Ed u c a tion. S he lives in L os Angeles w ith he r t wo so ns a nd hus b a nd , R o b er t I ger.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

115

Background I w a s a b r o a d c a s t jo u r n a lis t fo r t h e bul k of my car eer. Mor e r ecently, I w as a Se n i or Ed i t o r a n d t h e n a Se n io r St r a t e g ic Advi sor at the Huffi ngton Post. T hat allowed m e , at th e ve r y b e g in n in g o f t h e e x pa n s ion of Huffington Post, to wor k at the inter sec ti on o f co n t e n t c re a t io n a n d t e c h n o lo g y. T h a t ex pe r i en c e s h a pe d a lo t o f my thinking about j our nal i sm and communi catio n i n tec h n o l og y. T h e e dit o r ia l t e a m wo r ked si de by si de w ith the web pr ogr ammer s an d th e t ec h t e a m . Th e C TO wo u ld s it w ith the seni or editor s, and we woul d devel op new t oo l s a nd de ploy n e w t e c h n o l ogi es together, w hich is a differ ent model f r om w h a t I h a d see n be fo re in le g a c y medi a. F r om t h e r e , I c a me t o USC wit h a combinati on of l egacy and new media j our n al i sm e x p e r i en ce , an d wit h ma n a g e me n t exper i ence in the onl i ne space . Education and Technology I t h i n k i t ’s i m pe r a t ive fo r a ll o f o u r students, gr aduate and under gr aduate , to h ave a f l u en cy a n d e a s e wit h c o n t e mpor ar y technologies. And that includes emer gi n g tec h n o l og i e s. W i t h co di n g s pe c ific a lly, o u r g o a l is not to tur n our students into pr ogr ammer s. I t’s to i n t r o du ce t h e la n g u a g e o f c o ding, w hi ch r eal ly empower s jour nal i sts to cr eate , fi x, o r t we a k t h e t o o ls t h e y u s e t o communicate w ith thei r audi ences. I see i t as a l a n g u a g e r equ ire me n t ; ju s t u n der standi ng how that l anguage wor ks and how i t d ir ec t s a n d s h a pe s t h e t o o ls t h a t you use i s w hat’s i mpor tant. Q. How is virtual reality being integrated into the journalistic curriculum at USC? T h e p o i n t o f t h o s e c la s s e s is t o e x per i ment w i th these new technol ogi es, taking t h em ap a r t a n d p u tt in g t h e m ba c k t o g e t her agai n to develop a facility w ith somethi ng n ew i n t h e t ec h n o lo g ic a l s pa c e . A n d t h en usi ng them i n the ser vice of j our nalism, w h i ch m e a n s u si n g t h e s e t e c h n o lo g ie s t h r ough the lens of a j our nal i st. W h en I ca m e t o USC , o n e o f my g o a ls w as to par tner w ith j our nal i sts and technolo gi sts as we di d du r in g my t ime a t t h e H uffington Post. I thi nk it’s cr i tically impor tant th at th e t oo l s we c re a t e a n d de ploy be shaped w i th the val ues of jour nal i sm, w ith th e va l u e s of co mmu n ic a t io n . On e of t h e th in g s t h a t I h o pe we ’r e beginning to do as we move beyond the ear ly e x p e r i m en t a tio n ph a s e o f V R is t hat we begi n to think about and document b est p r a c t i c e s. W ha t a re t h e e t h ic a l c o nsi der ati ons of using VR ? Can we document, for e x a m pl e , t h e be lie f a bo u t V R, wh ic h i s that i t i ncr eases empathy by the ver y i nti m ate na t u r e of t h e s t o r y t e llin g ? It ’s o u r suppositi on that we can, but how do we k n ow th a t ’s t h e c a se ?

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

116

I t h i n k t h a t ’s s t a r t in g t o h a ppe n now. Car negi e- Knight i s doi ng i nter esting wor k, F r on t l i n e h a s be e n re a lly a c t ive in the VR space . I thi nk now that we’r e actu al ly s t a r t i n g t o u se it a s a jo u r n a lis t ic devi ce , we need to begi n to develop best pr ac ti ces an d be t h o u gh t le a de r s in t h is s pa ce , and not just technology user s. Q. What do you think is the biggest challenge with adapting journalism to VR? T h e r e a r e l o t s o f t e c h n ic a l c h a lle n ges to VR , both in ter ms of shooti ng i t, and th en d ist r i bu t i n g i t in a way t h a t ma ke s sense . I think that’s our fi r st l i ne , getting throu gh th o se t e c h n o lo g ic a l h u rdle s — fo r the w hol e i ndustr y, not j ust i n jour nal i sm. I al so th i n k t h er e a r e a lo t o f e t h ic a l c o n sider ati ons about a 360- degr ee vi ew of someo n e’s i nt i m a t e e x pe r ie n c e . Q. How do you think journalistic content will change as a result of VR? I t h i n k i t i s ch a n g in g —ju s t by t h e additi on of 360 or VR to the jour nal i sm m i x. I t i s i n c r e a si n g o ppo r t u n it ie s fo r c onsumer s to engage w ith new s and infor mati on c o n t en t . Q. Social media has resulted in news having unprecedented reach and momentum across its audience. Do you think that’s going to be sustainable in VR? T h e t ec h n o l og ic a l h u rdle s r ig h t n ow ar e going to keep this exper i ence nar r ow to a s m a l l su bset o f t h e po pu la t io n . I t h i nk w hat we see is the r ol e of the “anchor ” o r th e ro l e o f t h e “ re po r t e r” a s a g a t e ke eper dimini shing. I t appear s as if ther e’s no gu i d e o r g a t ekee pe r, bu t in f a c t t h e s h o o ter and pr oducer ar e doing j ust that but i n a l ess p hy si c a l w ay. Q. Is it less invasive this way? I’m n o t su r e it ’s le s s inv a s ive , be c a use i n a wei r d w ay it’s mor e i nvasive . What I m ean i s t h a t i n some way s t h e a bilit y t o tell that stor y i n such an immer sive w ay i s m or e i nv a si ve of you r pe r s o n a l s pa c e , bu t the r ol e of the guide , anchor, or r epor ter is n ot fron t a n d c e n t e r. Q. Digital media has given rise to individual people as brands that can be monetized. Do you think that could happen with VR even with the traditional “anchor” or “gatekeeper” removed? It i s h a p pe n in g wit h V R. Th e c o n t ent pr oducer has access to var i ous di str i buti on p la t for m s, a l o n g wit h t h e mo n e t izati on abi l i ty that comes w i th i t. Agai n, you’r e sti l l l i m i t e d by t h e t e c h n o lo g ic a l c o n s t r aints. B u t w h a t ’s c le a r is t h a t in t h is s pace , it’s not necessar y to have a bi g di str i buti on p la t for m b e h in d yo u like a n e ws n etwor k because it’s sti l l small enough and s m al l tea m d r i ve n . It ’s n o t in s t it u t io n a lly dr i ven yet.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

117

Q. Do you see VR becoming institutionally driven at some point? I c a n ’ t i m a g i ne it ’s n o t g o in g t o be . T he New Yor k T i mes is behind i t and F r ontlin e i s b e h i n d i t — t h a t ’s a lre a dy a s ig n o f it going that di r ecti on. Q. USC has been known traditionally as the hub of film education, and now with the introduction of VR we’re seeing an unprecedented merge between Hollywood and Silicon Valley. What are your thoughts on that? T h e r e wer e te n s io n s a n d t h e re s t ill ar e tensi ons. But i t’s been gr eat for Los Ang el es. I t h i n k L A i s e x pe r ie n c in g t h is s u r ge of ener gy combi ning cr eativi ty and tech, an d c r e a t i n g t h e s t re n g t h s o f me dia and the str engths of technology i n a w ay th at’s un i q u e t o t h is c o mmu n it y. Q. Emerging technologies often act as a catalyst for social change. Do you see any opportunities arising for women and minorities to make a name for themselves in this industry? In t h e o r y, a bs o lu t e ly. Ne w t e c h nol ogi es do offer oppor tuni ty for women an d m i n o r i t i es b e c a u s e t h e y a t t r a c t pe opl e w ho ar e intellectually cur i ous, w ho emb r ace th e a d ven t u r e o f e x plo r in g n e w t e r r i tor i es or new technologies! T hose tr aits ar e n ot l i m i t e d by g en de r, r a c e o r a ny o t h er di ffer ence . At t h e S ch oo l o f Jo u r n a lis m, o u r digi tal team, the pr ofessor s exper i menti ng w i th an d tea ch i n g e m e r g in g t e c h n o lo g ie s , is major i ty femal e and ver y di ver se . Women an d p e o pl e o f co lo r t e a c h c o din g , da t a jour nal i sm, soci al medi a and emer ging tech, l i ke V R. T h a t sa i d, even wit h No n ny de la Peña being one of the cr i tical thi nker s and d oer s i n t h e V R sp a c e , I do t h in k t h a t t he tech wor l d outside of academi a i s sti l l m al e d o m i n a t e d a nd n o t a s we lc o min g t o women as it shoul d be . Q. W h a t do yo u think ne e ds to happen for t hat rat io t o t ip? I t h i n k pe o ple a re min df u l o f dive r si ty and inclusi on i ssues. So i f a cluster eme r ges, w i t h a l i t t l e bit o f v is ibilit y, o ppor tunity fol l ow s. Back to education, this is w hy b r i n g i n g t h es e e me r g in g t e c h n o lo g ies into our cur r i cul um befor e they’r e ful ly b aked act u a l ly paves t h e way fo r g re a t er oppor tuni ty for women and people of c ol or, b e c a u se t h a t ’s re f le c t ive o f o u r s t u dent popul ati on. I n many w ays it’s the educat i on al p r oc e ss t h a t c re a t e s t h e o ppo r t u ni ty mor e so than the technological developm en t p r oc e ss, w h i c h s t ill t e n ds t o be in the mal e dominated pr ovince . T h e c u r r en t g e n e r a t io n o f s t u de n t s at U SC is ver y diver se in al l w ays — fr om gen d er an d or i en t a t io n t o e t h n ic it y a n d r ace . By empower ing them w ith the ability to u se an d d e pl oy t h e s e t o o ls fo r jo u r n a lism, but also by empower i ng them w i th a sense of fe a r l e ssn ess abo u t n e w t e c h n o lo g y, that cr eates a r eal oppor tuni ty for them and i n th e r e st o f t h e ir c a re e r s , wh e t h e r it’s VR or the next technology that comes our w ay. An d i f you ’r e bu ildin g a V R jo u r n a lism unit, ther e ar e few places that tr ain stud en ts b e t t e r t o do t h a t t h a n USC . contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

118

INTERVIEW: XING CHUNG, MINDMAZE (2016) 21 *INTERVIEW CONDUCTED BY SETH SHAPIRO

FEBRUARY 2016

X i n Chung, Found er, S ofa V R w as p r e v iously an executive at m ed tech V R com p any Mind m aze Accor d ing to a s p o kesm an for investor the H ind uja G r ou p, “M i nd Maze’s next-gener ation hum anma c hine inter face is alr ead y p r oving itself i n healthcar e , gam es, and its p otential for ap p lications in tr ansp or tation, d efense and var i ous typ es of m ed ia ar e now r ead y to b e tr uly exp lor ed .” P r ior to Mind m aze , Xin w as a Found er of V R U X.co, Tr ustCloud , B o t by, and V ykar ian, as well as Chief P r o d u ct O fficer at V inja. Q. You’re a classic Bay Area tech founder – what brought you to med tech VR? X i n : S o my ba c k g ro u n d a c t u a lly i s 3D ani mati on. I w as in visual effects and th e n 3 D for g a me s in a lo n g t ime . So in ter ms of w hat I ’m buildi ng, just so we have a l i t t l e mo re fo dde r t o play w ith her e i n the conver sation, my j ob is to bu i l d t h e m edia s u bs idia r y. We c all i t “medi a,” then it encompasses al l things from a ny so r t o f imme r s ive e x pe r i ence that you can have that can benefit fr om a n eu r a l c o mpo n e n t , wh ic h is wh at we’r e br i ngi ng—these neur al sensor s an d neu r a l si mu l a t io n s t h a t we c a n do to enhance . I t’s pr etty cutti ng edge i n ter m s o f w h a t we’r e wo r k in g o n , a n d br ingi ng emoti on and expr essi on to the medium i s a n o t h e r t h in g . I’m a h u g e f a n o f s o c ia l V R e x pe r iences. Maybe you’ve done the Toybox a fe w ti m e s w i t h Oc u lu s . If yo u ’ ve t r ie d t hat, just the sense of co- pr esence is somethi n g th a t h a s st ayed wit h me a n d dr ive n the pr oduct r oadmap di r ecti on that I ’ve bee n c h a m pi on i n g a t t h e c o mpa ny. So I defini tely w ant to be invol ved in thoughts an d c o nver sa t i on s o n h ow we c a n br in g i t w i th a ver y sol o exper i ence into somethi n g th a t i s c l oser t o a n o t h e r s lide da t a, a tr ue Metaver se avatar exper i ence . Q: It would be great to talk about sectors besides entertainment, because so much of the focus on VR has been around media. What are the implications for therapy and healthcare? X i n : M aybe I c a n ju s t de fin e a n a r c fo cu se d o n th e t re a t me n t o f s t ro ke use o f v i r t u a l a n d a u g me n t e d re a lity. tec h n i q u es, t h a t h ave n ’ t e vo lve d in a contributions@vrarproject.com

for w hat the exi sting body of wor k is. I t’s vi cti ms – str oke r ehabilitati on thr ough th e So they i nter weave pr etty tr adi tional r ehab l ong time , to i ncl ude vi r tual and augmented VR/AR Project

New Amsterdam Media

119

re a l i t y h a r dwa re t h a t is s o ld t o c lini cs, and doctor s, and hospitals to r ehab str oke vi c t i m s i n on e -t h ird o f t h e t ime , t hat had been done pr eviously. T h e y ’r e g oi n g f ro m t e n n is ba lls a n d wooden bow l s to VR HMDs that take advan tage o f t h e b r a i n ’s a bilit y t o h e a l it s e lf – via neur opl asti city – and doing it i n a w ay th a t ke e ps t h e pa t ie n t s e n g a g e d, it i n a w ay that i s much mor e measur abl e by th e c l i n i ci a n . T h es e a re t h in g s t h a t h ave been tr i ed in the past, but the technology w asn ’t th e r e t o g et t h e t y pe o f e n g a g e ment that we have now. S o we h ave a c o u ple pro du c t s t h at ar e out, and we’ve fi nished clini cal tr i als an d c e r t i fi ca t i on s in E u ro pe . A n d we ’ ve star ted wor king w ith fol ks like U CSF and Stan for d i n t h e S t a t es a s we be g in t h e FDA appr oval s over her e for thi s cl i nical har dw ar e , th i s m e d t ec h h a r dwa re . Even t u a l ly t ha t will move in t o over w hat we w ant to call “consumer wel l ness ” for th e m a sses, as o ppo s e d t o t h e c ur r ent system, w hich i s for cl i nici ans and st r oke vi c t i m s. I t ’s pre t t y n a r row in wh at i t does. And we’ve been looki ng at how th e we a r a bl e sp a c e h a s t a ke n a n d re imagi ned di ffer ent consumer- fr i endly technolo gi es i n t h e m e di ca l s pa c e , like t r a n s c r anial di r ect sti mul ati on. A couple companies th at ar e do i n g t h at a re c u r. me , a n d Thy nc .com. T h o se a r e ex a mple s o f n e u r a l t e c hnologies that have found their w ay, i nter weavi n g w i t h m obi l e ph o n e s t o be c o me c o nsumer technol ogi es. S o my ch a l l e n g e is t o s e e h ow t h e cor e tech that we’ve done for str oke r ehab can m ove i n t o t h e me dia s pa c e . A n d it ’s not just medi a in ter ms of soci al VR , but al so i n ter m s o f wel l n e s s , a n d f u r t h e r o n d ow n the r oad, we’r e l ooking at how we can m ake , how we c a n u s e s o me o f o u r t e c h to get past some of the chal l enges i n VR . O n e i s, fo r i n st a n ce , h ow lo n g yo u c a n wear a VR headset, how comfor table i t i s i n ter m s o f ex pe r i en ce s t h a t k ic k in t o g e t a r esponse . T hose ar e things that we’r e doing th e re sea r ch on . Q. C a n yo u ta lk a li ttle bi t a bo ut some of t he broad t herapeut ic applic at ion s or are a s i n w h i ch thi s is go i ng to ma ke an impac t ? X i n : Ou t si de o f s t ro ke , t h e re ’s a ny number of combinati ons of w her e we’r e usi n g s t i mu l a t i o n , d if fe re n t t y pe s o f n e u r al stimul ati on— non-invasi ve , of cour se , becau se th o se t h i n g s t h a t we ’re wo r k in g on r i ght now ar e non-invasi ve techni ques on th e s c a l p, o n t h e fo re h e a d, o n t h e f a c e . And those si mul ati ons have been applied t o, as yo u m e n t i on ed e a r lie r, a re a s in c lu d i ng phobias, depr ession, differ ent psycholo gi cal s t a t es t h a t ca n be me a s u re d, a n d then, using a combinati on of VR and sti mulati n g ha r dw a r e t o be a ble t o a f fe c t t h a t . Q: I s t h ere a pa r ticula r the rape utic use t hat st ands out as t ransformat ional? X i n : Ye a h , I wo u ld s ay t h a t t h e s t r oke r ehab, the existi ng one cal l ed My Emo ti on , th a t ’s o n e t h a t is o u t t h e re . In bro a d str okes, ther e’s a cor r el ati on of j ust how deep ly p r esen ce ch an g e s t h e t re a t me n t s that ar e out ther e today. T he tr eatments that ar e contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

120

o u t t h er e t o day, t h e y ’re o u t o f c o n text: they r equi r e a lot of effor t fr om the pati en t. T h e t r ea t m e n t s t h a t we ’re br in g in g to mar ket ar e able to keep the pati ents engaged , an d a r e a bl e t o c re a t e , a s mu c h a s the state of the ar t of that i s today, i s abl e to m a ke t h a t wo r k be t t e r. Th e e f fe c t is an actual, physical change . S t r oke r e h a b is a re a lly big ma r ket, and i t’s pr oven to r educe the tr eatment ti m e . An d a l so, pe o ple o n c e t h o u g h t t h a t ther e w as a w i ndow of w hich you could do th at re h a b. B u t a s it t u r n s o u t , t h e t y pe of tr eatment that we br ought to mar ket, i t can b e do n e m a ny mo n t h s o r e ve n ye ar s after the i niti al str oke . Q: So from your perspective as a serial what interests you most about this space? X i n : I h ave t o g o t o g o t o s c ie n c e fiction ther e because you menti oned R eady P l ayer On e w h en we we re t a lk in g e a r lie r. And as much as R eady Pl ayer O ne w as a b ou t goi n g a n d sh o o t in g s pa c e s h ips , a nd casting spel l s or w hatever, i t w as also ab ou t get t i n g a n emo t io n a l c o n n e c t io n , a soci al connection w ith other s i n VR in a w ay th at o n e wou l dn ’ t in t h e de f a u lt wo r ld. T her e’s the al t wor ld and the defaul t wor ld , for e x a m pl e . W h a t e x c i t e s me a bo u t t h is e n t ire thing is that it’s bl endi ng two: the most exci ti n g type s o f t e c h n o lo g y t o g e t h e r, a n d tr ying to get to a place that you see a b r ai n c o m p u t e r i n te rf a c e t h a t is s o v is c e r al, that is so l ow -latency, and that is so natur al – th a t ’s w h a t i s mo s t e x c it in g fo r me . And tr acking as much EMG data, EEG data, E O G (o c u l o g r a phy), e le c t ro o c u lo g r a phy w hi ch allow s thi s to tr ack your eyes si mi l ar to w h a t t h e ph on e do e s . D oi n g t h a t a n d t h e f a c t t h a t , fo r the time bei ng, you have thi s piece of plastic on yo u r f a c e a n d a s t r a p g o in g a ro u n d your head, until thi ngs get a l ot smal l er and a l ot m o r e f a n t a st i c – bu t t h e s e a llow u s to getti ng the sor t of data that we couldn’ t get b e fo r e , t o d r ive n e u r a l fo rwa rd. B ecause I w as actually ar ound i n ‘07- ’08, I wo r ked w i t h a co m pa ny c a lle d E mo t iv. I did a bunch of their game demos. T hat w as an E E G hea d set t h a t c a me o u t s ix , s e ve n year s ago for the fi r st ti me . And the pr obl em w i th th a t i s t h a t i t wa s h a rd t o c re a t e a conditi on w her e peopl e would have this thi n g on th e i r h e a d . A n d yo u wa n t t o do it fr equently and have a r eally clear r eason to d o i t. An d w i t h V R c o min g o u t o f t h e gate thi s time ar ound, super exciti ng, bec au se now, p e o pl e h ave it . Th e y h ave it on their head, and al l we need to do i s em b ed th e r i g h t sen s o r s re c o rdin g t h e r ight signal , and then dr opping them into di ffer en t e x p e r i en ce s. I de fin it e ly a m in t h e camp that r egar di ng 360 vi deo, that yes, i t’s n i ce an d i t ’s r e l a t ive ly e a s y t o c re a t e , but i t doesn’t r eally have the depth. And I ’ve b een ha r d- p r e sse d in t h e c u r re n t ba t c h of 360 vi deo exper i ences to feel pr esent. S o I t h i n k t h e pre miu m e x pe r ie n c e , fr om a mar ket segment point of vi ew, the b est an a l o g t h a t I c a n t h in k a bo u t is t h e har dcor e gamer, r i ght? Which is not so di ffer en t

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

121

from you r g un s t e r s r u n n in g a ro u nd in R eady Pl ayer O ne w ith thei r di ffer ent r i gs. Ever ybo dy ’s g o t t h e ba s e r ig t h a t p eopl e ar e tr i cki ng out their r i g to do the m or e awe som e t h i ng s . A n d it ’s k in d o f awesome that some of those things ar e happen i n g now, a n d I g et t o wo r k o n t h e m. Q: Are there specific projects that you’re a big fan of? Or on the other side, is there stuff out there that you think muddies the water, that for example, people watch enough 360 videos and then say “So what? This is 3D hype all over again.” X i n : Yo u ’r e e x a c t ly r ig h t . Ok ay, 36 0 video on a car dboar d, the bar to cr eate i t i s l ow. T h e b a r t o s h ip a millio n u n it s out this ear ly is l ow. So they went out and d i d i t, r i g h t ? L o o k in g a t t h a t a n e c do t e of w hat’s happeni ng r i ght now. What New Yor k T i m es di d a n d wh a t V r s e is do in g r ight now, I thi nk i t’s a gr eat fi r st step, but it h asn ’t b e e n t r a n sfor ma t ive fo r me . It may be tr ansfor mati ve for some of the mi l l i on peop l e th a t g ot t h i s s t r a n g e c a rdbo a rd t h ing i n the mai l . S o I t h i n k t h e re will be a pla c e fo r it. But for me , i t is the hi gher end exper i ence an d th e o n es t h a t h ave re a lly wo r ke d for me , if I can say w hen my bi g sea changes wer e i n V R a s a ba c ke r o f D K 1, fir s t wa s just the basic head tr acking, l i ke “Wow, thi s h ead tr a c k i n g su ck s ,” s o t h a t wa s t h e fir st one . T he second one w as actual ly getting han d s i nt o t h e si t u a t io n . So pe o ple s t a r ting put Leap Moti on i n the fr ont of their DK 2 s, s o I d i d t h a t , playe d wit h a de mo called “Wei ght Lift,” that w as done by some gu ys th a t wer e i n t e re s t e d in wh a t t h e Leap coul d do for VR . And then most r ecently, th e ha n d s d ow n at Oc u lu s C o n n e c t in O ctober, I thi nk I did Toybox seven times, becau se i t fi n a l ly c l i c ke d fo r me . T h e soc i a l V R po r t io n o f Toy box , t he physi cal i ty of it, just the sheer pr esence of i t w a s g r e a t . E ve n t h o u g h yo u c o u ldn’t see the per son’s face , i t w asn’t even a prop er av a t a r, i t w a s ju s t s o r t o f a t r a n slucent head w i th di sembodied hands w as m or e p r esen t t h a n wit h a ny t o p o f t h e line Ci sco, tel econfer ence system, or Skype sy stem th a t I ’ ve ever e x pe r ie n c e d, h a n ds dow n. T hat ther e w as ver y tr ansfor mati onal for me . W h a t we ’r e do in g in t e r ms o f be in g abl e to cr eate si mulations, w hether i t be hap ti cs, d e t ec t t h i n g s, wh e t h e r it be f a c ia l expr essions, to empower that, I thi nk that, at l east fo r m e , i s so me t h in g t h a t ’s g o in g t o move thi s thing fr om bei ng ni ce to have i nto th e nee d t o h ave t e r r it o r y be c a u s e it is so power ful . Q. What is the trajectory for social VR? How can VR become more communal? X i n : You ’ ve pro ba bly s e e n t h e a r t icl es and know the names. T he Phi l R osedal es of th e wor l d a n d H ig h Fide lit y, t h e A lt spaces of the wor l d, and then, of cour se , el ep h an t i n t h e r o o m , Fa c e bo o k , r ig h t ? Th e re ar e people cr eating the pl umbi ng under neath . G o o g l e a n d A n dre e s s e n pu t $25 m illion into a company called I mpr obable .io ou t o f t h e UK , w h ic h is s imila r t o High F i delity w ants to cr eate the plumbing for th e M e t aver se . I t h in k o n e ph r a s e is “Second Life getti ng a second act,” r i ght? T hat’s on e w ay t o l o o k a t it , a n d t h a t ’s fin e . Second Life didn’t ever fully ki ck. I t’s still ar ou n d ,

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

122

i t’s st i l l m a k i n g mo n e y, it ’s s t ill a c oupl e hundr ed empl oyees and it’s doi ng its t h i n g. B u t i t ’s n ot Min e c r a f t , r ig h t ? It ’s n ot F acebook. B u t w h e n you be g in t o br in g in the types of engagements and begin to l oo k at w h er e t h e m o n e y is a n d wh a t t h e behavi or s ar e of ear ly adopter s ar e , the segm en t th a t i s w i l l i n g t o do t h is a n d s pe nd their allow ances or Chr istmas w i shes on th i s s t u f f . L o o k i n g a t Twit c h fo r in s t a nce , r i ght? So spectator e- spor t, w hether t h ose s e ei n g a g a m e is a s e x c it in g o r a s sati sfyi ng as havi ng stadi um seati ng of a game , l i ke “L e a g u e of L e g e n ds ,” o r a fir s t -pe r son shooter. I t’s sti l l to be seen w hether or n ot th e v i sc e r a l ex pe r ie n c e t h a t o n e has of bei ng cour tsi de w i th a buddy, that’s p r etty i m p o r t a n t . I f yo u ’re g o in g t o be c our tsi de at a War r i or s game , you kind of w an t to b e w i t h a bu ddy. I g u e s s s o me pe ople w ant to be al one , but ther e’s a l ot of h i gh fi v i n g a n d h oo t s a n d h o lle r in g h a ppening i n that w ay, soci ally. S o i f you l o o k a t wh a t Twit c h is , h ow fast i t’s gr ow n, that type of e- spor t spectator, yo u m a p a so c ia l V R da t a , a n immer sive , social exper i ence over that, that’s p r etty e x ci t i n g t o m e a s a n a n a lo g . A n d t hen as you mentioned ear l i er, al l this stuff ab ou t na r r a t i ve s, a nd a g e n c y, a n d h ow t h at’s going to pl ay out, that’s awesome , and I can ’t w a i t . I ’ m n ot re a lly s pe a rh e a din g on that. I ’m l etti ng my fr i ends l i ke Nevi l l e Sp i ter i [of W E V R] se e wh a t t h e y c a n s e e i n that ar ea. Q. Do you think this is the year that VR breaks wide open? X i n : A s som e bo dy t h a t wa s a ro u n d for VR 1.0, Vir tual Boy, Dactyl Ni ghtmar e , all th at s t u f f , t h e l a st t ime a ro u n d a n d s e e i t sputter out, the best cr ystal bal l that I have i s th a t you r h a rdc o re g a me r s s ay, I’ ve hear d the phr ase , “I w ant so badly for this wor k, I’m w i l l i n g t o t h row u p.” I h e a r t h at over and over. And I myself w anted so bad ly to b e a t “ H a l f L ife 2” in V R t h a t I ma de mysel f nauseous. I t took me an hour to r ecover. I w a sn ’ t t h r owin g u p fo r a n h o u r, but I w as nauseous, r i ght? I took the headset of f an d I j u st we n t a bo u t my e ve n in g . I w as l i ke , “Goddamn, I feel shitty.” T hat’s h ow b a dly I w a n t ed it t o wo r k . S o t h a t be i n g s a id, I t h in k yo u ’re g oi ng to get pl enty of har dcor e gamer s pick u p a hi g h - en d r i g t h is ye a r. I wa s a h u g e fan of Gear VR . I thi nk they’ve got a ni ce l ead r i g h t n ow. I ’ m g o in g t o c a ll it t h e “Big F our,” to include Googl e i n ther e , PlayStati on , as t h ey d o, will c re a t e t h e ir own e cosystem and do the w alled gar den thi ng, O cu l u s, not t h e l e a st o f wh ic h is t h e y ’ ve g ot a gr eat team and beauti ful har dw ar e , and we’r e get t i n g a fi r st ve r s io n in a mo n t h , possibly, w i l l be w hat my eyes ar e on for the h i gh e n d . I h ave som e c o n c e r n s a bo u t HTC . I finally spent a l ot of time w i th a Vi ve i n a q u i et ro o m , t h e ve r y la t e s t V ive in a qu iet r oom. Lighthouse is noisy. T hat’s not some th i n g yo u w a n t t o h ave r u n n in g fo r a ny a mount of time in your livi ng r oom. I t’s l i ke a real ly l ou d a i r c o n d it io n e r. I’ m like , “Ok ay, that’s kind of a bummer.” An d t h e n G oo g le , o f c o u r s e , it le aked out that they’r e wor king on an HMD th at d o e sn ’ t r e qu i re a mo bile ph o n e o r a PC . So I assume that they’r e j ust going to take an d j u st e m be d s o me s o r t o f Ne x us model i nto an HMD.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

123

I g u e ss my fi na l t h in g is I s a id I’ m bullish on the Gear VR . But you’ve also hear d th e ad a g e t h a t pe o ple a re g o in g t o be thr ow i ng thei r Samsung phones i n the fr eezer b e fo r e t h e y g o in , r ig h t ? So I t h in k they’r e acceptabl e gr ow th pai ns. I s it going to b e ex pl osi ve t h is ye a r a n d t h is C h r i stmas enough to have a foothol d that it’s goi n g to br e a k l oo s e ? Bu t is it g o in g t o go i Phone 2009? I s i t going to go iPad 2012? Less l i kely. We ’r e n o t g o in g t o s e e t h a t sor t of expl osive gr ow th. T her e’s just not en ou gh th e r e . B u t I do n ’ t t h in k t h e fo o t h o ld is g o i ng to br eak fr ee . And I thi nk gamer s fir st, w hich i s ab sol u t ely fi ne , a n d t h e n it will move i nto the exper i ence , the immer si ve exper i en ce . A a f t e r t h e g a me r s a re do n e do in g thei r stuff, and Mom comes over, si ster com es over, w h a t h a ppe n s t h e n ? A n d t h a t’s w hat I ’m most exci ted about. And as I s ai d , I th i n k a l ot of t h a t is o n t h e s o c ia l si de .

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

124

INTERVIEW: CHRIS EDWARDS, THIRD FLOOR (2017) 22 *INTERVIEW CONDUCTED BY SETH SHAPIRO

APRIL 2017

Chr i s Ed w ar d s is Found er and CEO of T he T h i r d Floor V isualiz ation S tud io. T he T hir d F l o o r is the wor ld ’s lead ing visualiz ation s tud i o, contr ibuting to the m ajor ity of ma j or stud io-b ased enter tainm ent p r ojects thr o ughout the wor ld — using the vir tual wo r l d to im agine , p lan and p r oblem solve fo r stor ies in film , television, gam es, V R and the m ed attr actions. Background I grew up as a fan of classic Hollywood cinema. When I was young, my parents took me to a lot of film festivals, where I saw some of the best cinema that had been created to that time. And I was really inspired by Roger Eber t live at a film festival in Virginia. He showed me the power of cinematography and film language through the eyes of Gregg Toland and Orson Welles and Citizen Kane. He screened the laser disc of Citizen Kane— and pointed out over the course of three days, shot by shot, how the composition and other cinematic elements shaped the emotional impression on the audience. It was an epiphany for me: that there’s a lot more to cinema than what meets the eye. And so I went on a lifelong pursuit of this. Also, I wanted to combine it with the other side of my training, which is from my dad who’s an aerospace engineer, a rocket scientist. He trained me on how to use computers, and early forms of computer graphics. So I knew that there had to be a career out there that would combine computer graphics and classic cinema. But at the time it didn’t exist. So I went to film school with a dream to someday work with George Lucas and Steven Spielberg. I went to film school at the Rochester Institute of Technology, which is the home of Kodak. I learned about making films there, and about animation, but none of that had anything to do with computer graphics. Computer graphics was my hobby. Origins of Previsualization Ever y summer I would go to different internships to learn what was then the cutting edge: Silicon Graphics (SGI) hardware and software. That was my crash course in computer graphics, which became the basis of my student film, a 3-D, pseudo-Pixar shor t film that became my contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

125

calling card. The film caught the eye of Disney Feature Animation, and got me a job on a 3D animation project, Disney’s Dinosaur. Dinosaur was a breakthrough project at the time. It had live-action backgrounds in exotic islands, including Hawaii, to which computer-generated dinosaurs were added. But in order to plan where the dinosaurs would go and how wide a cer tain shot should be, and how the camera should be moving to cover the action of the theoretical dinosaurs, we needed to figure out what to shoot. So that was the genesis of previsualization for me. I went on to become a lead layout ar tist and animator at Disney Feature Animation. Over five years at Disney, I worked my way up the ranks and became a lead 3D layout ar tist. And I used that experience to get my next job and pursue my next dream, which was to apply what I’d learned to live-action filmmaking. The Third Floor I knew that computer graphics would give filmmakers the ability to design shot by shot, sequence by sequence, in some cases, across entire films. It would give them a huge advantage. But with visual effects, one needs to work in layers, and to plan in advance. And that became the origins of the modern previs industr y. For me, that began 15 years ago, when I joined Lucasfilm, on the third floor of George Lucas’ Skywalker Ranch house. We had a small team brought together to suppor t George Lucas on Star Wars: Episode III: Revenge of the Sith, and later Steven Spielberg on War of the Worlds. This small team forged a more sophisticated process than anything before, designing ever y shot for Star Wars, many times over—approximately 70,000 shot iterations by the time we were done. It was the digital blueprint for ever ything in the movie. And after our two-year tour of duty at the Ranch, I led a majority of the folks there to join forces, and create our own company, called The Third Floor, Inc. We went from the San Francisco area to Los Angeles to set up shop and cultivate a presence in the film industr y. I worked a second job at Sony Imageworks, on animated features, to fund our operations. In the evenings, I was a previs super visor at The Third Floor, where my co-founders and I began to establish our reputation for the highest quality previs. And about thir teen years later, I’m proud to say that the process of previs is almost universal. It’s been adopted by most of Hollywood, including almost all large-budget, effects-driven features. Teams at The Third Floor now touch over 75% of Hollywood blockbusters each year, including many of Lucasfilm’s projects and almost ever y Mar vel production. We have contributed to the past four seasons of Game of Thrones and major theme park attractions, like The contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

126

Wizarding World of Harr y Potter Gringott’s Ride, Fast and the Furious Supercharged and Despicable Me: Minion Mayhem. And coming full circle, we’re so proud to work on many projects within the Disney universe. What’s so exciting now is the proliferation of pre-vis tools that are increasingly real-time. So we can iterate with the creators and designers on the fly, and change literally any aspect of the vision, before it becomes the final visual in the movie, game or theme park. We are ver y excited about the promise of accelerated vir tual production techniques—literally the process of creating media of all kinds by allowing the filmmakers to step into their creations. And we can apply these techniques across film, television, themed enter tainment, games, and VR. And we’re increasingly getting involved, as a studio, in talking about the best ways to make transmedia launches for major film franchises. VR and Previsualization We initially had the thought to use VR as a tool for filmmakers in vir tual production. By vir tual production, I mean films like Avatar, Gravity, The Jungle Book, etc. These projects are largely digital, and require a live collaborative effor t using computer graphics on stage or on set. Then as VR headsets became more advanced, it became possible to put directors and production designers inside the sets being created—so that they can feel the world all around them, and change anything. We star ted using that on Miss Peregrine’s Home for Peculiar Children with Tim Bur ton, and several other projects since. And then there’s the second big promise of VR, which is really the bigger surprise for us, which was that VR itself was adopted so widely by technology companies, distributors and digital network providers. Since we’re already working on motion pictures creating previs assets that are essentially video game assets—we thought, “what if we can use those assets, repurpose them, to create bespoke VR experiences, that are additive to the film? At the ver y least, they can be used as adver tising. Or they can be a completely transformative extension for audience engagement beyond the theater. Location-Based Entertainment We’re excited that this dovetails nicely with what theater owners are attempting to do: upgrades to multiplexes, fancier seats, better food, more of an enter tainment destination. This trend can also incorporate custom VR experiences in theaters, as another option for motivated moviegoers who want to have a deeper engagement with the movie. The consumer adoption of VR technology will have a steady escalation. It’s going to take some time for consumers to be willing to buy the right computers and headsets, and to have the exper tise to set them up at home. So we are big believers in location-based experiences, experiences at Family Enter tainment Centers (FECs), malls and theaters. These locations are the best places to experience truly high-end VR experiences. contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

127

And we think it’s impor tant to make sure that early adopter VR audiences get the best quality experiences. The best way to get them excited in the shor t term is with these premium experiences, which will in turn increase the chances of them buying their own VR equipment in the coming years. The Third Floor and Semi-Interactive VR We’re privileged to have worked on notable recent productions like The Mar tian VR Experience, which was a collaboration between The Third Floor, the Vir tual Reality Company, and Fox Innovation Lab. It’s based on the amazing film The Mar tian, directed by Ridley Scott, and allowed us to create an experience that was par t-movie, par t-game. It takes audiences on a journey through some of the key moments in the movie, in which you play the par t of the lead actor, Matt Damon. We were able to publish the title on all three major platforms: Oculus Rift, HTC Vive and PlayStation VR and worked hard to make the interactivity ver y similar for all three vastly different platforms. And now we’re forwarding that experience into projects from stor ytelling episodes, all the way to theme park attractions requiring VR implementation. It’s just an exciting time to be involved. And we’re excited also to be working on the content side, because it’s often a matter of star ting early enough to create truly meaningful cinematic VR experiences. VR, Cinema and Games VR is an impor tant step in the histor y of cinema. It’s really leading a lot of directors and stor ytellers to rethink the medium that they’ve known for so long. And there’s an excitement, sor t of an arms race, to determine what are going to be the most popular genres and techniques in VR. I think we’re going to look back at these first five years of this stage of VR exploration and be stunned at the number of advancements: from the development of hardware and software, to the great work of creators, to the new language that’s being developed. So we’re excited to be par t of that wave. In some ways, VR is the product of the games industr y and the film industr y… and we’re right in the middle of the baby that they’ve made: VR, and soon AR. If you look at the film side of that spectrum, it’s what we call a classic narrative: it’s the same movie ever y single time. Then on the game side of the spectrum, you have a large audience of younger gamers. They want a completely open world of multiplayer experience, where they immerse themselves in an environment, and come up with their own stor y. We believe the largest audience for VR is right between those two areas: an audience that wants to go beyond the classic passive film—but are not going to the other extreme. They’re

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

128

not going to want to create a personality, set up an avatar, gather their friends, etc. It’s too much work. Right in the middle is what we call the semi-interactive narrative VR experience. It’s still based on a stor y, and there’s something hypnotic and engaging about it, but at times you are invited to par ticipate in and influence the stor y. This is also referred to as “branching narrative”: the “choose your own adventure” model. We’re excited to use our skill sets to help directors understand the potential of this new medium, and how to adapt their stories to utilize this medium, where you can invite the audience to become a par t of the stor y. This is cer tainly a challenge, and something that requires technical proficiencies many directors, scriptwriters and producers do not have. So our business plan for The Third Floor is to create an engagement with creators. We provide a sandbox for creators to iterate on their ideas, to use available technology to create interactivity without too much of a focus on the technical challenges. For VR experiences, we use a similar process to the one we use in film. It star ts with a brainstorming session: about how each sequence can be made more emotional, more effective. We’re not thinking too seriously at this point about technical limitations or even budget. We’re tr ying not to limit the idea, or to shoehorn it into a box. As the production continues, we get more serious about how we’re going to execute, so that it can be achievable and done on budget and in time. Augmented Reality I think that the promise of VR is going to be fur thered by AR. There are several dark horse companies brewing up new hardware that will be even more widely appealing to consumers because it will be based around practical, ever yday applications. And that is what AR is going to allow. It’s going to make our daily lives easier. Over time, we think that VR will be an extension of the AR toolset, so that you can go from the VR mode from your AR device and experience a form of enter tainment when you’re in the mood to do so, as we do with our phones when we’re traveling. This has already begun, but we think that it will be accelerated quite aggressively over the next several holiday seasons. The number of ver ticals that can be helped by this technology is enormous. Technology and the next Generation of Storytelling There are a lot of inspired creators that can really benefit from our exper tise in par ticular and from the previs process in general. My hope is that we can get this message out: that visualization is no longer cost-prohibitive for independent films or smaller projects. In fact, smaller productions sometimes benefit the most. They need to save money, to be ver y strategic about where they spend their resources. So previs can help these creators as much contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

129

or more than large-budget feature films. I feel like new creators should work across the landscape to VR, too. I think many people think the learning cur ve for VR and is quite high, or that the tools are out of their price range. But I beg to differ. We’ve walked through some of the early challenges. We’ve learned enough to be able to do surgical experiences and really cost-effective VR stor ytelling, that can be really affordable. It’s such an exciting time to be a creator. You have the ability to take a dream and place it in visuals affordably. You can get it to an audience through the Internet. Little guys have a much bigger chance than they used to. There are social networks and online distribution now, and all this is playing into the future of media in general. Lastly, I would say to anyone entering the industr y now: don’t be daunted by change, be excited by change. See it as an oppor tunity: to get ahead of the cur ve. We do, cer tainly.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

130

INTERVIEW: ROBERT HERNANDEZ,

USC ANNENBERG SCHOOL FOR COMMUNICATIONS AND JOURNALISM (2017) 24 *INTERVIEW CONDUCTED BY PERISA BROWN

MARCH 2017

R o b er t H er nand ez is an Associate P r ofessor o f Pr ofessional P r actice at the U S C A nnenb er g S chool for Com munication an d J o ur nalism . H is p r im ar y focus is on the i n t e r section of infor m ative r ep or ting and s to r ytelling, aim ing to engage p eop le in a c o m munity setting w hile b r oad ening his r e a c h thr ough d iffer ent for m s of d istr ibution. At A nnenb er g, H er nand ez has b een a p ioneer i n tr aining the next gener ation of V R and A R content cr eator s. H e and his stud ents p r o duce V R exp er iences und er their b r and: Background T h e fi r st A R c la s s I t a u g h t wa s in 2 013, w hich w as AR J our nal i sm. I n total, I ’ve b een at A n n e n b e r g fo r s e ve n a n d a h a lf year s. I spent about thr ee year s of doi ng cor ec la sses, t h en I t a u g h t a c o din g c la ss, and then I taught thi s one . I had been going to A R c o n fe r e n c e s fo r a fe w ye a r s befor e I put the cl ass together. My special tie s ar e e m e r g i n g t e c h n o lo g ie s a n d dig it a l stor ytel l i ng for jour nal i sm, non- fi cti on stor ytel l i n g. S o w h a t we en de d u p do in g wa s a ugmenti ng the LA Publ i c Libr ar y. We wor ked w i th th e l i b r a r y t o a u g me n t t h e s pa c e and to tell stor i es fr om a differ ent per spec ti ve . Yo u c a n down lo a d t h e a pp o n iTu nes: AR chi ve LAPL. To d o A R you n e e d a ma r ke r, o r GPS, or faci al mapping, so we ended up u si n g p hy si c a l o bj ec t s a s o u r v is u a l ma r ker s. We used the Metai o platfor m for that. B a si ca l ly, t h e re ’s a t o rc h in a h a llway, and w hen you launch our exper i ence and p oi n t at t h e t or c h , it h a s bu t t o n s ove r la id on it because i n the 1980s ther e w as a fir e th er e th a t c h a n g e d t h e c it y a n d c h a n g e d the l i br ar y. So ther e’s a vi deo piece that we d i d an d a p h ot o g a lle r y o f t h e fire . Th er e’s a kids secti on as well, in the back ther e’s a p u ppe t ’s st a g e . Th e libr a r y pu t s o n a puppet show ever y Satur day at 2pm, but if you m i ss i t , you mis s it . So if yo u po in t the devi ce to the puppet show stage , i t over l ays a m a g i c w i n dow o f a pu ppe t s h ow that w as pr e-r ecor ded. T h e r e a r e a wh o le bu n c h o f dif fe rent w ays you can use this. We also had access to s p e c i a l co l l e c t io n s , s o we ma de 3D model s and cr eated these book mar ker s to s h ow th i n g s l i ke t h e o r ig in a l pr in t in g o f Ulysses, and my favor i te one , w hich i s Edgar A l l en Poe ’s T h e R ave n illu s t r a t e d in a g or geous book. So you look at the model , you can ta p o n i t , a n d a ph o t o g a lle r y po ps up so you can fl i p thr ough the pictur es. contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

131

Q: So your class is set up like a mentorship, where you train your students they go out and create projects? Ye s my c l a sse s a re k in d o f like a h ack-a- thon style class. Q: When did you venture into VR? S o I di d A R Jo u r n a lis m in 2013, a nd then I di d a class based on Google Gl ass an d we a r a bl es, a nd t h e n I did V R, wh ic h I ’ve been doi ng for the l ast two year s. T hi s year I’m c o m pl et i n g my fo u r t h s e me s t e r teachi ng VR . When Google Gl ass kind of nosed ived a l i t t l e bit , Oc u lu s wa s o n the r i se , and w hat got my inter est w as Sam su n g G e a r. I s aw o ppo r tu n it ie s t o c re a t e s t u f f, so I cr eated a VR cl ass two year s ago, and f r om th a t I w a s a ppro a c h e d by t h e n e ws or ganization Pr oPubl i ca. We went to Hou ston d u r i n g spr i n g bre a k a n d did a piece about the Houston shi p channel . We d i d a 3 6 0 - de g r ee pro du c t io n , wh e re we mer ged CG and 3D gr aphics. We al so launch ed an a p p w h i c h is o n iOS a n d A n droi d, called J OVR NALI SM. After that, we wor ked at t h e C en t e r fo r Inve s t ig a t ive Repor ti ng, R eveal New s, to tell stor i es about th e d r ou g h t . I n J anu a r y t h is ye a r, I t o o k a coupl e students to Washington D.C . to cover th e p r e si de n tia l in a u g u r a t io n a n d t he Women’s Mar ch. We wor ked for the New Yor k T i m es a n d N PR a n d pu blis h e d o u r ow n pi eces ther e . T hi s semester we went to th e S a l t on S ea . We wo r ke d wit h Th e Deser t Sun and we’r e i n the middle of pr oduc ti on fo r i m m e r si ve s t o r ie s . T h e c l a ss i s a h a c k -a -t h o n wh e re I br ing student fr om acr oss the uni ver si ty — o bv i o u sly, m o s t o f t h e m a re g o in g to be jour nal i sm students because that’s th e s c h o o l t h e c la s s is o f fe re d in — but we’ve had students fr om gaming, engineer i n g, c i n e m a , e t c . T h e v a s t ma jo r it y o f s t udents come i n w ithout any exper i ence pr odu ci n g c o n t en t fo r V R. Q: So you teach VR production from the ground up? Ye a h , bu t i t ’s n o t like I’ m g o in g t o hol d your hand and teach you “thi s is w hat we’r e goi n g t o do.” Yo u le a r n by do in g it . T he fr amewor k is usually a media par tner a n d a topi c , w h i ch in t h is c a s e wa s Th e D eser t Sun and the Sal ton Sea. So then T he Deser t S u n wo u l d c o me wit h t h e ir re po r ter s and tell us about the Sal ton Sea, stud en ts wo u l d l i st e n , a n d t h e n pit c h ide a s of how they can tell immer si ve stor i es about th e S a l t on S ea . On c e a n ide a is pic ke d , then the r est of the semester i s spent figu r i n g o u t h ow we ’r e g o in g t o t e ll t h a t s t or y. We d i d a v a r ie t y o f dif fe re n t e x pe r iences. We also did a stor y on Ebola. Ebola w as th e new s st or y a t t h e t ime — it wa s m oving fast and ther e w as a lot of mi si nfor mati on . S o we c r e a t e d a n e x pe r ie n c e ba s e d on that to better i nfor m people about the i ssu e .

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

132

T h a t ’s i n t e r e s t in g t h a t yo u we re a bl e to allevi ate some misinfor mati on by usi ng th i s tec h n o l og y. D o yo u t h in k t h a t s h ow ing a si tuati on thr ough a 360- degr ee view gi ves p e o pl e a n i n h e re n t s e n s e o f t r a n s par ency and authentici ty about an i ssue? It do e s a n d i t do e s n ’ t . Ju s t like a nything, I can put it i n such a w ay to manipu l ate th e sc e n e , so it ’s ju s t a dif fe re n t way of doing it. I t doesn’t solve a pr obl em. Som e p e o pl e t h i n k “Oh my g o s h , in C G you can make up anythi ng. Her e come mor e li ar s.” T h a t ’s w hy I wa n t jo u r n a lis t s a n d e thical stor yteller s to come in to play to tel l th ese s t or i es. Th a t ’s why we ’re wo r k in g on these thi ngs at Annenber g, to hel p defi ne w h at th e se t h i n g s a re . Anyt h i n g you c a n c o me u p wit h , you can mani pul ate , so i t’s not “either /or ” w i th VR an d t r a n spa r e n c y. Fo r jo u r n a lis m, it comes dow n to cr edibi l i ty, my tr ack r eco r d , i f I’ve e a r n e d yo u r t r u s t a n d if I c a n retain your tr ust. I t doesn’t matter w hat platfor m yo u ’r e on — yo u lie in t e x t , yo u can l i e in vi deo and you can lie i n CG. So th at d o e sn ’ t c h a n ge , bu t t h e re a re t h in gs to be mor e aw ar e of. T h e r e h ave b e e n s o me s t u die s t h a t indicate a new l evel of thi nki ng that jour n al i sts o f t e n d o n ’ t t h in k a bo u t , wh ic h is t he level of r esponsibi l i ty about how effective th i s i nfor m a t i on o r s t o r y c a n be re c e ived. Exper i enci ng cer tain stor i es in an i mmer si ve w ay br i n g s a n e w le ve l t o t h e ir imp act, and we need to figur e out that r esponsi b i l i ty o f w h en i s so me t h in g t o o t r a u ma t ic or w hen do we induce tr auma. We don’t w an t to shy aw ay fro m impo r t a n t t o pic s . Even i f somethi ng’s offensive , peopl e need to b e aw a r e of i t , s o t h e s e a re n e w t h in gs for us to tackle . T hat agai n is par t of the cl ass an d w h a t we do h e re . Q: Do you think there’s a fine line between tact and offensiveness in using VR for journalism? It’s j u st l i ke any t h in g . I c a n s h ow you a l i ve video of somethi ng that i s not tas tef u l , th a t i s of fe n sive . Th e re a re live s t r eams of hor r i bl e things. T hese ar e j ust tool s. I alw ay s g i ve t h e a n a lo g y o f a t e le ph one: you can judge the technology of a tel ep h on e by t h e c o nve r s a t io n t h a t a 12-ye a r ol d had on i t, or by a conver sation done by th e p r esi de n t du r in g wa r t ime . It ’s t h e same device but used w i th compl etely diffe r en t l evel s o f i m po r t a n c e . So in t h e e n d, it’s j ust a tool . It’s u p t o u s t o t h in k a bo u t t h a t . You coul d total ly do i t w r ong. You coul d u se j ou r n a l i sm t o c a u s e h a r m, bu t yo u can al so use j our nal i sm to tal k about some r eal ly tou g h i ssu es. Q: I’ve heard this comment from a lot of VR content creators: “If you can think of another way to put content in front of people’s faces, then you shouldn’t make it on VR.” Do you agree with that? S o m e w h a t . Yo u wa n t a t h re s h o ld to justify that you’r e doi ng 360 for the r i gh t re a so n s. E a r ly o n I wo u ld s ay t h a t you have to exper i ment. But ther e’s a differen ce b e t we e n ex pe r ime n t in g a n d pu blishing. You have to figur e out w hat makes a good s h ot . S o, e x pe r ime n t n ow t o fig u re out w hat a good stor y i s using this techno l ogy. T h e t e c h n o l og y will g e t be t t e r down the r oad, but you have to exper i ment now an d contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

133

kn ow t h a t i f it ’s n o t re a dy, t h e n don’t publ i sh it. At some poi nt after you’ve d on e th e se e x pe r ime n t s a n d fig u re d o u t w hat wor ks and w hat doesn’t, w hen someon e ap pr o a ch es yo u wit h a s t o r y ide a you can make somethi ng competently. Ide a l ly, you w a n t V R t o be e x c lu s ive to those ki nds of exper i ences, but it dep en d s o n w h a t you r me t r ic o f s u c c e s s is . I s i t how many people you r each? I s it how l on g th e y ’r e i n t h e e x pe r ie n c e ? D o yo u w ant them to shar e the exper i ence , or to th i n k th a t ’s t h e be s t e x pe r ie n c e t h e y ’ ve ever had? I t can be any of those thi ngs. Q: Having experimented with different methods of content creation in VR, have you come up with any best practices? A l o t of t h i n gs I’ ve le a r n e d a re h ow not to shoot in 360. With us, our output i s for m o bi l e d e v i c e s be c a u s e we wa n t to r each a lar ger audience , not j ust the audien ce th a t c a n a f ford a h e a d-mo u n t e d displ ay. So we make a ser i es of compr om i ses b e c a u se of t h a t , bu t in do in g s o, we mer ge 360- degr ee vi deos w ith tr adi ti on al p h o t os a n d v ide o s . We a ls o br in g in CG gr aphi cs and I think that takes advantage of th e i m m e r si ve s pa c e . Th a t a ls o g ives you mor e options to pl ay w ith as opposed to j ust h ol d i n g a s h o t in 360.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

134

INTERVIEW: AARON KOBLIN WITHIN (2016)

*INTERVIEW CONDUCTED BY SETH SHAPIRO

JANUARY 2016

A ar on Koblin is a p r olific d ata ar tist and t he c o - F ound er of Within ( for m er ly know n as V r s e) , w ith Chr is Milk. We fir st m et w hen he was Cr eative Dir ector of the Data Ar ts te am at G oogle’s Cr eative L ab, w her e he l ed a s e r ies of gr ound b r eaking p r ojects includi ng b r i l liant T he J ohnny Cash P r oject and Ar cade F i r e ’s T he Wild er ness Dow ntow n. Within i s a lead er in V ir tual R eality exp er iences, d e l i ver s the Best O ver all V R Exp er ience o n i P hone , and w as the lead content p ar tner fo r T h e N ew Yor k T im es’ land m ar k V R /G oog l e C ar d boar d launch.

Q: What was it about VR that got you to leave Google to found Vrse?

Aa r on : C h r i s a n d I h a d be e n t a lk in g for quite a w hi l e about the potential for VR , as m a ny h ave . T h e billio n -do lla r qu e s tions w as: “When i s ther e goi ng to be har d w ar e th a t pe o pl e c a n a c t u a lly g e t t h e ir hands on, that w i l l al l ow for cr eati ng the kind s of e x p e r i en ce s we ’ ve be e n do in g fo r a long time – but now distr i butable to a l ar ger au di en ce , i n th e re a l wo r ld? ” F or m e , i t w a s a bo u t t h e impe n ding devices on the hor i zon – and the fact tha t we we r e a c t u a l ly g o in g t o be a ble t o a t l east exper i ment w ith a l ot of things we’ve b een d r ea m i n g a b o u t fo r a lo n g t ime . Q: What do those things look like? Aa r on : C h r i s a n d I h ave be e n t h in k ing about how you use technology for cr eati ng ar t, to c o n n e c t d e e ply wit h pe o ple . Th e w ay we’ve done that in the past has often u sed c r ow dsou r ci ng t o g e t pe o ple t o wor k together : thr ough the I nter net, exper i men ti n g w i t h pe r son aliza t io n , wh e re we c a n tailor an exper i ence that’s speci fi cal ly goi n g to re son a t e w i t h s o me bo dy ’s me mo r ies and exper i ences. An d a l so, g e n e r a l in t e r a c t io n … t h inking about i nstallation and physical status – h ow th a t a f fec t s yo u ve r y dif fe re n t ly f rom seei ng somethi ng on the scr een. We’ve b oth d o n e i n st a l l a t io n s t h a t a re in mu s eums, l ar ge-scale things. We’ve played w i th thi n gs l i ke ba ss a n d de e p s o u n ds t h a t a c t ually affect your body, and l ar ge scul ptur es i n th e s k y t h a t you c a n in t e r a c t wit h . Th e pur e scal e of these has a deep impact.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

135

I t h i n k t h e o ppo r t u n it y o f V R is t o enable these ki nds of thi ngs, to be able to m i x th e m a n d c o mbin e t h e m. It ’s a ls o t o allow peopl e to exper i ence them i n the com for t o f t h e i r h om e s , wh ic h is s o me t h in g we have never exper i enced i n the past. We can have t h a t sen s e o f pre s e n c e , a n d actual ly being ther e . You can al so have tai l or ed e x p e r i en ce s, a n d in t e r a c t iv it y, a n d connecti vity w i th other s… all these things w h er e yo u ’r e ba si ca lly h a c k in g yo u r s e n s e s w ith VR – convincing you that you’r e somewh er e yo u ’r e n ot . An d t h a t some wh e re t h a t yo u ’re not can actually be somepl ace that doesn’t exi st, an d c o u l dn ’ t e x is t . Yo u c a n ma n ife s t anything that you can dr eam up. S o t h a t ’s ba sic a lly t h e c o n c e pt . This is an amazing oppor tunity to begin to for ge w h a t t h i s m ediu m me a n s , t o e s t a bli sh a few best pr acti ces, and a mi x of r ul es th at c a n b e b r o ken . Th is is pro ba bly a o nce- i n-a- l i feti me oppor tuni ty to help define w h at th i s m e di u m c a n be . I t h in k t h a t was w hat r eal ly thr ew me into it. Q: What makes this a once-in-a-lifetime moment? Is it the notion of defining immersion, or presence, or agency, as opposed to the more passive POV that we’re used to in media? Aa r on : Yea h , so I t h in k t h e re ’s a t e mptation to think the pr esence as a bi nar y thin g. A l ot o f pe o pl e a re s ay in g , like , e it h e r i t happens or i t doesn’t happen. And I under stan d th a t t em pt a t i o n , bu t I a c t u a lly t h in k pr esence is mor e of a gr adi ent. When I thi n k of p r esen ce , t h ere ’s t h e phy s io lo g ic a l ver sion that i s basi cal ly: I tr uly believe that I ’m s o m ew h er e e ls e . A n d e ve n in V R n ow, the w ay i t cur r ently is, I don’t think anyb od y wo u l d co n sc io u s ly e v a lu a t e wh e re they ar e and decide that they ar e somew her e th at th e y ’r e n o t . Yo u s t ill k n ow yo u ’re in VR r i ght now. And I thi nk r eally w hat I ’m m or e i nt er e st e d i n is pre s e n c e a s it re la tes to w hat I think of as fl ow. S o i f you ’r e in a g re a t c o mpu t e r game , and i t’s r eally immer sive , you can actu al ly ach i e ve a ver y low le ve l o f pre s e nce thr ough that scr een. Basi cal ly, I get lost i n i t. I’m i m m e r se d in it a n d fo r g e t a bo ut ever ythi ng el se ar ound me , and I ’m having th i s am a z i n g e x pe r ie n c e . Wit h V R, it t a kes i t to the next level and ki nd of for ces you i n to th a t st a t e , or a t le a s t e n a ble s yo u to get to that state in a much quicker, easi er w ay. To m e , pr e sen c e is a bo u t t h o s e moments w her e you j ust for get that you’r e n ot act u a l ly i n t his pla c e . Yo u ’re re a lly in the moment, and you al l ow ever ythi ng arou n d yo u t o b e w ha t it is . I t h in k it ju s t b ecomes ver y easy w hen you have both visual an d s o u n d . I t h i n k we o f t e n fo r g e t wh at an i mpor tant r ole , a cr ucial r ole sound p l ays i n c o nv i n c i n g yo u t h a t yo u ’re s o mew her e , and having the sounds r eact r el ati ve to w h er e you ’r e lo o k in g , a n d h ave t h e dynami cs that you expect fr om the space th at yo u ’r e o c c u py in g . An d t h e n t h ere a re o t h e r t h in g s t hat ar e happening in the space , w her e peopl e ad d an a ddi t i on a l dime n s io n . A t s o me events, such as a fan or heat sour ce – some th i n g th a t p l ays i n to t h e e x pe r ie n c e a n d val i dates your subconscious expectations, r eal ly hel p s t a ke you t o t h e n e x t le ve l. It’s not an incr emental thing: i t’s an expone n ti al th i n g .

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

136

S o t h er e ’s a c t u a lly a g ro u p in Lo n don that al i gns these events w ith time code . T h ey act u a l ly cr e a te phy s ic a l s e n s a t io n s to align w i th their fi l ms – so if you’r e on a so ccer fi el d , t h e y h ave a h a n df u l o f g r a s s that they put under neath your nose , and i f th ey have a sc e n e wh e re yo u ’re in t h e k i tchen, and they have a heat sour ce that they tu r n o n w h e n you ’re n e a r t h e ove n . If you t a ke i t t o o f a r, like s o me t ime s happens i n Southeast Asia, w her e they someti m es have a sn a ke t h a t c o me s o u t u n d er your chai r, that’s cheesy and takes you ou t o f t h e e x pe r ie n c e . B u t a ny t h in g t hat doesn’t take you out of it and suppl em en ts an d r e i n for c e s yo u r e x pe c t a t io n s i s r eal ly effecti ve at taking you even fur ther i n to p r esen ce . Q: So an example would be the use of the fan and a faux wire in The Walk VR – to simulate what you’d feel like walking a tightrope between the World Trade Towers? Aa r on : Yes, e x a c t ly, wh e re yo u ju s t r eal ly feel like you’r e out on a cliff, because you have t h a t e x t r a s e n s o r y in pu t fo r thi s sensati on. Q: What’s the dance between the tech and the creative for you? This is chicken-andegg… but do you generally conceive of a vision for a project, and then find or create the technology to achieve it – or are there breakthroughs in technology that then inspire a creative leap? Aa r on : I t ’s de fin it e ly a n in t e r a c t ive pr ocess, so we’r e lear ning w hat’s possible an d th e n e x pe r i m e n t in g a n d bu ildin g on that. We’r e defi nitely a stor y fir st company so we t h i n k a b o u t , “Wh a t a re t h e a ma zi ng exper i ences and nar r ati ves that we can bu i l d up o n t o c r e a t e s o me t h in g t h a t will deeply connect w i th somebody?” S o t h e r e ’s k i n d o f t wo s c h o o ls o f thought i n VR r i ght now. O ne is w hat’s called th e re a l - t i m e r en de re d c a mp, wh e n yo u star t w ith i nter activi ty, w ith maximum fl exi b i l i ty an d r e sp o n si ve n e s s , a n d t h e n wo r k tow ar ds photor ealism. F or n ow, we ’r e a ppro a c h in g it g e n er ally fr om the photor ealism side , w her e we th i n k p e o pl e r ea l ly c o n n e c t wit h f a c e s and places that ar e ver y r eal to them. And th en we ’r e wor k i ng t owa rds t h e in t e r a cti vity side , and l ooking at things like br anch i n g na r r a t i ve s a n d o t h e r way s t o a dd i nter action. But, for us, the r eal power is r i gh t now on t h e s t o r y s ide , a n d o n t he connecti on si de , and on the ability to cr eate e x p e r i en ce s t h a t we s e e a s , h o pe f ully, ti mel ess r ather than ti mely – things that ar e goi n g t o h o l d u p de s pit e t h e f a c t t hat they’l l be technically embar r assing in the n ots o - di st a n t f u t u re . Th e c o n t e n t it s elf w i l l hopefully shi ne thr ough and still conn ect w i t h p e o pl e . Q: Tell us about Vrse and The New York Times Google cardboard project. Aa r on : Yea h , t h e Ne w Yo r k Time s Googl e Car dboar d gi veaw ay w as a huge mom en t fo r V R . I t h i n k it imme dia t e ly g o t a million peopl e into headsets and kind of he l p ed to c e m e n t i ts pla c e in c u lt u re . I thi nk that those br ands obvi ously have a lo t of att en t i on a n d a lo t o f s ig n ific a n c e in society, so I think them coming out and maki n g contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

137

th a t st a t e m e n t , a n d ma k in g t h a t gestur e , and maki ng that i nvestment w as r eal ly s i g n i fi ca n t . An d we wer e re a lly pro u d t o pa r t n er w i th them and wor k on the content w ith t h em . I C h r i s c a n sp e a k a lo t mo re t o t h e hi stor y of the r elationshi p, but wor king w i th J ake , Ed i t o r of N e w Yo r k Time s ma g a zin e , w as gr eat. I thi nk Chr i s and he clicked i nstan tly, an d h a d l ot o f ide a s fo r way s t h a t thi s technol ogy could be used for j our nal i sm . So we ’ ve p a r t n e re d t o g e t h e r o n a number of differ ent pi eces al r eady, al l of the fi r st p iec e s t h a t Th e Ne w Yo r k Time s wor ked on. Q: As you know, I’m a big fan of your work at Google, especially The Johnny Cash Project and The Wilderness Downtown. Was there a throughline between the stuff that you were doing with them into Vrse? Aa r on : A l o t . I t h in k in ma ny way s w hat we’ve been doi ng for the last decade i s to bu i l d u p a n d re la t e d t o s o me t h in g we’r e tr ying to do now. I t’s about usi ng pi xels an d m i ce — or pi xe ls a n d dig it a l in pu t d evices— to enabl e people to have an emot i on al c o n n ec t i o n wit h s o me t h in g a n d h ave an exper i ence that i s taki ng them somewh er e e lse . I t h i n k th a t ’s f u n c t io n a lly wh a t we ar e . I t’s, “How can we dr eam up spaces , an d p la ce s, a n d i d e a s a n d fin d a way t o get peopl e ther e i n a si gni fi cant w ay i n or der to c o n n e c t w i t h t h e m?” So I t h in k , s pecifically w i th those examples, you have the i d eas fro m W i l d e r n e s s D own t own o n a l ot of about i nter activi ty and per sonal i zati on an d how ca n you c re a t e a n e x pe r ie n c e for somebody. An d I t h i n k we h ave n ’ t re a lly s e e n that in VR too much yet, but I thi nk i t’s comi n g— e s pe c i a l ly a s we bu ild o n t e c h n o logies like ar tifici al intelligence , and we tap i n to o t h er t y pe s o f da t a ba s e s a n d re s p onsi veness. We’r e going to know things a b ou t how you ’r e i n t e r a c t in g . We ’ ll pro bably begi n to know thi ngs about w ho the pl ayer s ar e , w h o t h e u s e r s a re , a n d t h a t can be used dir ectly in the content i tself, I th i n k, i n m e a n i n g f u l way s . Simila r ly, wit h J ohnny Cash, I think that idea of communi ty an d c r ea t i on . We ’ ve a lre a dy s e e n t h e e xpl osive power of things like Mi necr aft, and i n VR , I t h i n k you ’r e g o in g t o be a ble t o s har e those spaces in even a mor e convi nci ng w ay, an d i t ’s g oi n g t o le a d t o a ll k in ds of cr eati ve tool s and shar ed cr eative capaci ti es, an d pe o pl e will ma ke a ma zin g t h in gs i nside VR . Q: I think everybody in this space had a moment where the light went on – where some experience made us think: “This is it. This is a quantum leap in the nature of media experiences.” But we’re obviously not at critical mass yet, and there are a lot of questions about what will get us there. What do you see are the things that are likely to make this space catch fire with the larger audience? Aa r o n : T h er e a re s e ve r a l way s t o answer that question. T her e’s the har dw ar e si d e w h i ch i s g o i n g t o c o n t inu e t o de ve lop, and i t w ill guide a lot of the w ay the industr y un fol d s. I t h i n k t h e mo bile s t u f f is going to continue advanci ng, and w ill likely be th e w ay t h a t m a j o r it y o f t h e pla n e t e x per i ences this kind of content. T h er e a r e a bu n c h o f big qu e s t io ns about w hich aspects get ther e fir st, l i ke if we c a n g et t o a po in t wh e re , le t ’s s ay, fi ber expl odes. O ne can i magi ne a wor ld w h er e contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

138

p e o pl e we a r V R de v ic e s t h a t lo o k mor e l i ke dumb ter mi nal s, and we could str eam d ow n c o n t e n t , low e n o u g h la t e n c ies that that coul d be game- changing. T he w ay th at fi b e r r ea l ly r u n s s o f a r, it may be a w hi l e befor e we can get l ow latency connecti on s to e n ou g h p e o ple , bu t I t h in k t h e mobile si de of it is pr obably going to adv an ce s i g n i fi ca n t ly in o t h e r way s a s we ll. Yo u ’r e g oi n g t o h ave t h in g s like G oogle Tango, w hich w i l l be built into the co m i n g m o bi l e ph on es , wh ic h will g ive s imilar functi onality to w hat’s in other headsets t od ay – bu t i n ph on e s . I t h in k yo u ’ ll a ls o get phones that have full spher i cal camer as bu i l t i nt o t h e m . Th a t wo n ’ t be t h e s a me as VR necessar i ly, or the stuff that we’r e tr yi n g to a spi r e t o. Bu t yo u ’ ll be a ble t o get i mmer si ve panor amas that w i l l have per s on al s i g n i fi ca n c e , be c a u s e yo u c a n re c o rd your ow n events, and your ow n family, and you r ow n f r i e n d s, a n d t h a t ’s a n a ma zin g exper i ence not many peopl e have had yet. It’s k i n d o f l ike lo o k in g a t V R fo o tage of peopl e’s ow n lives, fr om a thir d- pe r son p e r spe c t i ve . Th a t ’s s o me t h in g C h r i s and I have just been exper i enci ng for the fi r st ti m e , a n d I c a n t e ll yo u it ’s ve r y s t r ange to have that kind of out- of- body exper i en ce w i t h you r own h is t o r y. I t h in k t h a t ’s coming for a l ot of people . An d i f you l oo k a t wh a t ’s h a ppe n ing w ith those high-power headsets, gamer s ar e goi n g t o m a ke t h e inve s t me n t s t h a t ar e needed to r eal ly advance that next l evel of i m m er si on a n d in t e r a c t iv it y. So I t hink that’ll be l eadi ng the battl e cr y for sur e . Q: When we look at back in 10 years, what is 2016 going to be remembered as in VR? Aa r on : I t h i n k it ’s g o in g t o be t h e year of the launch. Q: Of HMDs, or generally? Aa r on : A l l o f t h is is c o min g o u t fo r the fir st time . Yeah, other HMDs ar e comi ng ou t fo r t h e fi r st t ime , a n d pe o ple a re going to take i t a l ot mor e ser i ously. But I sti l l th i n k i t ’s g o i ng t o be a lit t le wh ile b efor e we see tr ue kind of mai nstr eam adop ti on . It’s so h a r d t o s ay, be c a u s e we re a l ly haven’t had a moment like this thr oughou t th e c o u r se o f h u ma n h is t o r y. T h e r e h ave be e n me dia t y pe s la unched and huge technological advances, bu t i t fe e l s l i ke t h e re a re h u g e u n k n owns this ti me . T her e ar e many companies pu tti n g s i g n i fi ca n t a mo u n t o f mo n e y in , a nd all the eyebal l s ar e on these launches. So i t c o u l d b e t h e c a s e . We a lre a dy s e e an acceler ati on i n the pr ocess, and wor kfl ow, an d th i n k i n g , c o mple t e ly u n like a ny o t h er medi ums we’ve seen. Histor y of ci nema taki n g a h u n dr e d yea r s t o re a lly re a c h matur i ty, many ar gue , and we’r e ki nd of r un n i n g, s k i ppi n g , a n d ju mpin g t h ro u g h a lo t of the baseline limitati ons. B u t w i l l t h er e be a bit o f a g a p be t ween the i niti al l aunch and the sense for adop ti on ? I d o n ’ t k n ow. It ’s s t ill h a rd t o s ay. I t’s still goi ng to be expensi ve for your aver age c o n su m e r, so it ’s g o in g t o be ve r y much gaming off the bat. And then I think i t’s goi n g t o c o n t inu e t o be re fin e d a nd devel oped, and ther e’s going to be si gni fi can t

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

139

ad v a n ce s b e fo re fin e s s e ve r s io n s come out, and those w i l l pr obably be the on es b e t t e r, e ven m o re ma in s t re a m, I t h ink. B u t T B D. I t ’s a ls o t h a t h u g e qu e s t ion ar ound kind of the theater-style/theme-p ar ks t yl e ve r si on s o f t h e s e h e a ds e t s . So i t may be the w ay that people get access to th e m for so me t ime . A n d a ls o, o n the inter national side , I think, the mobile th i n g i s g oi n g t o b e h u g e , a n d it ’s po s s ibl e , I have no i dea if this w i l l happen, but I can i m a g i n e a sc e n a r io wh e re t h e re ’s something aki n to an I nter net café. Yo u m e n t i on e d t h is fe e lin g like t h e ear ly days of web br ow ser s befor e . Ever yb od y ke eps t h i n k i ng a bo u t it a s t h e a t e r s w her e peopl e go and put on 3D goggl es for VR . B u t i t m i g h t ac t u a lly e n d u p s o me t hi ng mor e like an I nter net café w her e you go an d have a c c e ss to V R g o g g le s fo r a n hour or w hatever it tur ns i nto, or you pay by th e hou r t o u se t h e m. Q. What do you think of the notion of location-based VR? Aa r on : I ’ m pe r s o n a lly ve r y in t e re s t ed in i t, I think. I t’s a ver y compel l i ng and enga gi n g e x p e r i en ce t o h ave t h e re a l-wo r ld aspects, w her e ther e actually i s a w al l ther e , wh er e yo u fee l l i ke t h e re s h o u ld be a wa ll ther e , and you’r e actual ly hol ding an object th at re sem bl es t h e o bje c t in v ir t u a l s pa ce . I do thi nk ther e’s huge potenti al for that, an d I w i sh t h a t I g r e w u p in a wo r ld wh e r e ther e’s that kind of enter tai nment –thi ngs l i ke th a t w i l l b e in c re dibly f u n . B u t , ye ah, I think ther e may also be a wor l d for physi cal o b j ec t s for t h e h o me , like g a min g obj ects that r esemble objects in the game . B u t th a t c o u l d a ls o a dd t h a t e x t r a le vel of r eal i ty to the physi cal - vir tual pai r i ng. We’l l s e e . I t h i n k , fi r st , wh a t pe o ple n e e d t o do i s like sol ve the chai r pr obl em. T her e’s g oi n g to be som e k in d o f de v ic e t h a t g e ts made that’s sl i ghtly better than a sw ivel ch ai r i n t er m s of we av in g , a n d mov in g , and potential ly tr ansfor mi ng i nto a bicycl e or m o t or c y c l e o r a h ove rbo a rd. Bu t t he natur al physical movement thing i s a bi g i ssu e , b e c a u se you ’re a lway s c o n s t r a in e d by the r eal i ty that you’r e in. I n vi r tual r eal i ty, you w a n t t o g o e ve r y wh e re , s o we n e ed to come up w ith a par adi gm and l anguag e to al l ow p e o pl e t o t r ave r s e la r g e s paces and then have l ocal explor ati ons al ong w i th th a t . S o I t h i nk t h e re ’s g o in g t o be some i nter esting chair s and per i pher als that c om e o u t o f t h i s wo r ld.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

140

INTERVIEW: YELENA RACHISTSKY, OCULUS (2017) *INTERVIEW CONDUCTED BY SETH SHAPIRO

25 APRIL 2017

Ye l ena R achitsky is a Cr eative P r od ucer at Oculus, p ushing for w ar d the effor ts on c r e ative outr each initiatives and find ing an d fund ing Exp er iences for the O culus p latfor m. P r i or to O culus, she w as the Cr eative P r o ducer of the annual Futur e of S tor yTel l i ng ( F o ST ) sum m it, an invitation-only event that gather s top executives, m ar keter s, c r e atives, and technologists w ith the vi s i on and influence to change how p eop le c o m municate and tell stor ies in the d igita l ag e . Ra c h it s ky i s a l s o a c o n s u l tant for the S und ance Film Festival’s New Fro n t ie r p r o g r a m and s p e n t four year s in the d ocum entar y d ivision a t Pa r t ic ipa n t M e d i a , wo r k i n g o n Food I nc . and Waiting for S up er m an. Background I w a s bo r n i n Ru s s ia , a n d s t a r t e d off my car eer i n fi l m. My fr i end’s dad w as a D P for do c u m e n t a r ie s , a n d I t h o u g ht, “O kay, that’s i nter esting.” I t filled a lot of th e bu cket s t h a t I wa s in t e re s t e d in. So I star ted wor king i n film, and j umped to Pa r t i ci p a nt M e dia , wh ic h ma de soci al good documentar i es—including An Inco nve n i e n t Tr u t h , Fo o d In c . , a n d Wai ting for Super man. An d a l o t of t h e s e films we re s t a r ted fr om the stance that stor i es can r eal ly cr eate i m p a ct , a f fe c t pe o ple e mo t io n a lly, cr eate a deeper under standing of somethi ng th e y di d n ’ t k n ow be fo re , a n d po s s ibly change the w ay that they act and live . I w a s t h e r e for a bo u t fo u r ye a r s , a nd I star ted getti ng intr oduced to mor e tec h n o l og y- d r ive n t h in g s . A n d I s t a r ted to feel like peopl e wer e havi ng similar c o nver sa t i on s , a n d in s o me c a s e s d oing thi ngs in the same kind of w ay, l i ving i n a bu bble . T h e n I we n t t o Su n da n c e a n d s t u mbled on New Fr onti er. At that ti me , i t w as d ow n i n t h e ba s e me n t a n d wa s mostly made up of inter active instal l ati ons. And b e i n g i m m e r se d in t h e s e la r g e in s t allations gave a differ ent kind of feeling to a n e x p e r i en ce …mo re e x pe r ie n t ia l t h an a specific pl otl i ne . T hat left me w ith a fee l i n g th a t st ayed wit h me . S o I st a r t ed dig g in g a lit t le f u r t h e r, and came acr oss the Nati onal F i l m Boar d contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

141

o f C a n a d a , w h o h ave a n in t e r a c t ive site . T hey’ve been putti ng funding tow ar ds e x p e r i m en t a tio n in in t e r a c t ive s t o r ytel l i ng for over a decade now. And they ha d s o m e r e a l ly in t e re s t in g s t u f f t h e re , i ncl udi ng one semi nal pr oject cal l ed “Wel co m e to Pi n e Po i n t ”, wh ic h is min ima l in inter activi ty but has a r eally compelling stor y, an d e n ou g h ag e n c y s o t h a t I re a lly fel t l i ke I as cr eati ng my ow n exper i ence th at i t too k m e t h r o u g h . T h a t w a s t h e fir s t in k lin g fo r me o f, “O kay, this i s inter esti ng. Peopl e ar e usi ng tec h n o l og y a n d s t a r t in g t o e x plo re space ,” and then I j ust star ted getti ng deep er i nt o i t u n t i l I wa s like , “Ok ay. I ju s t gotta jump. I gotta leave [Par tici pant], and I ’ve got t o fi g u r e o u t wh a t t h is wh o le space i s.” So I left, w i th no other j ob, for an i nsp i r ed bu t po o r t wo ye a r o f my li fe! T h e n I d i g de e pe r in t o wh a t e ve r yone w as doing: at Tr i beca, at the I F DA Doc Lab i n A m st e r da m , u n de r s t a n din g wh a t peopl e wer e exper i menti ng w ith: pr oducti on m a p pi n g , p e o ple h a c k in g c o n n e c t sensor s to cr eate inter active instal l ati ons. L e a r n i n g m ore a bo u t in t e r a c t ive web- based stuff… T he New Yor k T i mes came ou t w i t h “ S n ow Fa ll” wh ic h wa s k in d o f a semi nal pi ece for them, even though it was s o m i n i m a l o n in t e r a c t iv it y. A ll t h ese di ffer ent thi ngs and di ffer ent topics. An d ser e n d i p it o u s ly, a s I wa s lo o k ing for w hat my next thing w as, someone said , “S u n da n c e i s lo o k in g fo r s o me o n e to over see the New F r ontier s stor y lab w hil e th e n ew pe r so n is o n ma t e r n it y le ave .” I t w as a per fect segue , so I j umped i n an d i t w a s t h e pe rfe c t s pa c e fo r de e p expl or ati on, of seei ng and under standing w h at p e o pl e wer e ma k in g . B u t t h e r e w a s s t ill ju s t n o dis t r ibu ti on str ategy for it, no w ay to r eally shar e th e wo r k . I t w a s s t ill ve r y v a g u e , a n d people didn’t r eally under stand it. But it w as al so r e a l ly f u n be c a u s e yo u ’re k in d of the weir d ki ds i n the cor ner ! S o t h e n I m ove d ove r t o h e lpin g progr am for the New F r ontier exhibi tion at th e fest i v a l , wh ic h wa s g re a t . I did that for a couple year s and that’s w hen the K i ck st a r t e r c a mpa ig n c a me o u t fo r O culus. And someone’s l i ke , “Hey, look at thi s.” An d I w a s l i ke , “H u h , t h a t ’s in t e re s ti ng.” So one of the pr oducer s and I went d ow n t o t h e D K o f fic e in Ir v in e , wher e I thi nk ther e w as one R ift station i n the c o r n e r. D K 1 , o n e PC , ju s t a bu n c h of guys… obvi ously w ay befor e the F acebook acqu i si t i o n . An d I t r i e d a c o u ple t h in g s . On e was cal l ed VR Ci nema, w her e you’r e i n a theater by you r se l f , w a t c h in g a mov ie . It w as si mi l ar, i f you think about i t, the beginning of fi l m , w h e r e t he y ba s ic a lly pu t a c a m er a on a live stage pl ay. And I ’m sur e if peop l e b a c k t h en , were pro ba bly t h in k in g , “O h, that’s amazing. I ’m abl e to w atch this i n a d if fer e n t sp a ce .” S o i t fel t l i ke t h a t mo me n t —wh e re I w as in a stage movie theater w atching a m ov i e , be for e t h e y s t a r t e d ma k in g wor k for the new medium.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

142

S o we pr o g r amme d Oc u lu s a t Su n d ance that year and we had about four pieces: C h r i s M i l k ’s Be c k 360 pie c e So u n d and Vision ( w hich star ted as a web piece an d t h e n w a s po r t e d t o Oc u lu s ), Cl ouds, VR Cinema ( show ing the shor t fi l ms a t S u n da n c e ) , a nd o n e o f t h e e a r ly g ames. An d t h a t yea r, I wa s a ls o pro g r a mmi ng the panels and conver sati ons. We had o n e p a n el on T h e Fu t u re Of St o r y t e lling, w ith Nate Miller and Chr i s Milk and a few o t h e r pe o pl e . A n d t h e qu e s t io n was, “What i s going to be the futur e?” And the an swer w a s, “V ir t u a l re a lit y.” It w a s f a sc i n a t in g . Th a t wa s t h e pivotal year at Sundance , w hen the br oader film c o m mu n i t y sta r t e d pay in g a t t e n t io n to New Fr onti er at Sundance , and the word s t a r t e d sp e e din g a bo u t t h is n e w inter esting thi ng. Stor ytel l er s star ted to see t h e i m p l i c a t i on s. It wa s a n e x c it in g t ime . Af t e r t h a t , I st a r t e d wo r k in g o n t h e F utur e of Stor ytelling confer ence , and moved to N e w Yo r k . So t h a t wa s a t ime o f ver y br oad thinking about stor ytelling i n the d ig i t a l a g e — th e ps y c h o lo g y, t h e mar keting, the technol ogy. And i t became mor e ap pa r en t t o me t h a t t h is a ll le d t o VR . V R i s t h e c o mpila t io n o f e ve r y t h in g. I t’s pr oj ecti on mappi ng, shi fti ng a physi cal s p a ce a r ou n d yo u , wh e re yo u fe e l li ke you’r e tr anspor ted somew her e else . Now yo u d o n ’ t a c t u a lly n e e d t o g o t o a physical l ocati on to exper i ence that. And you c a n c r e a t e i nt e r a c t iv it y wit h in a V R pi eces— w hi ch web-based stuff w as doi ng for i nt er a ct i ve sto r y t e llin g … bu t V R is actually better for that. VR cr eates that space fo r you t o r e a lly be fo c u s e d a n d inside . S o a f t er m o r e e x plo r in g , I de c ide d that I w asn’t going to do the tr adi tional d o c u m en t a r y. A n d t h e n I ju mpe d over to O cul us Stor y Studi o, and focused sol ely o n v i r t u a l r e a lit y. An d t h a t w a s e x c it in g , be c a u s e t h e past couple year s have been a ver y exciti ng ti m e i n V R . A n d pa r t o f t h a t is be c ause lots of peopl e w ho wer e doi ng di ffer ent th i n g s be for e a re ju mpin g in V R n ow. You have r eally smar t, for w ar d-thinking tec h n o l og i st s . Yo u h ave c re a t ive a r ti sts. You have cr eati ve game maker s. You have fi l m m a ke r s, s t o r y t e lle r s , a n d wr it e r s. T her e’s a conver gence of gr eat talent putti n g th e i r m i n d s t o g e t h e r, c o lla bo r a t ive ly, to come up w ith new solutions. Whi ch is s o m ew h a t a s o ppo s e d t o film, wh er e peopl e often feel l i ke they alr eady know th e an swer s. Th ere h a s be e n re a l s pir it of innovati on in VR . B u t I ’d say u nt il la s t ye a r, t h e re wa s still a question: “can you r eally tell a stor y i n V R? ” An d Oc u lu s St o r y St u dio wa s put together to answer the question. Henry T h e fi r st e x pe r ime n t wa s Lo s t , wh i ch had mini mal stor ytelling but paved the w ay. A n d t h a t pu s h e d fo rwa rd t o Henr y, w hich w as much mor e of a thr ee-act contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

143

s t or yt el l i n g st r u c t u re . An d t h e n a s yo u k n ow, n ow we ’re seei ng an i nfl ux of content fr om al l par ts of th e gl o be — f r o m a pe r s o n in t h e ir ba s ement i n Poland to major VF X houses, to l ar ge H o l lywoo d films t o s o c ia l is s u e s . S o n ow I ’ ve le f t St o r y St u dio a n d I’m wor ki ng w ithin O culus on a secti on we’r e c a l l i n g “ E xpe r ie n c e s ”. Be c a u s e V R is much mor e than games, and O culus is c o m m i t t e d t o e x pa n din g t h a t , t o f undi ng, pr oduci ng, mar keting and distr i buti ng non - g a m i n g co n t e n t . We’r e pa r t n e r in g wit h re a lly we ll-know n br ands such as Disney, and as well as w i th i ndi e de ve l ope r s . I t h in k we ’ ll be able to shar e a l ot mor e new concepts, new d isc ove r i es, w ay s t o ma ke c o n t e n t better and mor e i nter esting. T h e t h i n g t h at I a lway s e v a n g e lize is: go i nto a VR pr oj ect w ith the feeling that you w a n t t o c r e a t e , r a t h e r t h a n like s c r i pt. I n the tr adi tional sense of stor y, w hat i s th e fe e l i n g ? A n d if it fe e ls wro n g , t h e n ther e’s something you have to change . An d a l o t of t h is wo r k is be in g do ne i n a time w her e ther e r eal ly isn’t a pl aybo ok. F or e x a m p l e , H e n r y wa s a n a ppro ach to exper i menti ng w ith thr ee- act str ucture w i t h sl i g h t i n t e r a c t iv it y. It a f fe c t e d a lot of peopl e , because it had a familiar i ty of s t or yt el l i n g , bu t a ls o pe o ple fe lt invol ved. T hey wer e seei ng in it a ki nd of i nvi si bl e e le m e n t o f i n t e r a c t iv it y, in wh ic h Henr y w as abl e to l ook at you. An d t h o u g h now we t h in k a bo u t t hat as a smal l thing, being one of the fir st to acc o m pl i sh t ha t , fo r a la r g e r, g e n e r al audience cr eated an i mpact. I thi nk it al so m a d e pe o pl e re a lize d h ow in t ima c y can wor k w i thi n VR , and how to achi eve th at. H e n r y i s r e a lly h e a r t fe lt . T h e n t h er e ’s a mu s ic pro je c t Old Fr i ends by Tyler Hur d called, w hi ch is a c o m p l et ely wa c k y pro je c t : yo u h ave thi s cr azy mar chi ng band i n w hich ever yth i n g i s r e a ct i ve t o yo u . So wh e n yo u lo ok up, the sun l ooks at you, and smiles at you . An d you ’r e i n t h e c e n t e r o f t h is wor l d gone mad. [l aughs]. Dear Angelica D ea r An g e l i c a e mploye d a t r a dit io nal illustr ator, Wesley Al l sbr ook, w ho came in an d st a r t ed dr awin g f la t . Th e dr awi ngs wer e al l scanned, unti l one of our coder s c r e a t e d a d r awin g t o o l s pe c ific a lly for that pr oject. An d e ver y si n g le lin e wa s h a n d-dr aw n by her, i n that w ay. I t cr eated a pr oject th at no o n e h a s eve r s e e n be fo re . Th e re i s somethi ng power ful about the hands- on e le m e n t , be c a u s e h u ma n s c a n re a lly connect to a human line ver sus a computer line. I t h i n k V R st o r y t e llin g c a n be c o me l ess about thinking of how we see stor i es a n d m ov i es, a n d mo re a bo u t s t o r ie s t h at we see i n our imaginati on. Because VR is a l ot contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

144

m o r e o f h ow we c o n c e pt u a lize t h ings or i magi ne thi ngsâ&#x20AC;Ś an d o n e w ay is wit h o u r ima g in a t ions and our memor i es, ve r su s w h a t ou r e ye s s e e o f t h e wor ld. F or i n st a n c e , why do we n e e d c e ili ngs and w hy do we nee d f l o o r s? W hy do we n e e d t h in gs to l ook as they ar e? W h en we h ave a me diu m in wh ic h anythi ng can happen. T h i s g i ves t h e o ppo r t u n it y fo r yo u r imaginati on to r eveal i tsel f o u t si de o f t h e t h in g s t h a t o u r eyes see . And the m o st e x c i t i n g t h in g s a re t h e a r t is t s that ar e under standing th a t a n d pu sh in g o u t o f t h a t . S o I t h i n k t h at a s H e n r y wa s a g re at step in teaching us ab o u t h ow i n t ima c y c a n wo r k , D e ar Angel i ca w as a gr eat s t ep i n t ea ch in g u s h ow we c a n bend the r ul es of r eality, w h i c h i s i n cr e dibly e x c it in g .

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

145

INTERVIEW: TED SCHILOWITZ FOX INNOVATION LAB (2016) 26 *INTERVIEW CONDUCTED BY SETH SHAPIRO

FEBRUARY 2016

Te d S chilow itz is a Futur ist and V R exp lorer at 2 0th Centur y Fox, b est-know n for c ham p ioning the acclaim ed T he Wild V R e x p e r ience , a com p anion to the R eese W i t her sp oon film ; and T he Mar tian V R Ex p e r ience , a w id ely-r egar d ed 20 m inute i n t e r active excur sion thr ough the m ain s e gm ents of the featur e , available on each o f Gear V R , V ive and O culus R ift. Ted is also s e l f - p r ofessed them e p ar k junkie w ho has b e c om e the b est-know n evangelist for V R w i t hin the H ollywood stud io com munity. Ted w as p r eviously “ lead er of the r eb ellion” at R e d Cam er a, and w as r ecently announced a s the n ew CineVangelist at Bar co. Q: What have you seen that’s interesting lately? Do you think this is the year of VR and AR? Te d: F i r st t h i n g is I s e e t re me n do u sly inter esti ng thi ngs all the ti me . However, wh at I wo u l d say i s t h e v a s t ma jo r it y o f t h ings I see ar e not inter esti ng at all and don’t p u sh th e enve l ope , a n d do n ’ t a llow me to expl or e , and don’t chal l enge me . T hey ar e safe an d t h ey a r e in e pt mo s t o f t h e t ime . So now I ’l l be mor e dipl omatic , tr uthful ly by n o fau l t of t h e wo r k t h a t ’s g o in g o n b ecause the monetization hasn’t r eally happ en ed ye t , so you h ave a lo t o f pe o ple t h at w ant to get i nto somethi ng but don’t have th e s o p h i st i ca t i on , wit h a bu dg e t , o r t h e r eal aplomb to actual ly cr eate somethi ng t h at’s m e a n i n g f u l , bu t t h e y wa n t t o t r y, a nd they w ant to l ear n. S o I ’ m a l l for t h a t , bu t it ’s pre t t y clear that we’r e at the beginning of a con ten t re vol u t i o n , n o t t h e middle . Wh ic h means that good and i nter esting things ar e few an d f a r be t we e n . Q: When you’re introducing people to VR, what’s the first thing you show them? Te d: T h e t h i n g I like t o s h ow fir s t t o peopl e on a Gear VR i s Chr i s Mi l k’s, Evol uti on of Ver se , b e c a u s e it ’s s h o r t , s we e t , t o the poi nt, extr emely immer sive , takes advan tage o f t h e p l a t for m we ll, a n d is t r a n s for mati onal. People come out of i t and go, “Wow, th a t w a s a m azin g . Th e re ’s a ma zin g col or s, and l i ght, and you move me up and d ow n , an d t h e n you pu t me in t h e c r a dle of a giant baby at the end.” And ever ybody goes, “Oh , my g osh, t h a t ’s a g ia n t ba by.” I t’s cr azy. contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

146

An d t h a t ’s some t h in g t h a t c o u ld n ot r eally be expl or ed well i n tr adi tional medi a, so I t h i n k i t ’s a v a lu a ble t h in g t o s e e . When you talk about inter activi ty, w hich I th i n k i s of c o u r se c r it ic a l t o t h e s u c c e s s of the medium, ther e’s a few ki nd of key tou ch p o i n t s: m o bi l e , a n d t h e n we c a n h o p up to tether ed. T her e’s a wonder ful game cal l ed Espe r. A g a i n , t h e y c o mbin e t e c h n ology i n stor y r eal ly wel l , the stor y ethos r eal ly we l l . S o i t ’s l i ke a n E SP jo u r n e y wher e you contr ol things your “mi nd,” w hi ch m ean s yo u ’r e j u st m ov in g a lit t le t h in g a round, but i t completely wor ks, and it’s fun, and i t’s charming. T h e ot h e r t h in g t h a t I love a n d love to poi nt peopl e to to pl ay i s somethi ng c al l ed K e e p Ta l k i n g a n d No bo dy E x plo des, w hich is an inter active game that used th e R p lu s t r a di t i on a l, like o pe n a la pt o p, and you can do i t r emotely. You basically have to tal k som e bo dy t h ro u g h de f u s in g a bomb, the per son that’s i n VR . I t’s gr eat. T h e n e w A n sh a r Wa r s t h a t ’s o n Gear VR is r eal ly, r eally wel l thought out and d on e as a fi r st p e r s o n s h o o t e r, s pa c e s h ooter thing. And something called Gunj ack, wh i ch i s k i n d o f l i ke a re du c e d, re fin e d, mobile ver sion of “EVE: Valkyr i e”. Same gr oup, CCP, d id i t . Th ey ’r e bo t h g re a t . Q: What about the HMD side? Te d: Th e M a r t ia n pro je c t t h a t we di d, I bel i eve is r eal ly tr ansfor mati onal. I t i s c o m p l et ely c o mpe llin g a n d in t e re s ting, and i t is stor y fir st technology. I t’s abo u t a ha l f h ou r. Yo u live in V R fo r a h a lf hour. You’r e usi ng ver y advanced contr ol s, so you have h a n ds i n V R, yo u h ave a bo dy in VR . You ar e essenti ally playing the par t of M ar k Wa t n ey, a n d yo u ’re liv in g t h e jo u r n ey of the book/the movie , the R i dley Scott m ovi e , an d of c o u r se , Ridle y wa s ve r y involved in cr eating it. So it w as a pr etty big h on or now for t wo ye a r s in a row t o be accepted into Sundance for T he Wi l d l ast year an d fo r Th e M a r t ia n t h is ye a r. An d p e o pl e I do n ’ t t h in k pu t t h e cor r el ati on together that you can’t just like sh ow up t o S u n da nc e a n d g o, “Hi, we ’re Fox, put our stuff in.” You have to be accepted j u st l i ke a fi l m h a s t o be a c c e pt e d a n d go thr ough a cur ati on pr ocess. So you know, you c a n ’ t bu y t h e m o f f . Th e y h ave t o like i t, and they say, “Come and be a par t of it .” So i t’s a ct u a l ly qu it e big t h in g . Q: When you think about VR, do you think about theme parks? Te d: S o pe o ple a s k wh a t g ive s me the r i ght to be a futur i st, do this stuff for a b i g m ov i e st u di o, a n d mo re t h a n a ny t hi ng, it’s because I gr ew up in O r lando, F l or i d a. An d I w a s a t h e me pa r k k id. A n d I gr asped and under stood at a ver y ear ly age th e c o n c e pt of i mme r s ive s t o r y t e llin g and bel i evability. T h e w h ol e i d e a o f wh e n yo u g o t o a theme par k is you’r e supposed to believe you we n t t o a n e w pla c e . Th a t wa s wh at Di sney w as the master in.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

147

H e c r e a t e d wo r lds . A n d yo u we nt to “Tomor r ow l and,” and, “Fr onti er l and,” an d , “A dve n t u r el an d.” W h e n yo u we n t o n the “Pi r ates of the Car ibbean,” r i de , you tho u gh t yo u wer e on a bo a t in t h e C a r ibbean. And w hen you went on “Space Mountai n ,” i t’s not j u st a r o lle r c o a s t e r, yo u ’re in space , man. So the thematic and the stor y on top o f t h e r i de me c h a n ic s , a n d t h e re was actual ly a thing cal l ed “Mission to Mar s.” It w a s a l i t t l e c lu n k y, bu t if yo u c oul d set your sel f in the believabi l i ty of i t, it w as re a l ly c o o l . T h e re wa s a ls o, in a ve r y ear ly for mati ve thing for me , and some pe op l e have se e n i t , bu t it wa s n ’ t s u c c e s s f ul commer ci ally, they had a concept cal l ed D i sn ey Qu est . T h er e wa s o n e in Or la n do, one i n Chi cago, and then maybe a coupl e oth er s. An d o n e o f t h e t h in g s t h e y h a d w as the “Aladdi n Magic Car pet” r i de that w as act u a l ly a t r ue V R e x pe r ie n c e . Now this i s in the ‘90s. An d you wore t h is g ig a n t ic bowlin g bal l hel met thi ng that bal anced r eal ly well, bu t i t w a s l i ke t wo C RT t u be s a n d re quir ed an SGI O nyx for each system to r un. T h e p r obl em w a s t h ro u g h pu t h ow do you get it thr ough to peopl e? And you sat on l i ke th i s sa ddl e t hin g , a n d yo u f le w t h e magic car pet. It’s l i ke you u s e d t o h ave t o g o t o the theme par k, now the theme par k can c om e to you . An d t h a t ’s re a lly wh a t a lot of this i s. Because now you can br i ng it hom e , yo u ca n m a ke mu c h lo n g e r fo r m exper i ences. You don’t have to wor r y about r i d e th r o u g h p u t , an d s a n it a t io n , a n d a ll these thi ngs. You’r e star ti ng to br i ng it hom e . In my t a l k t ha t I g ive , I ma ke a lo t of r efer ence to the Wi i and how impor tan t th e W i i w a s. B e c a u s e if yo u t h in k a b out w hat the Wi i did, they actual ly vir tual i zed e n t e r t a i n m en t e x pe r ie n c e s t h a t c ome fr om the r eal wor ld. Yo u we r e l i ke , “Th is is re a lly f u n .” Im agi ne w hen it gets one step fur ther and we can p u t ou r sel ve s in t o a n e x pe r ie n c e w her e we get one mor e step of bel i evability, wh i ch i s w hy I o f t e n ma ke t h is re fe re n c e that you’ve hear d me say off and on many ti m es, an d so m e w h at c o n t rove r s ia lly, a bout defi ning the language and the semanti cs : ar e we r e a l ly t h i nk in g t h is is V R, o r is thi s an i mmer si ve 360 visual exper i ence? Both of w h i c h a r e g r e a t , bo t h o f wh ic h c a n be phenomenally i mpor tant and val uabl e… bu t m o st o f t h e t h in g s t h a t I s e e t h a t p eople woul d cal l VR , I woul d not cal l VR . I wou l d c a l l t h e m “ i mme r s ive 360 v ie win g exper i ences,” or even, “immer sive 360 m i l d ly i nt er a ct i ve e x pe r ie n c e s .” F or V R t o be V R, pe o ple u s e dif fe rent cr i ter i a, like pr esence . And that’s a defi n i n g c h a r a ct er i st i c , bu t fo r me , it ’s a bo ut believabi l i ty: ar e you in an envi r onment w h er e yo u fee l l i ke t h is is a c t u a lly h a ppe ning to you? And is most of your br ain acce p ti n g i t? N ot a l l o f it – be c a u s e t h e h e a dsets ar e still too cr ude; we sti l l see the pi xel s, we s t i l l fee l t h e phy s ic a l pre s e n c e o f what we’r e wear i ng – we’r e still ti ed to a comp u ter. B u t eve n w i t h a ll t h a t , if it ’s we ll-c r afted and well-cr eated, you star t to believe - an d yo u st a r t t o t h in k , “C a n I a c t u a lly d o thi s?” I look dow n over the l edge and I ’m l i ke ... yo u k n ow, i t ’s h a rd t o t a ke t h e s t e p.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

148

N o t i m po ssi ble t o t a ke t h e s t e p, but har d to take the step. R i ght? At some poin t i n th e e vol u t i o n o f t h e t e c h n o lo g y, and the stor ytelling, and the cr aftsmanshi p f r om p e o pl e d o i n g t h is if t h e me dia a t this point r eal ly fi nds an economi c base , i t w i l l get so g oo d – bo t h t e c h n ic a lly a n d cr eatively – that your br ain w i l l not be abl e to d isc e r n w h e t h e r it is a c t u a lly h a pp eni ng to you or not. Whi ch then opens up al l th e se r e a l ly in t e re s t in g dis c u s s io n p oi nts about is that a utopi an wor ld, a dystop i an wo r l d , o r some wh e re in t h e middle? Q: So when you think about immersive 360 vs. VR, is there a clear point of demarcation? Te d: Ye a h , I t h in k it ’s le s s a bo u t t h e har dw ar e and i t’s mor e about w hat peopl e ar e d o i n g w i t h t he ir c re a t ive e x pe r ie n ces, using softw ar e and har dw ar e . So I thi nk you c a n u se a l l t h e t o o ls t o c re a t e re a lly i nter esting things. But I think that today, and th i s w i l l a lw ays b e , t h e re n e e ds t o be a cer tain thr eshold of w hat the har dw ar e i s cap abl e o f t o r ea l ly cre a t e be lie v a bilit y, o r else you j ust r eally have a moder n Vi ew master s y st em . An d t h e re ’s n o t h in g wro ng w ith that. I mean obvi ously View master n ow re l ea se d a n e w V ie wma s t e r s y s t e m . B u t G oo g l e C a rdbo a rd, fo r a ll o f its success i n ter ms of scale and scope , is a l ook ar ou n d v i ew in g s y s t e m. It ’s n o t a v i r tual r eal i ty system, although they’ll be happy to c a l l i t t h a t a nd s o r t o f la t c h in t o t h at. I don’t bel i eve it’s that. And that is by the w ay, not a c r i t i ci sm o n wh a t t h e y ’re do ing. I t’s just an aw ar eness for me as to w hat i s. It’s n ot a ny l e s s awe s o me , it ’s ju s t not the ful l enchilada. So that’s kind of my sor t of take on i t . A n d I re a lly like t o pre ss ever ybody. I w ant to see peopl e keep push i n g th e e nve l ope . I wa n t t h e m t o pu s h the envelope unti l the envel ope star ts to p u sh b a c k on t h em. Th a t ’s wh a t we did on the “Mar tian.” We pushed so har d that, l et m e tel l you so m e t h in g , it re a lly s t a r t s to push back. You’r e l i ke , “You cannot do thi s.” Q: The technical limitations, or the physiological limitations? Te d: B ot h , e ve r y t h in g . Ye a h , a ll a c ro ss the boar d. And we’r e r i ght at the r azor ’s ed ge , w h i c h m a kes it e x c it in g . Q: How does all of this sit in the history of storytelling, narrative, all that? Te d: E ver ybo dy t h a t s t u die s me d i a, and stor ytelling, and any ki nd of l angu age to en t e r t a i n a n d in fo r m, k n ows this innately, is that new for ms of media an d e n t e r t a i n m en t e me r g e , t h e fir s t t hing most do is r each back i nto the ol der for m s an d t r y a n d co bble t h e m in t o t h e new for m. So w hen tel evision fir st hit the wor l d , r a d i o w a s t h e o rde r o f t h e day, a nd people wer e liter ally saying, “We’l l sor t of p u t r a d i o o n t h i s n e w me diu m.” An d i t w a sn ’ t u n t il a c t u a l in n ov a t o r s like Lucy and Desi Ar naz said, “Not exactly. I t’s m o r e l i ke t h e a t e r. It ’s mo re like a p lay that we can bui l d i nto thi s little box and n ot p e o pl e st a n din g in f ro n t o f mic ro phones in the theater of the mind,” w hi ch is r ad i o, w h i c h i s of co u r s e h av in g a wo n de r ful r esur gence w i th podcasts. And I am a big f an o f l i st e n i n g t o “Th is A me r ic a n Life ,” and, “Ser i al,” and, “R adio Lab,” w hi ch by the w ay, contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

149

I h i g h ly r ec o mme n d t o a ny bo dy t h at loves stor ytelling. Which by the w ay, i s vi r tu al re a l i t y. S o I do n ’ t n e e d a h e a ds e t t o c re a t e vir tual r eal i ty. Listen to a R adi o Lab epi sod e or T h i s Am e r i ca n Life e pis o de a n d le t your br ain be the vir tual r eality devi ce . I of ten m a ke t h e r e fe re n c e t h a t t h e o r ig i nal vir tual r eal i ty devi ce is a r eally good b ook, w h i c h i s you ’re lo o k in g a t wo rds on a page and your mind is buildi ng a wor ld th at yo u a r e se e i ng , a n d be lie v in g , a n d enjoying, and consumi ng. And w hen you’r e d on e w i t h a bo o k , if it ’s a g re a t bo o k , you felt like you went ther e . “Th e M a r t i a n ” is a pe rfe c t e x a mple of thi s: it’s a gr eat, gr eat book, w hi ch mea n s i t c r e a t e s t h e illu s io n o f v ir t u a l re a lity, w i th nothing you’r e putti ng on your head . I t’s j ust you r m i n d. It ’s ju s t yo u r ima g in ati on, w hich is mor e power ful than any technol ogy I t h i n k . S o t h a t ’s s o r t o f a n in t e re s ti ng poi nt. So r adi o tr i ed to become T V, and th en T V sa i d , “ N o, n o. Th is is TV.” A n d then T Vs and movi es star ted to collide toge th er, an d p e o pl e we re like , “No, t h e y ’re differ ent ki nd of ar t for ms. We can do diffe r en t th i n g s w i t h b o t h .” An d t h er e you s e e a ma zin g filmma ker s like Alejandr o w hich does Bir dman, and th en R e ve n a n t , a nd h e ’s … yo u wa n t t o tal k about pushing the envelope unti l i t real ly s t a r t s t o pu sh ba c k , t o t h e po in t wher e they al most ki l l themsel ves out ther e , r i gh t? T h e l evel of filmma k in g c in e ma is i ncr edi ble . An d t h a t i s so r t o f v ir t u a l re a lit y. You w atch i t on a big giant scr een, you bel i eve yo u ’r e t h e r e . Yo u be lie ve it h a ppe ned to you. But i t’s sti l l a fl at pi ece of medi a th at d o e sn ’ t g i ve yo u t h e a bilit y t o move in and ar ound it and inter act w i th it. W h a t V R do e s s o mu c h be t t e r t h an other media i s that: i t gives you the abi l i ty to i nt er a ct w i t h t h e wo r ld, a n d move ar ound i n the wor l d like you do in the r eal wor l d , w h i c h i s w hy we s t a r t t o re fe r t o it as vi r tual r eality. And someone made thi s p oi n t, i t w a sn ’ t m e . So me o n e s ma r t e r t h an me made this poi nt in a gr oup discussion, says, “Wel l , we kee p t a lk in g a bo u t v ir t ual r eality, but i sn’t it vir tual fantasy mor e th an vi r t u a l r ea l i t y ?” Fo r t h e mo s t pa r t , the kind of enter tai nment you’r e cr eati ng i s tr yi n g to t a ke pe o ple t o s o me pla c e t h a t isn’t actually r eal , that i sn’t actually the r eal wor l d . An d I sa i d , “ Yo u ’re a bs o lu t e ly r ig h t. Most of pur e enter tainment i s vir tual fan tasy.” W h er e you s t a r t t o t a lk a bo u t v ir tual r eal i ty i s, “What if I can br i ng someone to a s p o r t s e ve n t o r a t h in g t h a t ’s h a ppeni ng i n the new s and not documenting t o fi l l i nt o a f l a t sc re e n , I’ m lit e r a lly pu t t ing a camer a i nto the wor l d and...wow. I ’m in th e new s.” S o t h e be l i e v a bilit y e f fe c t is re a lly i nter esting, so ar tists have been tr ying to fi nd th e m e di u m of be lie v a bilit y fo r ye a r s . And as we get better technology, better pi xel s, b e t t e r scr e e n s , be t t e r, be t t e r, be t ter, but i t’s sti l l a pain, it’s still a w i ndow to th e wo r l d . V i r t u al re a lit y bre a k s t h e w indow, l i ke now you’r e on the other side of th e l oo k i n g g l a ss. Yo u a re in s ide a wo r ld, and you can cr eate stor i es that w ay. Al m ost e ver y t a l k I g ive o r pa n e l dis c u s si on I ’m on, ther e’s alw ays the question of l i ke , “W h a t a b o u t t h e dire c t in g a n d t e lling a nar r ati ve stor y and tel l i ng a di r ected sto r y? ” contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

150

An d I say, “ L o o k , t h e re ’s a lo t o f thi ngs I ’m r eal ly wor r i ed about i n ter ms of w h at V R c a n be , a n d will it be s u c c e s s f u l, and ar e we cr eating enough of a per fect st or m , an d i s t h e r e t h e r ig h t c o n t e n t a t t he r i ght time , at the r i ght pr i ce poi nt, at the r i gh t m a r ke t i n g , a nd t h e r ig h t a ll t h a t s t uff? All that pedestr i an stuff that makes some th i n g s u cc e ssf u l . Th e o n e t h in g I’ m c o mpl etely not wor r i ed about i s that r eally cr eati ve p e o pl e w i l l fin d a way t o t e ll re a lly compelling stor i es in the new medium. An d I ’ m se e i ng it e ve r y day. We g ave you some examples. Li ke Bullet Tr ain is a stor y th a t i s f u l ly in t e r a c t ive , a n d yo u ar e believi ng thi s thi ng i s happening to you, an d th e r e i s a st or y a rc . A n d t h in g s a re going on. “Mar tian” is a stor y in VR . I t’s not j u st V R for V R ’s sa ke , t h e re ’s a be g in n ing, middle , and end. T hings happen i n a seque n ti al o r de r, bu t you h ave f re e do m, wh ic h you don’t have w hen you’r e w atchi ng a m ovi e s c r ee n . I g u es s yo u c o u ld le ave t h e theater, that’s your fr eedom. I n our wor ld, i t gi ves yo u a bi g g er c a nv a s , a wo n de r la n d. T h e c a nv a s i s n o t f la t , it ’s n o t ju s t 3D, i t’s the canvas of r eal life cr eated ar ti fi ci al ly. Q: Is there a minimum technological threshold for that definition of virtual reality? In other words, is there 360 video tech that will just never really be VR capable, that is really apples and oranges? Te d: I t h i n k so me pe o ple mig h t disagr ee w ith me on this, pr obably a lot of peop l e wo u l d . Th er e a re a ma zin g 360 im mer sive videos that I ’ve seen that i t cr eates i nc r e di bl e e mo t io n a n d e mpa t hy. Someti mes peopl e call VR an empathy mach i n e , w h i c h I co m ple t e ly a g re e wit h be c ause i f i t’s ar ound you, you star t to bel i eve , r i gh t? B u t i n t er m s o f my t h re s h o ld, a n d maybe j ust because I ’m doi ng it so much I ’ve bu i l t a l e vel of so ph is t ic a t io n , my t h re s hol d of w hen I w atch a 360 video shot w i th a 3 6 0 c a m er a i s I t h in k it ’s a ma zin g , a n d I think it’s...l i ke I look ar ound and I star t to get a s e n se o f w h a t ’s h a ppe n in g a ro u n d me . B u t I n e ver a c t u a lly be lie ve it ’s real i ty. I didn’t get on a tr anspor ter fr om a Star Tr ek a n d a ct ua lly dro p r ig h t in t o t he w ar zone . I believe I ’m w atching someone th at re c o r de d t h a t wit h a ve r y s o ph is t icated camer a that I can l ook ar ound. Because th e m i nu t e I t r y a n d le a n in t o it , o r move ar ound i t, or do w hat I would do i n the r eal wo r l d i f I w a s a c t u a lly in a wa r, like hide under the r ock and tr y and get aw ay f r om b e i n g k i l l e d, t h e illu s io n is bro ke n. I ’m stuck in space . I know that I can’t actu al ly d o t h a t , so I ’ m in a c irc le v is io n 3 60 exper i ence . Ver y advanced but I ’m not f u l ly b e l i e v i n g . It do e sn ’ t m e a n it ’s n o t t o u c h in g my emoti ons deeper than a tr adi tional medi a, w h i ch i t t ot a l ly i s, bu t I’ m n o t f u lly be lie v ing it. When we get to vol umetr i c moti on pi ctu r e c a p t u r e , l i ke we h ave vo lu me t r ic C GI , then I thi nk I w i l l star t to bel i eve . Like if I w as i n a w a r zo n e a n d t h e s t u f f wa s a ll happening ar ound me , and people wer e r un n i n g ar ou n d m e , a n d t h e y we re r u n n in g ar ound me , and I saw them and I fel t them , i f I l oo ked t h i s way, t h e y wo u ld s o r t of move ar ound. Like I do i n the r eal wor ld. T h en s u d de n ly, t h a t le ve l o f be lie v a bilit y w ill get to over 70% .

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

151

N ow fo r m e , wh e n e ve r I wa t c h 36 0 spher i cal captur e , of w h i c h by t h e way, t h e re ’s s o me a mazing wor k done . Chr i s M i l k i s a n ot h e r g re a t e x a mple o f wow, that’s incr edi ble . And F el i x a n d Pa u l, t h e re ’s t h is a ma zin g Bill Cl i nton piece w her e yo u g o t o Af r ic a a n d yo u ’re wit h Bill Clinton and you’r e in hi s o f fi c e , a n d it ’s g re a t Bu t it ’s n o t r eal. I t’s gr eat i mmer sive 3 6 0 c a p t u r e s t o r y t e llin g . B e c a u s e I ’m never like actually b e l i e v i n g h i m in Bill C lin t o n ’s o f fic e . Now, if you wer e her e an d I w a s se e in g yo u we re Bill C lin ton, and it w as shot w i th a vol u m e t r i c t o o l, a n d a s I le a n e d i n he woul d physically get c l oser t o me a n d h is c o f fe e c u p woul d get cl oser to me an d h i s d e sk . . . T h e n my l i z a rd br a in wo u ld k in d of go, “Holy shit. I s thi s act u a l ly h a p pe n in g t o me ?” A n d t hat’s vir tual r eality, and th e n a v i r t u a l f a n t a s y, a v ir t u a l enter tainment. Now the th r e sh o l d i s re a c h e d. So I t h in k we’r e cl ose . I thi nk we’r e c l o ser t h a n a lo t o f pe o ple t h in k . T hat technology is being d e vel ope d, t h e re a re a lo t o f c o mpani es wor king on it, th e r e ’s m on ey be in g s pe n t o n it . CG does i t today. T hat act u a l ly i s t h e n e x t s t e p.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

152

INTERVIEW: ERIC SHAMLIN, SECRET LOCATION (2017) *INTERVIEW CONDUCTED BY SETH SHAPIRO

27 APRIL 2017

Er i c S ham lin is an Em my and Cannes L ion aw ar d -w inning stor yteller w ith over 20 ye a r s of exp er ience acr oss multip le cr eat i ve i n d ustr ies; film , television, visual effects, ad ver tising and d igital. H e is V P, Managing Di r e ctor at S ecr et L ocation, a lead ing stu di o at t he for efr ont of em er ging d igital content .

Tell us about your background. I’ve be e n wo r k in g in me dia fo r a bout 23 year s now. I star ted as a field pr od u cer an d br o a d c a st n e ws pro du c e r fo r ABC New s, cover i ng l i ve events and tr avel i n g to i n t er n a t i o n a l h o t s po t s … a lo t of documentar y, conflict, and natur al di saster c ove r a g e . O u t o f a ll t h e c h a pt e r s i n my car eer, that’s pr obably one of the m ost fun a n d a d ven t u ro u s . Th a t c u lmin ated w i th me w i nni ng an Emmy and a coupl e of nom i n a t i on s fo r do c u me n t a r y wo r k. F r om t h e r e , I move d t o t h e We s t Coast, and got i nto visual effects for sever al ye a r s a t a sho p c a lle d Ra diu m. We did effects for big featur es and hundr ed s of c o m m er c i a l s, a t a t ime wh e re a lo t of shops wer e sti l l tr ying to figur e out w hat th e i nt er n et w a s a ll a bo u t . I dove h e adfir st i nto tech and digi tal tr yi ng to figur e th at o u t . I t h e n t r a n s it io n e d t o a big , gl obal digi tal adver ti si ng agency and led gl ob al d ig i t a l m edi a e f fo r t s fo r s o me o f the bi ggest br ands: Chevy, Nintendo, HP, Ad ob e , D or i t o s, B u dwe is e r. I ma de a n a me for myself in tr ansmedia… br i ngi ng substan ti al l i ve a ct i o n , v is u a l e f fe c t s a n d dig it a l ski l l sets to bear on some r eally i nnovative wor k at t h e t i m e . An d t h e n I go t a c a ll f ro m A pple , aski ng me i f I w anted to joi n thei r team. Wh i ch w a s obv i o u sly in c re dibly f la t t e r in g and tur ned into a gr eat ti me; I l ear ned a lot— i t’s a r ea l ly wo n d e rf u l g ro u p o f pe o ple . U lti mately though, I decided that I ’m less of a p r odu ct a n d t e c h g u y, a n d wh ile I speak that language wel l , I ’m fundamentally m or e o f a st o r yt el l e r a n d wa n t e d t o g e t back to my stor ytelling and media r oots. S o I l ef t t h e B ay A re a , move d t o LA, and star ted fr eelanci ng at a number o f b i g ad a g e n ci es a n d pro du c t io n c o mpanies. I lucked out and star ted wor king wi th a c o m p a ny c a l le d St o pp, wh e re I wa s blessed to wor k w ith Chr i s Milk, a leading voi ce i n t h i s i n du st r y, a n d Fre de r ic k Fr izell, w ho is now the CO O of Her e Be Dr agons.

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

153

I ju st h a p pe n e d t o be in t h e r ig h t p l ace at the r ight time wor king w ith some i nsa n ely tal en t e d pe o ple , do in g pro ba bly t he most i nnovati ve wor k in the space at the ti m e . T h e n J a m e s Milwa rd, my c u r re n t boss and Pr esi dent and F ounder of Secr et Loca ti on , hea r d a bo u t me . H e a s ke d if I wa n ted to come on boar d, and l ead the U.S. offic e for th e m . I ’ ve be e n wit h Se c re t Lo c a t i on about two year s now, and we ar e pavi ng th e w ay for t h e f u t u re o f e n t e r t a in me nt. I t’s been a gr eat r i de so far. I t e l l p e o pl e t h a t I be lie ve t h a t 2017 i s the w i nter of VR , ther e may be a lot of s h op s c l o si n g a n d some c o n s o lida t io n in the industr y. But as far as I can tel l , ther e’s noth i n g bu t bl u e sk i es fo r Se c re t Lo c a t io n r i ght now. Q. After all of the spaces you’ve been in, what made you fall in love with working in VR? T h e B e c k Pr o je c t by C h r is Milk did that for me . I saw the potential of w her e i t cou l d go, a n d i t w a s s o me t h in g I didn ’ t w ant to l et go. T her e’s a small cohor t of cr e ati ve p e o pl e w h o have do n e t r a dit io n a l and di gital and games — and VR uni tes al l th ose s k i l l set s. I f yo u s e e yo u r s e lf a s a tr ansmedia, cr oss- media pr ofessional , thi s i s th e p e r fec t m e diu m. B e c a u s e yo u re a lly do need to have a good gr asp of the var i ou s p r e- ex i st i n g me dia t o c o me in a n d not scr ew things up. An d i t ’s t h e bir t h o f t h e me diu m. So to be abl e to play at the bi r th of the med i u m , s e t t h e r u l es , a n d g e t t o da bble a nd exper i ment as it fi nds i ts w ay, finds its mar ket, fi n ds i t s a d o pt io n … I c a n ’ t ima g in e a mor e r ew ar ding place to be r i ght now. Q. How did Secret Location’s Sleepy Hollow Experience with FOX come about? In t h e su m m er o f 2014, Se c re t Lo c ati on w as talki ng to F ox about doing some d i gi tal m a r ke t i n g ef fo r t s a ro u n d t h e ir s h ow Sleepy Hollow. T his w as w hen the DK1 h ad c o m e o u t f r om Oc u lu s , a n d o u r t e am w as ear ly i n the Kickstar ter. We got a bun ch of th e h e a d set s in t h e o f fic e a n d s t a r ted ti nker i ng w i th them to see i f we coul d fi gu r e o u t h ow t o do V R s t u f f . T h e n we h a d a me e t in g wit h Fox, w ho w anted us to do some di gital mar keti n g ar ou n d “ S l ee py H o llow.” Th e y me ntioned that they had space at Comic Con, an d w a n t ed t o do a n e x pe r ie n t ia l a c t iv ation ther e . And basi cal ly i n the r oom, we pi tch ed th e m on t h i s ide a : “Wa it , we ’ ve g ot the DK1, you w ant an exper i enti al thi ng, l et’s fi g u r e t h i s o u t . Le t ’s do a c o o l V R thing for Comi c-Con.” An d t h e y sa i d ye s . Th e pro ble m was i t w as seven weeks befor e Comi c-Con— w h i ch i s a mu r de r o u s t ime lin e . T h e n t h e D K 2 c a me o u t t h e we e k befor e Comi c Con. Which r esul ted i n our develo p er s b e i n g on t h e ph o n e ba c k a n d fo r t h w ith the O culus dev suppor t team j ust to m ake s u r e i t wor ke d! Th e re h a dn ’ t be e n a pr ogr am that had done this yet. T h a t pr o j ec t wa s t h e fir s t t o do live action char acter s in a ster eoscopi c game engi n e

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

154

fo r V R . S o i t wa s qu it e a t e c h n ic al challenge , asi de fr om the cr eative chal l en ge . We wer e wor k in g wit h Oc u lu s o n how best to do thi s—how to make i t a good e x p e r i en ce , to ma ke it wo r k , pe r io d. B u t a s soo n a s t h e fir s t c o n s u me r audience went thr ough i t, it w as amazing. I t w as a sm a sh h i t a t C o mic C o n t h a t ye a r. Mor e than 10,000 people exper i enced it… j u st a r ea l ly bi g suc c e s s . In t h e f a l l o f 2015, Se c re t Lo c a t ion w as the fir st company i n the wor l d to w in an Em my fo r a VR pro je c t . V R wa s n ’ t o ur cor e busi ness at the time , but since that C om i c C on i n 2 0 1 4, t h e ph o n e h a s n o t s topped r i ngi ng for us to do VR . I t’s been a sl ow tr a n si t i on , bu t o f fic ia lly a s o f t h is past August, we shed all our VR ser vice wor k , an d we ’r e n ow foc u s in g e x c lu s ive ly o n VR or i ginals—VR nar r ati ves, VR games and VR tec h n o l og y. Q. Halcyon was a groundbreaker for VR and TV. Can you talk about that? Ye s. We l a u nc h e d Ha lc yo n la s t f a ll. T he inter national ver si on w as in collabor ati on w i t h S y F y I n t e r n a t io n a l, wh o wa n t e d us to devel op a shor t-for m ser i es to fi t i n th ei r i nt er st i t i a l w in dows in E u ro pe a n m ar kets. Essen t i a l ly you ’re in s ide a v ir t u a l envir onment: the livi ng r oom of the cr i me sc en e . Yo u g e t t o i n te r a c t wit h t h e c r ime scene , put together the puzzle of the cr i me sc en e . In be t we e n are in t e r a c t ive v ig n e t t es and puzzle por tions, w her e you actually get to w a t c h t h e l i n e a r e pis o de s o n t h e b ig scr een T V i nside the cr i me scene . S y F y I n t er n a t io n a l h a d t h is in t e re s ti ng chal l enge , w her e they air U.S. for mats o f 2 2 an d 4 4 m i nu te s wh ic h le ave s 6 minute gaps i n the pr ogr amming. And w hen they ai r th e m i n E u r o pe , t h e y c a n ’ t fill a ll t he extr a time w i th ads… so they wer e i nter ested i n u si n g t h e t ime t o t e s t a TV s h ow idea. We ca m e u p wit h a s h o r t -fo r m s e r i es that can al so live as an hour-long dr ama . I t’s a sc i - fi st or y a bo u t t h e fir s t mu rder in vi r tual r eal i ty, and i t w as concei ved a s an hou r- l o n g pi l o t t h a t wa s s h o t it in such a w ay that we can cut i t up into six-minu te c h a pt er s. I t a ire d t h a t way, a s in t e r stiti al mater i al, in Eur ope and then at the en d of e a ch i n t e r st i t ia l it wo u ld dr ive awa reness of the VR ver si on. So then you’d go to you r Oc u l u s, a n d d own lo a d it a n d e x pe r ience that. I t’s been a big success, and a ma ssi ve s e r i e s of l e sso n s fo r u s , a s a TV / V R hybr i d. G o i n g for w a r d, we ’re ve r y in t e re s ted in buildi ng par allel uni ver ses i n VR for a T V s h ow. We h ave t wo pro pe r t ie s wh ich w i l l be announced shor tly, that ar e fol l ow i n g o n t h a t a n d g rowin g t h a t mo de l. Insidious Insi d i ou s w a s a pa r t n e r s h ip wit h Focus F eatur es. I t w as gr eat… it w as ano th er e x p e r i en t i a l 4 D ma r ke t in g e x pe r ie nce . I t tr avel ed the countr y for months i n the l ead

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

155

up t o t h e m ov ie . Fo c u s Fe a t u re s br ought us on to do a the exper i ence in the wor l d o f t h e m ov i e – c o mple t e wit h t h e s tar of the fr anchi se . I t w as cr eated in par tne r sh i p w i t h a physi c a l c o n s t r u c t io n c o mpany that bui l t a tr avel i ng haunted house . T h e r e we r e t wo t r a c t o r t r a ile r s t hat would tr avel to 10 di ffer ent ci ties over th e c o u r se o f t h at ye a r. Th e y wo u ld pu ll up and constr uct a physical, r eal haunted ho u se . R e a l ly wel l do n e , ju s t a n a ma zin g job. T h e n , i f you we n t t h ro u g h t h e physi cal haunted house , you got an opti on fo r an ad di t i on a l c h a pt e r t o t h e Ha u n t e d House Exper i ence , but in VR . So you went thr ou gh th e physi c a l o n e , a n d t h e n yo u did the VR haunted house at the end. T h e r e sp o n se wa s in c re dible . Th e re w as a moment w hen a lar ge number of peop l e c o u l d n ’ t fi n i s h it . . t h e y ke pt r u n n in g out. T hey would take off the headset and scr eam an d l eave t h e h o u s e (la u g h s ). An d a t fi r st , we we re like , “may be we made i t too scar y!” T hat sai d, for a hor r or fi l m , i t w a s a l m o st pe rfe c t in t h e e n d. Q. About Secret Location L a u n ch ed i n 2009 a n d a c qu ire d by E nter tai nment O ne ( eO ne) in 2016, Secr et Locati on c o m b i n e s cu t t in g -e dg e t e c h n o lo g y w ith tr adi tional stor ytel l i ng to cr eate dig i tal ly acc e l er a t e d ex pe r ie n c e s fo r t h e web, mobile , tabl et, and new platfor ms l i ke Vi r tu al R e a l i t y ( V R ) a n d A u g me n t e d Re a lity ( AR ) . I t i s the fi r st company in the wor l d to w i n a P r i m et ime E mmy ® Awa rd fo r a Vir tual R eal i ty pr oj ect and the cr eator o f th e fi r st o r i g i n a l s e r ia lize d V R n a r r a t ive . S e c r e t L oc a t io n is fo r g in g t h e f u t u re of i mmer sive content distr i bution, br i ngi ng n ew p la t for m s l i ke Vu s r t o ma r ke t t h a t ar e changing the face of enter tainment. Wor ki n g w i t h t h e b i g g e s t V R pu blis h e r s in t he i ndustr y, like NY T VR , Vusr is a game- changi n g p la t for m t h a t wa s bu ilt by le ve r a g ing our exper i ence i n pr oduci ng VR for the U ni ted N a t i on s, G o o g le , Oc u lu s , G E a n d m any other s. Q. Do you have any short- and long-term predictions for VR/AR? In t h e sh o r t t e r m, like I s a id, I t h ink it i s a w inter of VR for a lot of compan i es. I th i n k we’r e a lre a dy s e e in g t h e 360 video mar ket is star ti ng to dr y up a bi t bec au se o f t h e g ol d r us h la s t ye a r. A ll o f t h e major br ands w anted 360 video campai gns, th at w i l l st i l l g o o n . W h a t we ’ ve le a r n ed, or w hat we’ve decided, is 360 video is a ver y c h a l l e n g e d co n s u me r me diu m r ig h t now. I t’s r eally har d to tel l good stor i es in 3 6 0 vi de o. A s soo n a s yo u pu t a h e a ds et on, you w ant some l evel of i nter activi ty, an d so th a t for c e s you t o s t a r t u s in g g a me engi ne tools, and i nter active tool ki ts. F or V R t o be c o me a c o n s u me r-f a c ing medi um, you have to embr ace inter activi ty. As a st u di o a t t h e fo re f ro n t o f e me r gi ng digi tal content l i ke VR and the pl atfor m s to d isc ove r, we wa n t t o ble n d n a r r a t ive and games w her e we can. In t h e n ea r te r m t h is is la r g e ly a gamer ’s mar ket, fr om a consumer standpoint. We al so t h i n k t h e re will a lo t o f ro om for open pl ay exper i ences l i ke T ilt Br us h by VR/AR Project contributions@vrarproject.com New Amsterdam Media

156

G o o g l e . A l ot o f t h e 360 wo r k wil l dr y up. Shops and br ands that continue in 3 6 0 w i l l pu sh for mo re pre miu m e x pe r iences, and better image qual i ty but ther e wi l l b e fe wer o f t h em. Pre miu m 360 will r equir e better camer as and a higher pr i ce p oi n t. We’ l l co n t i nu e t o e x pe r ime n t wit h that stuff. I t’s r eal ly fun. As a c o m pa ny t h o u g h , we ’re c re a ti ng mor e inter active , mor e game- l i ke , nar r ati ve e x p e r i en ce s. L o n g t er m , I t h in k lo c a t io n -ba s e d w i l l pl ay a bi g r ole— such as Dr eamscape an d IM A X V R . I ’ ve be e n t o C h in a , I t h ink, four ti mes i n the past year. Location-base d VR i s g oi n g t o be ma s s ive t h e re . I t h i n k t h a t ’s g o in g t o be mo re o f a n episodic medium or shor t pl ay sessi on med i u m . I a l so t h i n k VR g a me s a re g o in g t o explode . T he r umor s ar ound the next X box ar e p r om i si n g . T h e Play St a t io n s ig n s a re pr etty pr omi si ng. A lot of the O culus and Vi ve e x p e r i en ce s a re a ma zin g . It a l so see m s like a ll t h e h a rdwa re manufactur er s ar e coming out w i th self- contai n ed hea d set s, a s a middle g ro u n d be t ween mobile and ful l - on PC: a mid- tier of sel f c o n t a i n e d, m ore powe rf u l 6D OF devi ces. T hat’s i n addi tion to the top tier head sets e m br a c i n g w ire le s s - fo r e x a mple , O culus sneak- peaked their Santa Cr uz pr oj ect b a c k i n O c t obe r a n d V ive is wo r k in g on a w i r el ess headset… so ther e ar e a bun ch of c o m p a n i es d r iv in g in n ov a t io n in t h at space . I think that’s goi ng to be r eally exci ti n g over t h e n e x t t wo t o t h re e ye a r s : to see how sel f-contai ned headsets and w irel ess 6 D OF t e c h n o lo g ie s g a in mo re a n d mor e tr action. M o bi l e i s pr e t t y c h a lle n g e d, be c a u se your phone i sn’t r eally bui l t for i t. I t over h eats an d t h er e a r e pro c e s s o r c h a lle n g e s. Having a sl i ghtly mor e power ful mobi l e de vi ce th a t ’s j u st de dic a t e d t o V R is a re ally good sol ve for the industr y, and I think it w i l l b e c o m e t h e mo bile V R s o lu t io n in the futur e .

contributions@vrarproject.com

VR/AR Project

New Amsterdam Media

157