Land Beyond The Lens by Amanda Ngieng

Land Beyond The Lens

Studio XXII Master of Architecture University of Melbourne Semester 2, 2015 Amanda Ngieng 377998

Land Beyond The Lens

Contents 7

Abstract

Behind the scenes 9 13 43 73 109 153 185 217 233 269

Getting Started Chapter 1: Through the Looking-Glass Chapter 2: Down the Rabbit-Hole Chapter 3: Non-Euclidean Corridor Chapter 4: Vertical Hallway Chapter 5: Little Ocean Chapter 6: A Castle’s Rise and Fall Chapter 7: Flying Chapter 8: Falling Chapter 9: Waking

Research 279 295 327 363

Pixels Virtual Reality Visual and Perception Experiments Interaction Experiments

Archive 377 381 385 393

Wonderland “And... Fire!” Discarded Ideas Rejected Chapter: Luna

A virtual reality: A space you can engage. Architecture for the body, In the digital age.

Abstract When virtual reality started taking its hold in the digital age, it brought with it a blank canvas; an infinite site that we can design in and build on. This virtual space is very unlike this physical world that we are used to â&#x20AC;&#x201C; it is bound by a different set of laws. The overriding architectural concerns are not structure or weatherproofing, but on the human experience and what a space feels like. Land Beyond The Lens is a project that envisions the world of Alice in Wonderland for the digital age â&#x20AC;&#x201C; a classic fantasy story told through immersive environments. No longer do we need lucid dreams, mushrooms or magic potions to fall down the rabbit hole; the Rift leads you in. Focusing on the design of space and the effects on the body, it boldly conducts experiments on phenomenology in digital space, where sight and sound is most dominant and (almost) anything is possible. It is a land that only exists in the digital, a surreal experience that brings forth the child in us.

Getting Started

Setting up

The Oculus has a positional tracking camera, which tracks where the Oculus is in physical space, within the bounds of the camera view. If the Oculus is moved out of the camera bounds, it simply stops being tracked, and the scene appears to freeze in place. This setup allows for people 140-180cm tall to play the game.

180cm 160cm 140cm

130cm

145cm

12 80cm dia

Through the Looking-Glass

On entering the Rift, you find yourself disembodied, small. You can look around but you canâ&#x20AC;&#x2122;t actually move. Directly in front is a massive mirror, softly lit.

Book design

The pages of the book were design based on the original manuscript of Alice in Wonderland. The words are digitally traced and turned into abstract symbols.

Original illustration (Source: http://britishlibrary.typepad.co.uk/inspiredby/2013/04/alice-in-wonderland-originalillustrations-at-the-british-library.html)

You look behind and notice that the mirror is not really a mirror, for it is not reflecting its environment accurately.

Red pill, blue pill

“You take the blue pill - the story ends, you wake up in your bed and believe whatever you want to believe. You take the red pill - you stay in Wonderland and I show you how deep the rabbit-hole goes.” - Morpheus, The Matrix The red and blue pills are popular culture symbols. In the context of The Matrix, the red pill represents the choice to face reality, while the blue pill represents the choice to live in blissful ignorance. Is virtual reality real or an illusion? By placing the red pill beyond the mirror, this chapter subtly suggests that virtual reality is an unknown reality. It is a very real space constructed out of pixels - a space you can be in. It questions what we perceive as real. As the pills are expected to come in two colours, they were also placed to encourage the user to look around. When seeing red pills, will you be inclined to look for the blue pills? Many don’t - their attention was occupied by the question on the book in front of them.

Mirror threshold

Turning back, you look curiously at the question. Of course you want to enter. But how do you say â&#x20AC;&#x153;yesâ&#x20AC;?? There are no buttons to click, no keys to press.

Input methods

In an earlier draft of the chapter, the keyboard was used as a method of input. While it was logical and reliable, it required another person to press the keys. Even if the user could locate the correct keys without looking, reaching for the keyboard would interfere with the calibration of the HMD1. To avoid these issues, head gestures were used as a form of input instead. Since the HMD has positional tracking, the position and orientation of the HMD can be processed and converted into head gestures. By clearly asking a yes/no question, it prompts the user to nod in response.

Draft: You look behind and notice that the mirror is not really a mirror, for it is not reflecting its environment accurately. Blue pills lie before instructions to escape, red pills lie before instructions to enter. Press <Enter>, and youâ&#x20AC;&#x2122;re informed you can now move when leaning forwards and back.

HMD: Abbreviation of Head-Mounted Display.

You instinctively nod in response to the question on the book, which causes the text on the book to change. Adjusting your position, you wait for the calibration to complete.

If DetectNod = True

2 sec

As â&#x20AC;&#x153;facing forwardsâ&#x20AC;? means to face the positional tracker of the Oculus, the text may no longer be visible on screen. Audio cues are used to inform the user that it is still undergoing calibration.

1 sec

The sequence of events

The material of the book is dynamic, fading from texture to another.

If DetectShake = True

Reset HMD

If DetectNod = True

Restart

You are sure that the HMD has been calibrated correctly, so you nod in affirmation. If you shook your head, you would have been taken back to the beginning.

As you are given instructions on how to move, you find that you are no longer locked in one place. As you lean forwards, you find that you are constrained on an invisible rail, with control over the speed of motion.

A different way to navigate

Virtual reality allows digital space to be experienced in a different manner compared to a computer screen. This calls for a move away from the typical WASD keyboard and mouse control, toward different methods of navigation. Using positional tracking data from the HMD, The lean amount can be calculated, which is in turn used to calculate the speed and direction of movement.

Backward

Still

Forward 200cm/s

60cm/s 10cm 15cm

15cm

Leaning forwards you move towards, then through the mirror. You sense a crossing of a threshold as the music abruptly changes.

You notice that you are going to collide with the book. Hesitantly you pull back, but find that you are merely retracing the path that you came from. The only way to proceed is to push forwards.

You get very, very, very close to the book ...

Down the Rabbit-Hole

... and rip right through it.

Design of the tunnel

Down, down, down. Would the fall never come to an end? “I wonder how many miles I’ve fallen by this time?” she said aloud. “I must be getting somewhere near the centre of the earth. Let me see: that would be four thousand miles down, I think-” - Alice, Alice’s Adventures in Wonderland

The tunnel is made of rings that appear to pulse as the user travels through. These open up towards the center, where a change in scale creates a change in pace. By gradually rebuilding the rings to have increasingly distinct edges, the tunnel creates a transition to a low poly aesthetic; a transition into the digital. Incidentally, the tunnel is approximately 243m long.

Influences on tunnel design: Coraline (Source: http://vectors.umwblogs.org/2012/02/16/alices-importance-in-the-adaptation-world-and-coraline/). Alice in Wonderland (2010 film). Serpentine Pavilion 2015 (Source: http://www.serpentinegalleries.org/exhibitions-events/serpentine-pavilion-2015).

Tunnel generation

Entrance

Exit 1

1 Rings appear to pulse as you travel through. 2 Opening up into the center of the earth. 3 Rings are rebuilt to have increasingly distinct edges,

4 Rings are randomly rotated to add a degree of complexity to the form. 5 Alternate rings are expanded to create a layered effect and sense of depth. 6 Rings are lofted or extruded accordingly, converting lines into solids.

The tunnel appears to pulse as you travel through it. As the tunnel dips down you look down as well. You can hear the sound of air moving slowly; the tunnel sounds really deep.

The tunnel curves to the left, and you feel yourself turning with the tunnel. You lean forwards a little more, trying to move faster.

Camera control

How much control should the user have over where they look? In a roller-coaster ride, the body would be forced to follow the direction in which the roller-coaster is moving. In virtual reality, it is possible to simulate this control of the direction that the head is facing; however, unlike with roller-coasters, the body is not actually moving. The head knows that it is looking forwards, not downwards; so it would not feel like it is going down. In this chapter, only the yaw of the body is controlled. Without needing to turn his head, the user continues along the path through the tunnel, feeling himself descending.

Above: Script only affects the yaw rotation of the head (in use). Below: Script sets the rotation of the head to completely follow that of the route it is following (discarded).

Regardless of how much you lean, you find you are not moving faster. Your maximum speed appears to be decreasing, while in fact you are moving at the same pace. Ahead is a gaping darkness... Is it the abyss?

You hear a low rumble and the tunnel ahead fades into existence. The rumbling gets louder and you detect the sound of molten lava. Slowly, you pass through the center of the earth.

Gradual reveal

When the entire tunnel can be seen, users tend to squint to see what is ahead. By gradually lighting up the tunnel, elements of interest can be concealed and revealed at the right times, creating a more fascinating experience.

You find that you are starting to move faster. The tunnel ahead narrows.

Another section of the tunnel lights up as you speed through.

The colours blend from warm to cool and the sound of molten lava fades away.

Ahead, the tunnel morphs into a square exit. You feel your speed increase at an abnormal rate.

Too fast! You hastily lean back to slow down.

III

Non-Euclidean Corridor

... It’s quiet.

You head towards what obviously looks like the next destination, but find that you cannot pass through the hole in the wall.

You look to your right. Can you go this way? You turn and lean forward.

Movements in virtual reality need to be taught

“In the world of design, intuition is a dangerous word — highly individual assumptions driven by what we know as familiar. No two people have the same intuitions, but we’re trained by our physical world to have triggered responses based on our expectations. The most reliable “intuitive gestures” are ones where we guide users into doing the proper gesture through affordance. When an interaction relies on a user moving in a certain way, or making a specific pose, create affordances to encourage this.” - VR Design Best Practices by Leap Motion1

Land Beyond the Lens is designed to slowly introduce the user to its possibilities. Instructions are integrated within. At the start, the user is only required to lean forwards and back, taking a ride similar to that of a user-controlled roller-coaster. The user is then dropped into a corridor, whose sharp turns indicate that one is able to change directions by physically turning. Users seem to relish the fact that they are no longer constrained to an invisible set of rails.

Source: https://medium.com/@LeapMotion/vr-design-best-practices-bb889c2dc70#.pq852ip5y

Guiding movement

During an early stage in the development of this virtual environment, it was thought that users should be completely constrained on a rail, so as to avoid users getting stuck in the virtual environment. They will only be able to control of the speed of movement, and can explore the virtual environment at the pace that they set. However, on testing this, it was found that it could no longer be called an exploration. Something important was lost as users were no longer required to think of what they need to do next, or where they need to go. Users cannot go closer towards the things that interest them. It was confusing to suddenly turn in an unexpected direction. Instead of being constrained on rails, users will be guided subtly though the environment.

Above: The rail on which users move along (discarded).

You find that you are no longer constrained by rails. Following the corridor, you turn again.

You try to turn to the right, but the wires connected to the HMD is preventing you from doing so. You turn yourself a full circle to the left instead.

Corridor layout

The layout of the corridor was designed to introduce some elements of spatial nonsense. The first few iterations were overly complicated, with a series of portals and false walls that turns the corridor into a labyrinth that would be extremely frustrating to navigate. As the virtual reality headset (Oculus DK2) being used has problems with 360 degree head tracking, later iterations of the corridors were adjusted such that users will not need to move directly away from the sensor. The final iteration of the corridor avoided portals altogether, and only included false walls.

On the next page Top: Early iteration, too complicated. Middle: Fixed to account for problems with 360 degree head tracking, but is still too complicated. Bottom: Final iteration with only false walls.

Something seems to be shimmering ahead. You head closer.

It appears to be a translucent brick wall. Perhaps you can walk through it.

False wall

These walls are one of the nonsensical elements of the world, and are textured differently to indicate that it is special. It plays with our sense of materiality. On one side of the wall it appears to be completely solid. On the other it appears translucent. They make use of the fact that digital surfaces are usually only textured on one side, which naturally creating this effect. The diagram on the right is a nonsense element similar to the false wall that did not make it into the Land Beyond the Lens. When the surface is visible, users can walk through it; when invisible, users are blocked. This was deemed too complicated to explain without words, and would only frustrate the user. Since it is easy to confuse the inverse wall with the false wall, the false wall was chosen for its simplicity, and the inverse wall discarded.

On the next page: Inverse wall (discarded).

Trigger Collision Surface (1-sided)

As narrow as possible Min 20cm

DEFAULT

Collision ON

TRIGGER BEGIN OVERLAP

COLLISION END OVERLAP

Collision OFF

Event

TRIGGER END OVERLAP

Collision ON

You realise that you are back where you started. Confused, you walk back down the corridor.

This time, you notice another brick wall. You think that you must have walked past it before, but did not notice it since you were following the turns of the corridor. Perhaps, like the previous wall, you can pass through this? You walk into the wall ...

... and passed right through. You feel pleased with yourself for figuring it out.

You turn the corner and is slightly confused. If memory serves, the circular hole should be on the left wall, and a corridor on the right.

100

101

Non-euclidean

As the user passes through the trigger, the corridor is switched out to a slightly different one, creating an impossible effect of the exit moving.

102

Above: What is expected. Below: What is seen.

103

You walk forwards anyway and realise that the exit is below. It appears that the corridor has been rotated.

104

105

An invisible force pulls you towards the gap in the floor. Instead of falling, you are pulled up though the circular hole and everything fades into darkness.

106

107

IV Vertical Hallway

109

The darkness recedes. Looking down, you feel yourself floating up.

110

111

As you slow to a stop, you look up. Seeing tubes winding off into the distance, you deduce that you are meant to go up.

112

113

Wires

What should the walls of the vertical hallway be made of? In contrast with the simple corridor, this aims to try something more chaotic. The winding tubes are reminiscent of wires, a nod towards the insulated conductors of electricity that drive the digital.

On next page: Iterations of the colours of pipes. Red and blue were originally chosen, but was changed to black and blue after the inclusion of red mushrooms.

114

115

You notice that the mushrooms near you are pulsing and decide to touch one.

116

117

Visual cues

The pulsing of the mushroom draws the users attention and attracts them towards it. Before the pulsing of the mushrooms, users tend to lean to move upwards, which is extremely tiring and very slow. Their eyes are naturally drawn towards the end of the hallway. A flaw with the pulsing of the mushrooms is that if the user is already looking up or down the hallway, they are likely to miss the pulsing altogether.

118

119

As you touch a mushroom, it blinks rapidly and you find yourself at a higher position than before.

120

121

Feedback

The sound of hitting the mushroom, combined with the visual feedback of it blinking informs the user that they have â&#x20AC;&#x153;consumedâ&#x20AC;? the mushroom. An earlier iteration of the mushrooms allowed users to grow the moment the mushroom is touched, without giving any feedback that the mushroom has been touched. Users were understandably confused - it appeared that the mushroom did something, but everything happened very quickly.

122

123

Youâ&#x20AC;&#x2122;ve stopped moving, but youâ&#x20AC;&#x2122;re not sure what happened. It feels like you should have grown, but based on the surroundings, you are still the same size.

124

125

You look down and realise that you have indeed travelled upwards, even if it does not feel like you have grown.

126

127

Am I growing?

[...] all she could see, when she looked down, was an immense length of neck, which seemed to rise like a stalk out of a sea of green leaves that lay far below her. - Alice’s Adventures in Wonderland

Alice was told by the caterpillar that eating one side of the mushroom would make her grow taller, and the other side will make her grow shorter. In the Vertical Hallway, the mushrooms are one of the means of vertical navigation, and only makes you “grow”. Using the red mushrooms of Mario as reference, it aims to inform the user that they are indeed going up. To avoid make the users feel really sick, the mushrooms grow them in steps. Rather than seeing the world whip by as they ascend, they see flashes of the world. If the users were looking down or up, the effects would be obvious; however, if they were looking straight ahead, it can be difficult to comprehend what has happened. Although Alice only grew taller, not bigger, it might be necessary to simulate growth by shrinking the environment.

128

Left: Looking down while being teleported up. Right: Looking forwards while being teleported up.

129

At one point you accidentally collide with the walls. You watch in fascination as the wall flickers.

130

131

You try entering one of the many green pipes ...

132

133

... And is spat out. What happened?

134

135

Orienting oneself

The pipes are the â&#x20AC;&#x153;doorsâ&#x20AC;? of the vertical hallway that teleport the user up. Each of the pipes are linked to another pipe higher up. While the pipes do transport the user up, because the users have no frame of reference on what is happening, it is difficult to determine what exactly the pipes did. Because of this, users do not feel anything; it does not affect them. The surroundings fail to give a frame of reference as everything looks too similar.

Walls

The hallway is constructed out of three layers of tubes. Each layer is identical apart from scale and rotation. This added a necessary density to the walls without making the walls completely solid. However, despite the change in scale, the pipes still look too similar.

136

Left: 1 layer. Middle: 2 layers. Right: 3 layers.

137

Consuming mushrooms and entering pipes, you continue to ascend vertically until you see what looks like the bottom of a massive key.

138

139

Going up a little higher, a short tune plays and you find yourself being pulled upwards.

140

141

You stare as the key moves towards what looks like a solid keyhole.

142

143

A keyhole with no door

This nonsensical element plays with our understanding of the door, the lock and the key. It is understood that a key is used to unlock a door, which can then be opened. The keyhole is a hole that you can insert the key into. In the digital, such mechanisms are unnecessary. The unlocking of the keyhole with no door highlights this.

144

145

Something unlocks (the clouds?) and you continue to ascend, floating past the key.

146

147

The clouds twist as you approach them, as though to let you through.

148

149

As you pass through the clouds, massive plant stalks fade into existence; a surreal sight. You continue going up, past the massive leaves ...

150

151

V Little Ocean

153

... and land with a splash on the ocean. You realise that you can go above and below the ocean as the ocean level is a single sheet rippling in front of your eyes.

154

155

The ocean divides

As the ocean surface is an excellent threshold, it can be used to divide the world into something other than “above water” and “underwater”. The original intention was to have the ocean surface divide the day and night, however on testing, the change in the brightness of the environment was quite painful for the eyes. Wanting to encourage diving in and out of the ocean, it was decided that the ocean surface should simply be used to divide “above water” and “underwater” after all. The composition of other elements is used to create a surreal effect instead.

On next page: Sketches of initial plan for ocean.

156

157

Types of fogs

The digital fog has many parameters, including: Fog Density: The fog layerâ&#x20AC;&#x2122;s thickness. Start Distance: Distance from the camera that the fog will start. Fog Inscattering Colour: Primary colour of fog. Fog Height Falloff: Height density factor, controls how the density increases as height decreases. Smaller values make the transition larger. These parameters can be adjusted to create a variety of moods.

158

Some fog iterations: 1 Density 0.1, Start distance 2000, Black. 2 Density 0.5, Start distance 2000, Black. 3 Density 1.0, Start distance 2000, Black.

4 Density 0.1, Start distance 2000, Pink. 5 Density 0.5, Start distance 2000, Pink. 6 Density 1.0, Start distance 2000, Pink.

159

You are accelerating and moving forward even though you are not leaning. Looking down you can see the silhouette of fishes swimming. The clouds appear to have dispersed.

160

161

You move up and out of the ocean, hearing the muted underwater sounds change to that of waves crashing. Feeling like a dolphin, you happily dive in and out of the water.

162

163

After playing with the waves, you decided to go underwater to follow the fishes. As you look around, you see the silhouette of a huge shark.

164

165

Adding details

Our physical world has so many things in it that the digital often feels empty in contrast. While it is not feasible to fill the digital world with details due to performance issues, Little Ocean attempts to include elements that will only be seen by some users, and not others. A shark swims below the water, a massive deer stands above. These two elements are opposite each other. A simple gesture like this is able to create an element of surprise, particularly as users enter the world multiple times. â&#x20AC;&#x153;Am I going in the right direction?â&#x20AC;? In an open world where users are not given anything explicit to do, events need to happen in order to assure that the user that they are actually progressing through the world, allowing them to relax and enjoy the experience of skimming past the waves. There needs to be points of focus; things to see and study and stare at.

166

167

Scale

Elements are sized and position accordingly to have the desired effect. In particular, the deer was set extremely far away in order for the fog to conceal it correctly; to compensate, it was made bigger so that it still looks huge in first person perspective.

168

Vertical Hallway

Vertical Hallway and Little Ocean seen in elevation.

169

While following the shark, you notice that the ocean is bubbling. You fly above the water to check it out. The ocean appears to have become flat and still - the waves are not rippling.

170

171

Types of oceans

Parameters of the ocean can be changed during runtime to create new landscapes.

172

173

Sequence of events

The changing of the ocean is paced so that it does not feel too fast or too slow, while allowing users enough time to look around. In one test, the ocean constantly changes from one colour to the next. This was simply distracting, and did not add to the experience. It felt like too much was happening, but nothing interesting.

Above: Initial sketch of the ocean events.

174

Above: Script of the ocean events.

175

The earth groans. The ocean - or is it the ground? - is turning green and deforming. The sound of the waves gradually turn into that of an open field.

176

177

The ground continue to groan and deform as you skim around the mountain peaks.

178

179

Checkered patterns appear as the ground starts to crinkle like paper.

180

181

Transition

As users are not restrained in the direction that they are going, there may be times where instead of going straight forward, they decided to go towards the left or right. In order to ensure that users proceed to the next chapter, the elements of the next chapter are set to spawn directly in front of the user, leaving no doubt that that is the next area to explore. Since the user also needs to be above water for the next chapter, it checks the position of the user and makes sure that the user is indeed above water before loading the chapter. At the same time, the shark underwater swims away and out of sight. The deer disappears when the player is not looking.

Above: Script to calculate the location and rotation of the next chapter.

182

Above: Shark swims away to prevent users from following it indefinitely.

183

A Castleâ&#x20AC;&#x2122;s Rise and Fall

185

Directly in front of you, a castle is inflating as the ocean flattens.

186

187

The castle stands proud and tall. The ocean has become completely flat, although it appears to be still rippling - if marble was alive, it would probably look like this.

188

189

Door swings open

The door swings open only when the userâ&#x20AC;&#x2122;s gaze is centered on it, and the user is not too far away. This ensures that users will always see the door opening, catching their attention and directing them to enter the castle through the doors.

190

191

Alternate entrances

In the case that the users miss the door and veer off to the side of the castle, the user is able to enter through the top as well - wherever there are gaps. In an earlier iteration of the castle, it would technically be impossible to proceed if you entered by the top; you needed to pass through the door in order to trigger the rest of the events. By allowing other plausible (though illogical) entries, it makes the environment more robust and rewards the user for trying something different.

192

193

Passing through the doors you enter the castle, although all you can see is a corridor leading to nowhere. You then notice that the walls have a brick texture and realise that they are probably false walls.

194

195

False perspective

Initial sketches show the original intention of designing a castle that uses false perspective, however this did not end up being carried though as the overall of the design of the castle got more complicated.

196

False perspective is applied instead to the tunnel in the castle; it is interesting to note that most people do not notice this. It has the effect of making the user feel like they are moving faster as they travel down the tunnel.

2.1m

48m

7.8m

197

As you pass through the many layers of bricks, the ground changes into lava. It feels dangerous to be here.

198

199

Perception of space

â&#x20AC;&#x153;Reality, as an invention of the human mind, is often thought of as a set of accepted conventions and constructs, such a reality has an inherent dependency upon cognition where spatial and temporal principles may be defined within the natural and built environment, and further embraced within a cultural context.â&#x20AC;? - Reality and Virtual Reality1

The brick texture is applied again to the walls to indicate to users that they can walk through them. Because the lighting is flatter than before, the texture might not be noticed right away; but once users notice them, they are able to make the link that they can pass through the walls. Already the false walls are treated as a normal part of the environment. The bricks are much deeper than the ones in the Non-euclidean Corridor. Should the user attempt to walk in the wall, they will see sectional cuts of the brick wall.

200

Glenn Goldman & Michael Stephen Zdepski, ed., Reality and Virtual Reality, (New Jersey, 1991), 71.

201

You seem to be in the innards of the castle. Large monuments are suspended precariously. Something bad is going to happen.

202

203

A loud explosion. The monuments crumble. It is a cave in!

204

205

After you stop panicking, you realise that you can push against the broken pieces of the castle. Apparently you have a very tough head.

206

207

Balancing performance and effect

The more the castle is broken down, the better the effect. However, performance will take a big hit due to the number of individual pieces. On testing, it was found that at 200, the castle has been sufficiently broken down, and the fps was still reasonable. There is a slight lag but given the dangerous atmosphere, the lag adds to the effect.

On next page, from top to bottom: 100 pieces; 200 pieces; 300 pieces.

208

209

And invisible force pulls you up to the mouth of the volcano. Where did the volcano come from?

210

211

A single, unbroken experience

The Queen gasped, and sat down: the rapid journey through the air had quite taken away her breath, and for a minute or two she could do nothing but hug the little Lily in silence. As soon as she had recovered her breath a little, she called out to the White King, who was sitting sulkily among the ashes, “Mind the volcano!” “What volcano?” said the King, looking up anxiously into the fire, as if he thought that was the most likely place to find one. “Blew-me--up,” panted the Queen, who was still a little out of breath. “Mind you come up-the regular way-don’t get blown up!” - Through the Looking-Glass & What Alice Found There

A transition device was required in order to move from the Castle to the next chapter. Through the Looking-glass provided inspiration in the form of a volcano. From this idea came the breaking down of the castle, and the propelling of users into space. Smoothly transitioning from one event to the next is one of the main focus of the Land Beyond The Lens, as it aims to create a single, unbroken experience.

212

Chapter overviews from Wonderland, the alpha version of the Land Beyond The Lens. Each chapter was standalone, and the only transition was fading to black and back. Continuity was poor.

213

As you are pulled up, you look down at the rubble, fascinated. The castle has fallen.

214

215

VII Flying

217

You pause momentarily at the mouth of the volcano. For a moment nothing happens. Then ...

218

219

A loud explosion. You are engulfed in flames.

220

221

As you shoot upwards, you look down to watch the smoking volcano fade into darkness.

222

223

Stars speed by as you fly up.

224

225

The importance of context

Stars closer to the player shoots past, while the ones further away move slower. The relative movement of the stars gives a sense of speed and depth. When the stars are too close to the user, they move too quickly, making the stars look like dotted lines. When the stars are too far or not dense enough, the user does not feel like they are moving. When the stars are too dense, performance is poor and the environment looks strange. The density, scale and distance of the stars have to be just right. These have to be tested while the camera is moving.

On the next page: The script randomly generates spheres of light, in a set boundary.

226

227

The sky, originally black, opens up into space.

228

229

You feel yourself slowing as you reach the peak.

230

231

VIII Falling

233

The wind howls as you fall with increasing speeds. The stars start to dim, and you halt to a stop as you hear the sound of a parachute opening.

234

235

Controlling speed of flight and fall

Rather than calculating forces (e.g. gravity), the speeds that the player is moving at is completely controlled by timelines. When going up, the speed of the user gradually increases and decreases. Logically, since the user is being propelled by a volcano eruption, he should start moving at a great speed and slow down gradually. However on testing this, the user exits the volcano far too quickly, and does not get engulfed in flames. When going down, the user falls with increasing speed. The abrupt stop is accompanied by the sound of parachute opening, which acts as a clear indication that the user has stopped moving, and something else is going to happen.

Above: When going up, gradually speeds up and slow down. Below: When falling, gradually speeds up and stop suddenly.

236

237

410m/10sec

460m/10sec

For a moment you see only darkness, with a few dimly lit stars, You look around, and see rabbit ears in hats.

238

239

Using field of view

Aiming for a creepy environment, the elements of this environment only appear when it is not within the playerâ&#x20AC;&#x2122;s field of view. It as though things have gotten close to you without your knowledge they are suddenly there, and multiplying. This effect is possible in digital space where it is possible to track every item in the userâ&#x20AC;&#x2122;s field of view.

Above: Script to check if objects is within camera bounds.

240

120Â°

Using field of view: Objects only appear when not within the userâ&#x20AC;&#x2122;s field of view.

241

It seems that the more you look around, the more rabbit-in-hats appear. It is quiet. And creepy.

242

243

As you stare at the rabbit-in-hats, you hear the sound of giggling. The rabbit ears are moving up, and attached to it is a note. An invitation?

244

245

Designing the “invitation”

“Have some wine,” the March Hare said in an encouraging tone. Alice looked all round the table, but there was nothing on it but tea. “I don’t see any wine,” she remarked. “There isn’t any,” said the March Hare. “Then it wasn’t very civil of you to offer it,” said Alice angrily. - Alice’s Adventures in Wonderland

By repeating “tea” multiple times, the handwritten invitation plays on semantic satiation, a psychological phenomenon in which repetition of a word will cause the word to start looking wrong. This is an attempt to use a simple visual to convey the madness of the original novel characters.

246

247

Teapots start to appear, replacing the rabbit-in-hats. The teapots are responding to your head movements.

248

249

Upbeat music starts to play. As you nod to the music, the teapots nod as well. You try conducting the teapots with your head.

250

251

Dancing teapots

The teapots roughly follow the head movements, making them feel very dynamic and responsive.

252

253

The music starts to distort, and you hear it glitching. Bewildered, you watch the teapots glitch too.

254

255

Glitch

In Wonderland1, when rotating the virtual camera to turn the user upside down, an interesting glitch was found by accident. The camera of the player was being constantly flipped, resulting in the surrounding objects going up and down simultaneously. This glitch effect was purposely recreated and applied in this chapter.

Wonderland is the alpha version of the Land Beyond The Lens. On next page: Glitch effect found in an early exploration. 1

256

257

Sound

“Take care of yourself!” screamed the White Queen, seizing Alice’s hair with both her hands. “Something’s going to happen!” - Through the Looking-Glass & What Alice Found There

Although virtual environments are largely visual, the effects on sound on perception cannot be underestimated. Sound gives life to objects. They enhance atmospheres. They trigger memory as our minds draw references. They can give a sense of place. They provide sensory feedback that could give an illusion of touch. The audio is based on the Tea Party song from the 1951 film of Alice in Wonderland by Disney. The music slowly distorts into glitches, followed by a period of suspense before leading to a jump scare. All events were timed to sync with the audio, and are completely out of the user’s control.

258

Normal tea party song

Glitch

Suspense

Scare

Time when the glitch flickers

Above: Breakdown of audio. Below: Portion of script timing the glitch on screen to the sound.

259

The glitch stops, but something is wrong with the teapots. They are no longer following your head movements. In fact they seem to be...

260

261

The teapots turn suddenly to face you, and slowly moves in your direction.

262

263

The teapots are coming!

264

265

The teapots attack!

266

267

IX Waking

269

... and you wake suddenly. You appear to be seeing the world outside in black and white. It has never looked so scary.

270

271

You take off the HMD, returning to reality. A moment later the screen fades and returns back to the start.

272

273

Infrared

The leap motion is able to create a grayscale stereo image of the near-infrared light spectrum, separated into left and right cameras.1 This can potentially be used to show things (e.g. “Welcome back”) that can only be seen in the infrared spectrum, and not in the visible spectrum. For instance, through the Leap Motion, black cotton appear white, while black duvetyne appears black. If the black cotton were to be tacked onto the duvetyne, it would look like black on black in the visible spectrum, but show up clearly as white on black in the infrared spectrum. Objects that are illuminated by the Leap Motion Controller’s LEDs, light bulbs, halogens, and daylight will be able to be seen.

274

Source: http://blog.leapmotion.com/hardware-to-software-how-does-the-leap-motion-controller-work/

Infrared Coatings manufactures opaque black inks and paints which are transparent to infrared light. (Source: http://www.infraredcoatings.com/magic_black_spray_paint.htm).

275

Research

Pixels

279

Pixel definition

Pixel is the abbreviation of the picture element. As the name suggests, it is the most basic unit of the picture; the smallest, controllable part that make up an image, just as atoms would make up matter. It is not a physical unit, but a logical unit. It does not have a physical size; this is affected by screen size and resolution. The same image on a bigger screen would have bigger pixels.

280

The digital and the physical. Image: Babyâ&#x20AC;&#x2122;s hand in hand (Source: http://imgkid.com/mortalidad-materna.shtml)

281

Pixel and the digital

The pixel have always been closely related to everything digital. When something looks pixellated, we automatically think it is from the digital, even if they are not. These are actually agricultural fields viewed from space, showing crops at their different stages of life.

On next page: Top: Viewed from satellite. Orange River, between Namibia and South Africa. Irrigation projects turn parts of the landscape an unnatural green. Bottom: Viewed from satellite. South-western coastline of the Netherlands. The patchwork of colours indicates the various stages of growth and harvest on the agricultural islands.

282

(Source: http://www.theguardian.com/environment/gallery/2011/mar/29/agricultural-patterns-space?CMP=twt_ gu#/?picture=373100538&index=11)

283

Pixel and the physical

There have been the desire to materialise the pixels of the screen; to materialise parts of a game world. Minecraft in particular is a game that is embraced for its pixellated aesthetic, although there has been a shift towards more realistic, albeit blocky environments.

284

Top: Minecraft version of the Kingâ&#x20AC;&#x2122;s Landing in Game of Thrones. (Source: http://www.cnet.com/au/pictures/kingslanding-from-game-of-thrones-minecraft-style-pictures/) Bottom: Minecraft foam pickaxe. (Source: http://www.thinkgeek.com/product/e847/images/8835/)

285

Time

Time in digital space may not always exist, but when it does, the time can be changeable. Play, pause, fast forward, and even going back in time is such the norm in games that we hardly think twice about it anymore.

Example taken from game: The Sims 3.

286

287

Physics

In Red Faction Guerilla, the modellers act like structural engineers, modelling buildings as they would be built in the physical world, so that the player can destroy buildings in a realistic manner.

288

Game: Red Faction Guerilla Top: https://www.youtube.com/watch?v=kzBSHHWuolY Bottom: http://img.mundogamers.com/galerias/ps3/red-faction-guerrilla/red-faction-guerrilla-1245428514.jpg

289

The surreal

Despite not needing to follow any law of physics in the digital, there is still a tendency to simulate the real. This is to create a sense of reality, which then helps to create the surreal. We need something to relate to. Even in fantasy worlds, there is a resemblance to our world. For instance, The waves in the water planet was based off physics simulations. The ice planet in the movie Interstellar is taken in Iceland. To capture the look of an ice planet, the cast and crew of Interstellar shot on location at Iceland’s Svinafellsjokull glacier. Located in Iceland’s Vatnajökull National Park, Svinafellsjokull is part of the largest ice cap in Europe.

On next page: Top and middle: Render. Water planet, Interstellar. (Source: https://www.yahoo.com/movies/remember-when-thespace-crew-in-interstellar-114054637517.html) Bottom: Svinafellsjokull glacier, Iceland. Ice planet, Interstellar. (Source: http://www.unfinishedman.com/icelandssvinafellsjokull-glacier-exploring-skaftel-national-park/)

290

291

Visual glitches

Glitches do not exist only in the digital. Because of the way our mind works, it is easily tricked with optical illusions. When our brain process colours, they are reinterpreted according to what we have seen before.1 Information that the brain thinks are unimportant are edited out.2

1 2

292

http://discovermagazine.com/galleries/zen-photo/i/illusions http://www.livescience.com/28911-7-ways-to-trick-your-brain.html

293

Virtual Reality

295

Dominating vision

“More than just sight is measured in terms of visual acuity; vision is the process of deriving meaning from what is seen.” - Thomas Politzer1

In virtual space, materials are merely a visual product of computer data, behaving as they are scripted to. Unlike matter in the physical world, they cannot - technically - be touched. Pixels are, after all, purely for visual representation; it is only what we see. However, saying that it is only what we see might be a little misleading. Vision is arguably one of our most dominant senses. The rubber hand illusion, and a variation of it - the invisible hand illusion, gives us an insight of how vision, combined with the sense of touch, can fool our conscious minds.1 The person undergoing the experiment (person on right of image) would have her real hand out of sight, while watching the space in front of her being stroked in the same time her real hand is being stroked. She begins to believe that she has an invisible appendage, and will flinch, expecting pain, when someone hits the empty space. The McGurk effect shows how our vision can override what we hear.2 In the video demonstration, the only sound that the man is ever saying is “bar”. However, when the sound is played over an image of a man that appears to be saying “far”, our ears end up interpreting the sound being said as “far”. In virtual reality, pixels dominate our sight. This makes it quite possible to play tricks on the brain; particularly when other senses are also involved.

296

Invisible hand illusion2

1 2

Source: http://www.brainline.org/content/2008/11/vision-our-dominant-sense_pageall.html Source: https://www.youtube.com/watch?v=G-lN8vWm3m0

Image: Invisible hand illusion. (Source: http://phenomena.nationalgeographic.com/2013/04/14/the-invisible-hand-illusion/)

297

In â&#x20AC;&#x153;Donâ&#x20AC;&#x2122;t Let Go!â&#x20AC;?, a game designed for Oculus, you sit behind a desk holding the ctrl buttons on your keyboard. Within the virtual world, you see a pair of hands on a keyboard too. Similar to the rubber hand illusion, the brain can be tricked into believing that the digital hand belongs to its body. When a spider climbs over a virtual hand, you start to feel ghosts of sensations that are not actually there. In an experiment with my 8 year old brother, I trailed a piece of cloth over his arm in the same time the spider climbs over his arm. Interestingly, he does not pull away; despite being slightly scared of spiders, he understands that the spider is only digital and does not exist. Nevertheless, he was fascinated that he could feel it, and to this day does not realise that the feeling was real, and not something he imagined. There was no imagined feeling when I pulled my arm through a knife. It is most likely because I knew what I was trying to do.

298

Donâ&#x20AC;&#x2122;t Let Go! can be sourced from https://share.oculus.com/app/dont-let-go

299

Lava Inc is an example of how a roller-coaster ride same be simulated in the virtual. Even though you are not actually moving, you feel the forces acting on you; a steep drop causes your stomach to feel as though it has dropped. The illusion of movement is able to feed the imagination a feeling of inertia. On a side note, the lava itself was rather irrelevant to the experience. This might be because my brain cannot conceive a lava that do not give off heat; particularly as Iâ&#x20AC;&#x2122;ve never experience lava in physical reality before.

Lava Inc can be sourced from https://share.oculus.com/app/lava-inc

300

301

The first VR head-mounted displays

1957 Sensorama Machine by Morton Heilig: Possibly the first true VR system. Simulator with 3D visual, stereo sound, vibrations of the seat, and wind in the hair.1 1960 Telesphere Mask by Morton Heilig: Possibly the first headmounted display with 3D visual, wide vision, and true stereo sound.1 1995 Virtual Boy by Nintendo: First â&#x20AC;&#x153;portableâ&#x20AC;? video game, runs on battery for approx 4 hours. 3D visual in hues of red and black only. 22 games were released for it before it discontinued a year later. Not immersive. 2

Vision may be our most dominant sense, but it is not enough. To create a truly immersive experience, many people have tried to engage with multiple senses. This can be traced back to the Sensorama machine in 1957, which is possibly the first true VR system with 3D visual, stereo sound, vibrations of the seat and wind in the hair. Despite already having the theory to create virtual reality headsets over 60 years ago, virtual reality never really took off as computers were not quite powerful enough at the time, and technology have not developed sufficiently to support virtual reality.

1 2 3

302

Above: http://www.mortonheilig.com/InventorVR.html Middle: http://nintendo.wikia.com/wiki/Virtual_Boy Bottom: http://tomodachi.wikia.com/wiki/Virtual_Boy (image)

303

The current VR head-mounted displays

However, since the Oculus was founded by Palmer Lucky and a Kickstarter campaign launched on 2nd August 2012, virtual reality has gained increasing interest in the public. The Oculus Rift is a virtual reality headset that has low latency 360 째 head tracking, which is able to translate our physical head movements to movement in virtual space. It provides 3D vision with a wide field of view, allowing users to feel as though they are actually in the virtual space. Along with the Oculus Rift, other headsets have been developed, such as Samsung Gear VR and the HTC Vive.

On next page: Top: Oculus DK2 (Source: http://www.pocketgamer.co.uk/r/iphone/oculus+rift/news.asp?c=54852) Middle: Samsung Gear VR (Source: http://www.pcauthority.com.au/Review/403544,review-samsung-gear-vr.aspx) Bottom: HTC Vive (Source: http://arstechnica.com/gadgets/2015/03/htc-partners-with-valve-for-virtual-reality-headset/)

304

305

Virtual reality gadgets: Movement

2014 Oculus: The Oculusâ&#x20AC;&#x2122;s own head tracking camera can be used to translate motion. This game tracks the up and down movement of the head, translating jump-jogs into movement. 2014 PrioVR: Sensors are strapped across the body to accurately translate movement. 2014 Cyberith Virtualizer: An omnidirectional treadmill that can simulate walking, crouching, etc.

While virtual reality headsets only focuses on vision, other sensors are increasingly being used in conjunction with them to create more immersive and interactive experiences by translating our physical actions into reactions on screen. As virtual reality allows digital space to be experienced in a different manner compared to a computer screen, new methods of interacting with such a space are continually being developed.

306

307

Virtual reality gadgets: Touch

2013 Reactive Grip: Haptic feedback simulating weight, wrenching of objects, etc. 2015 Dexmo F2: Haptic feedback that locks your fingers when it detects contact in the virtual world, preventing the fingers from closing further. 2014 Leap Motion VR: Although the leap motion does not actual provide haptic feedback, it allows one to actually see their hands in digital space. Just by being able to see the virtual hands move as the physical hands move is very powerful; the feeling of being able to touch and manipulate digital objects is made to become all the more real.

Gadgets that provide haptic feedback gives an illusion of touching virtual objects.

Reactive Grip: http://tacticalhaptics.com/reactivegrip_gdc2015/ Leap Motion VR: https://www.leapmotion.com/product/vr

308

309

Virtual reality gadgets: Smell

2009 Virtual Cocoon: This helmet seeks to add touch, taste and smell to virtual reality. 2015 FeelReal: An add-on to the Oculus, this gadget is a mask that tries to bring smell - and even wetness and temperature - to virtual reality. 2011 Polymer Matrix: A polymeric matrix of cells, which can be opened and closed individually, is used to handle thousands of smell with much fewer controls. It simulates smell through the understanding that the world is made out of many smells.

Smell is a very important sense, which some believe is tied more closely to memory than to any of our other senses. It is, however, a very difficult thing to simulate; there is just too many smells that mix together. If only certain smells are generated, it would not feel right.

Virtual Cocoon: http://www.wired.com/2009/03/realvirtuality/ FeelReal: http://www.cinemablend.com/games/FeelReal-An-Oculus-Rift-Add-Lets-You-Smell-Virtual-Reality-70580.html Polymeric matrix: http://www.rsc.org/chemistryworld/News/2011/June/21061101.asp

310

311

Virtual reality gadgets: Taste

2011 Meta Cookie: This involves giving taste to plain cookies through the making use of the fact that taste is affected by what we see, hear, smell, as well as the texture of the food. 2013 Digital Lollipop: Electrical stimulation is sent to different parts of the tongue to simulate flavours, rather than stimulating the sense of smell.

Taste, like smell, is probably best left to the real world. Perhaps when technology gets to the level of that in The Matrix it would be possible, but right now it just seems to take things a little bit too far. The examples below are more to do with augmented reality than virtual reality, but it is good to know where technology currently stands in regards to taste simulation.

Meta Cookie: http://spectrum.ieee.org/computing/embedded-systems/virtualreality-scent-system-fools-flavor-sense Digital Lollipop: http://www.npr.org/blogs/alltechconsidered/2013/12/10/250001919/the-digital-lollipop-you-canliterally-taste-the-innovation

312

313

Virtual reality: Demos

Virtual reality allows you to see what you could not normally see; to do things that you could not normally do. With games, one of its most important features is its interactiveness; without that, the game might as well be a 3D movie. Many game demos have been developed so far that look for innovative ways to use the Oculus.

SightLine: The Chair is a demo that creates realities that can only be experienced in virtual reality, where seeing is believing, and notseeing is change. Things will only appear when they are completely out of your field of vision. VIRTUAL HANDS: VirtPaint has a clean and easy user interface. A target marks the center of the screen to be used as a pointer, which disappears when you start drawing. Focusing the target onto an icon and holding it there for a moment will start “loading” the icon, and after a short wait the icon will be selected. Whether or not anyone can actually draw properly with their head remains to be seen. Crime Watch: The developer team for this demo had determined hand tracking to be out of the picture, and “wanted to try and implement a new kind of game play method that would make sense in VR”. Taking advantage of speech-to-text services, players are able to ask questions verbally to NPCs, just as they would ask their commanding officer, if they had one. Apnea uses the noise generated from the head movement while walking and translates it into movement in the virtual world. Using a microphone, it is able to check if you are breathing, which affects the amount of oxygen you use in game. Breathing out generates bubbles, which is an interesting effect; quite realistic.

Sightline: https://forums.oculus.com/viewtopic.php?t=12158 VIRTUAL HANDS: VirtPaint: http://vrjam.challengepost.com/submissions/36960-virtual-hands-virtpaint

314

Crime Watch: http://vrjam.challengepost.com/submissions/36754-crime-watch Apnea: http://vrjam.challengepost.com/submissions/36764-apnea

315

Virtual reality: Outside the game industry

Apart from games, virtual reality has also been appropriated for other uses, with some more successfully than others.

Medical: Taking advantage of the fact that there are two screens, Diplopia is a game that tricks the brain to use the weak eye. By brightening the screen for one eye and dimming the other, you can force the brain to use the other eye. Education: Virtual reality can be used to teach. By creating an interactive and immersive experience, students are better able to engage in the learning material. Or they may just be extremely distracted by their virtual hands spazzing in and out of the screen. Car industry: At Ford, virtual reality has been used as part of the design process. It allows people to easily test if the sight lines are okay; if things such as the speedometer are in the right place; if the colour feels right. Apart from the obvious limitation of not being able to do crash tests in it, the graphics are not that good, and you cannot feel the materials. Advertisement: Combining the Oculus Rift with the Kinect, custom 7.1 sound design, scented air, a flight harness and a shipping container, 5GUM creates a multi-sensory virtual reality that is unforgettable. Users fly through atmospheric worlds based on each of 5GUMâ&#x20AC;&#x2122;s flavours; worlds that they can touch, smell, see and hear.

Medical: https://www.diplopiagame.com/ Education: https://youtu.be/wsUfaNRBARQ Car industry: https://youtu.be/BEFfp2QhHZU Advertisement: http://www.unit9.com/project/5gum

316

317

Words on virtual reality

“VR at its best shouldn’t replace real life, just modify it, giving us access to so much just out of reach physically, economically. If you can dream it, VR can make it. It’s a medium for progress, not the progress itself.” 1 - Matthew Schnipper

“Twenty years after Osmose, I still believe that immersive virtual space has rich potential for enabling people to step outside their habitual assumptions about reality to see freshly” - Char Davis

“Virtual reality is already a reality. A gestural iPhone emoticon or a simulated packaging foam — all is a 100 percent actual and based on very physical resources. Increasingly transparent and immersive wearable devices will force our notion of the “real” materiality.” - Maiko Gubler

“Then, when you see reality, you suddenly see it with this kind of detail and this density. You see just the sheer reality of it. You just feel things from it. It’s really incredible. To me, that contrast, that feeling that you have when you’re out of it after you’ve used it, has universally been more precious than what happens in it.” - Jaron Lanier

1 2 3 4

318

http://www.theverge.com/a/virtual-reality/intro http://www.theverge.com/a/virtual-reality/oral_history http://www.theverge.com/a/virtual-reality/tenyears http://www.theverge.com/a/virtual-reality/interview

Existence in the digital

“To be entangled is not simply to be intertwined with another, as in the joining of separate entities, but to lack an independent, selfcontained existence.” - Karen Barad1

Even without the immersive experience of virtual reality, we start to mix our physical existence with that of the digital. In games, it is common to refer to the avatar that you are controlling as yourself. If you have played Counter Strike or other FPS games, you may have heard people say: “Ouch! I got hit!”, “I’m dead.” or perhaps “I killed them.”

Karen Barad, Virtual materiality, potentiality and subjectivity, p.5

319

A comparison with the real world

I modelled my ikea bed to scale to compare digital space with the physical. Just as in the scene, the bed was to my right in roughly the same location. Based on the camera bounds I readjusted the camera, then tried to climb onto the bed. Spatial depth is perceived exactly as it would be in the physical world. There are some glitches which are not necessarily bad: it is possible to look into the digital bed to some extent as the digital bed is not exactly lined up with physical bed. Virtual reality is incredibly real. Even though it is currently still possible to pick up the individual pixels of the screen, and graphics are not quite there yet, it is undeniably a kind of space.

320

321

Floor constitution

We know the floor as the surface below us; they can “fall away or out from under, but they are usually there, idling beneath us.”1 A floor negotiates between gravity and the upright body, each step magnetised to the ground. The modern floor as we know it today “assumes complexity in its section in inverse proportion to the simplification of its surface detail.”1 It has a thickness, usually consisting of multiple layers that give the floor its necessary properties e.g. slip resistance, acoustic insulation, thermal insulation, structural support, drainage in wet areas. In contrast, cutting a section through a virtual floor will give you a single line of its surface profile. There is no visible thickness to it, for the stuff that gives it its properties are not visible to the eye. In general, the virtual floor consists of: 1. A visible layer - What the player sees; a single surface. 2. A collision layer - Usually wrapped closely around the visible layer, this layer is invisible and tests for collision, preventing objects from falling through the floor. 3. Parameters - Numerical values that affect the properties of the surface, e.g. friction, restitution, density. These are additional control over a floor’s properties.

Keller Easterling & Rem Koolhaas, Elements: Floor (Venice: Marsilio, 2014), 2.

On next page: Top: Visible layer - Wonderland: Section through “solid” ground. Middle: Collision layer - Wonderland: Collision hulls. Bottom: Parameters - Physical Material (Source: https://docs.unrealengine.com/latest/INT/Engine/Physics/ PhysicalMaterials/PhysMatUserGuide/index.html)

322

323

Gravity and orientation

In virtual space, gravity is but a parameter. It is a number used in physics simulations to imitate the physical world. It is not - does not have to be - ever present, and we often see fantasy worlds take advantage of this.

Zero gravity

A space station interior â&#x20AC;&#x153;is a weightless world where anything can be a floor, and simple directional orientation is a matter of ad hoc agreement.â&#x20AC;?1 In outer space where there is zero gravity, there is no up or down. Virtual reality may be able to create a virtual space where virtual objects behave as though they were at zero gravity, but it cannot truly replicate the feeling of being in outer space. By allowing you to fly, liberated from the floor, it may make you feel as though you are floating - but not at true zero gravity. Unlike in outer space, our physical bodies are still being affected by gravity in the physical world, and our sense of balance allows us to tell up from down.

324

Keller Easterling & Rem Koolhaas, Elements: Floor (Venice: Marsilio, 2014), 70.

Top: Assassinâ&#x20AC;&#x2122;s Creed. Player character moves with liberating agility. There is obviously gravity in the world, but the player character appear to bend these laws with his freedom of movement. (Source: http://assassinscreed.ubi.com/ en-us/games/assassins-creed-unity.aspx) Bottom: VR Demo Weightless (Source: https://apps.leapmotion.com/apps/weightless/windows)

325

Visual and Perception Experiments

327

The digital aesthetic

Aesthetic-wise I decided to focus on minimalistic textures and the surreal. Minimalistic textures are functional as they are good for performance. When used well with colours, they can be very beautiful and will not be inferior to more detailed textures - it is simply a different aesthetic.

On the page: Monument Valley (Source: http://techcrunch.com/2014/03/26/monument-valley-price-and-launchdate/)

328

329

Coloured lighting

As virtual environments are constructed out of pixels, everything is made of light and colour. Using coloured lighting rather than materials to colour the world gives a better sense of depth, resulting in smooth blends of colour, light and shadow.

330

331

Camera position

Changing the position of the camera gives a different way to look at a space. While there is no change in the actual body size - in fact, there is no body in these examples - there is a sensation of being small when the camera is close to the ground, or being tall when it is higher up.

On next page, from top to bottom: Low - 10mm Child - 800mm Adult - 1500mm Insect view

332

333

Camera perspective

Changing the depth of field resulted in warped environments that is difficult to imagine in a 3D space. Unfortunately, the Oculus Rift overrides any depth of field changes as it requires very specific values to create the 3D illusion; a different depth of field value is likely to make one feel very ill.

334

Camera perspective: Depth of Field = 90 Depth of Field = 120 Depth of Field = 150

335

Post processing effects

Using post processing is a powerful way to change the feel of an environment. It is basically a filter laid over the original environment; sort of like putting lens over oneâ&#x20AC;&#x2122;s eyes and filtering what is seen. Because the environment is ultimately made of pixels, a 3D space can be altered (almost) as easily as a 2D image.

336

Indirect Lighting Colour = 1,1,1 ; Illumination = 0 Colour = 1,1,1 ; Illumination = 1 Colour = 1,1,1 ; Illumination = 2

337

Indirect Lighting Colour = 1,0,0 ; Illumination = 1 Colour = 0,1,0 ; Illumination = 1 Colour = 0,0,1 ; Illumination = 1

338

Scene Colour Vignette = 0 Vignette = 0.5 Vignette = 1

339

Colour Adjustment Saturation = 0 ; Contrast = 0 Saturation = 1 ; Contrast = 0 Saturation = 2 ; Contrast = 0

340

Colour Adjustment Saturation = 0 ; Contrast = 1 Saturation = 1 ; Contrast = 1 Saturation = 2 ; Contrast = 1

341

Ambient Occlusion Intensity = 0 Intensity = 0.5 Intensity = 1

342

Ambient Occlusion Intensity = 0.5 ; Radius = 0.1 Intensity = 0.5 ; Radius = 100 Intensity = 0.5 ; Radius = 200

343

Scene Colour Grain Intensity = 0 Grain Intensity = 0.5 Grain Intensity = 1

344

Depth of Field Focus = 0 ; Focal Region = 0 Focus = 800 ; Focal Region = 0 Focus = 100 ; Focal Region = 500

345

Scene Colour Fringe Intensity = 0 Fringe Intensity = 40 Fringe Intensity = 80

346

Scene Colour Colour Grading Colour Grading Colour Grading

347

Ambient Cubemap Intensity = 1 Intensity = 1 Intensity = 1

348

Ambient Cubemap Intensity = 1 Intensity = 1 Intensity = 1

349

Split screen

Considering that there are two screens, it is possible to overlay two different image on each screen. I experimented with using two different materials on a single object, with a material in each eye. The result is most definitely something that one should not stare at for too long.

350

351

Material modification

Materials in the digital are all made through textures. There are a few properties that make up a material: • • • • • •

352

Base Colour: The colour of the material. Metallic: How metal the surface is, where 0 is for non-metals and 1 for metals. Specular: How shiny/reflective a non-metallic material is. Roughness: How rough a material is from 0 to 1. Emissive Colour: Whether the material appears to emit light (no actual light is emitted). Normal: A texture map that sets how bumpy a surface is, where the Red and Green channels give a sense of depth and the Blue channel represents the actual surface. It only gives the illusion of depth through the calculation of shadows; the surface is not actually altered.

353

The dirt material is one of the simplest, consisting of a base colour and a normal map. Beginning from this material, I played around with the material properties, changing its values and combining it with other texture maps. This experiment looks at how realistic textures can be combined in such a way to create strange new materials. The result is one that is a lot more fantastical when viewed on screen, rather than when viewed through the Oculus Rift. Since normal maps do not alter the actual surface, when there is depth perception, the materials look underwhelmingly like flat wallpaper, albeit funky shiny ones.

354

Original

355

356

357

Scale and the body

When units are purely arbitrary, space becomes relative. Measurements become, in a sense, closely related to the body, as they used to be so before universal standard measurements came into existence; A handful, a stoneâ&#x20AC;&#x2122;s throw, a hairâ&#x20AC;&#x2122;s breadth. An object that looks to be a good size on screen can change drastically depending on the size of the person that it is compared to. This is largely because we are used to having scale people to judge the scale of an object in an image. In the scaleless virtual world, methods of determining scale depends on the human body as well as familiar objects of known size. It is possible to feel tall, short, small, big. Raising the camera makes you feel taller. Lowering it makes you feel short. Dynamic changes in height can make you feel like you are shrinking. Enlarging the world makes you feel small. You might feel to be of a normal size in a room that is ridiculously large. Or maybe feel really small in a room that is of normal size.

358

359

Depth cues

Binocular Vision, Stereoscopic Imaging and Depth Cues1 -- The brain uses differences between your eyes’ viewpoints to perceive depth. -- Don’t neglect monocular depth cues, such as texture and lighting. -- The most comfortable range of depths for a user to look at in the Rift is between 0.75 and 3.5 meters. -- Set the distance between the virtual cameras to the distance between the user’s pupils from the OVR config tool. -- Make sure the images in each eye correspond and fuse properly; effects that appear in only one eye or differ significantly between the eyes look bad.

While stereopsis creates the illusion of depth, it is only one of many depth cues that our brains process. When designing form and space we must also consider monocular depth cues, which conveys depth even when viewed as a flat image. Numerous techniques can be used to give a better sense of depth, and distort our sense of scale. Without these, the illusion of depth is broken and pixels centimetres from your eyes become obvious.

360

http://developer.oculusvr.com/best-practices (accessed 13 August 2015)

361

Interaction Experiments

363

Disappearing / Reappearing

A simple script tests if an object has been looked at for 2 seconds; if it has, the object disappears, and reappears 2 seconds after the player has looked away.

Partial script

364

365

Crumble

A simple script tests if an object has been looked at for 2 seconds; if it has, an impulse of a force is spawned on the object being looked at. Created as a destructible mesh, the object crumbles (or explodes) into pieces.

Partial script

366

367

Explosions

A simple script tests if the Oculus has been roughly still for 2 seconds; if it has, an explosion is spawned at the area that the player is looking at. An invisible boundary is placed around the world to dictate where the explosion should spawn.

Partial script

368

369

Reaching out to interact

The Leap Motion is able to track both hands and each individual finger, allowing the movement of the hands to be translated into the digital world. While it is fascinating to be able to reach out into the digital world, it does not come without limitations; the hands tend to glitch, and the maximum reach of the sensor seems to be at 420mm, which is not quite long enough for many arms. The Leap Motion works best when hands are right in front of the sensor and not fully outstretched - a limitation that has been aptly dubbed â&#x20AC;&#x153;T-rex clawsâ&#x20AC;?. However, regardless of limitations, it is undoubtedly still better than no hands.

370

42cm max

371

Hand gestures

The leap motion translates the movement of your hands, allowing users to use actual gestures to pick up in game objects. In an environment where head and hand movements can be tracked, and where objects need not follow the laws of physics, hand gestures can also used to simulate super powers such as telekinesis.

372

373