1
inExistence
by Jingyan He
2
inExistence
3
CSA Research Report
Project Details Project Lead:
Jingyan He (Aesa)
Design Participants:
David Di Duca, Lucy Jones
Title:
inExistence
Type:
Research Project & Installation
Location:
KIRK Lecture Room, UCA Canterbury campus
Project Dates:
1 June - 24 July, Project design 25 July - 25 August, Building and testing on site 25 August - 1 September, Installation open to public
Design Period:
06/2017 ---- 09/2017
Budget:
£300
Scale:
6m x 6m x 3m
Support:
UCA
4
inExistence
5
Fig.1. Simulation of the landscape in Dungeness, 3D models are built and set in Unity, which is a software for game scene making and running. The form of the boat, plant, shed, nuclear power station are all get from Dungeness.
1
inExistence
6
Research Statement
7
Research Agenda and Process Overview In our lives, we experience and inhabit hundreds of thousands of different spaces. With the development of modern technology, we are also keep exploring virtual spaces. And the boundary between real and unreal is blurred gradually. Additionally, the more people get close to the edge of ‘reality’, the more feelings of fear and the uncanny emerge. In Johnson’s words, uncanny ‘is nothing but representation ... the uncanny make us feel mortal, and feel the danger of decay, again and again’(2010: 54). ‘It is, in its aesthetic dimension, a representation of a mental state of projection that precisely elides the boundaries of the real and the unreal in order to provoke a disturbing ambiguity, a slippage between waking and dreaming’ (Vidler,1999: 11). By penetrating the boundaries and connections between real and unreal spaces, this projects explores and presents a distinctive, uncanny experience of the spatial paradoxes between the real and unreal. Taking H.G.Well’s science fiction The Final Men as inspiration for a fantastic virtual space. This project takes the unreal and isolated atmosphere of Dungeness, a desert-like town on the UK coast (see fig.2). It constructs two representations of Dungeness to explore different opportunities for interaction. The representations are both real and unreal, physical and digital.
Research Questions 1.
How to construct an uncanny space in real space?
2. How to construct an uncanny space in virtual space? 3. How to combine a real space and a virtual space, and make them work collaboratively?
Fig.2. The shingle beach with abandoned railway and boat.
inExistence
8
9
Proposal
Design Proposal Significance and Contribution Exploring landscape and psychology. ‘Unhomeliness was more than a simple sense of not belonging; it was the fundamental propensity of the familiar to turn on its owners, suddenly to become defamiliarized, deraelized, as if in a dream’(Vidler,1999:7). Whether interior or exterior spaces, different components in an environment can bring people various feelings. The composition of objects in space can also affect people’s mental state to some extent. Many artists’ installations focusing on this topic are set in one single space. They often: make real space looks unreal; make virtual space look realistic, or make both of them look uncanny. So in this project I am trying to fuse these concepts, blurring the boundary between real and unreal in two different spaces with one interactive installation. In this project, people’s fear could be explored by experiencing the simulation landscape of Dungeness.
Methodologies
To test ‘uncanny’ spaces through the making of a digital protopype, I selected three representive elements in the landscape of Dungeness. These are: the nuclear power station; an abandoned shed and a fishing boat on the beach, There compose three ‘outer worlds’, which in relation I build three ‘inner worlds’ each looking slightly different from the ‘outer worlds’, to explore the uncanny feeling. To test technologies of connecting spaces, I chose projectors and webcameras. The projection on the wall is a meaning to extend the interior space of the real
To test ‘uncanny’ spaces through the making of specifically accurate digital protopype.
2. To test technologies of connecting spaces through projectors and a webcamera.
The installation is designed in such a way when people interact with the game, their actions become a part of the scene in exhibition space. Which means people in this installation is acting the player and spectator simultaneously.
Key technological outcomes of proposal 1.
1.
room, to bring a virtual space into a real space. The webcamera can capture all the thing happened in the exhibition space. The projector represent the teleport from unreal to real and the webcamera means real to unreal. It is also designed to provoke a feeling of blurry interleaving of spaces.
To create a space which is simulates the landscape of Dungeness in Unity, connecting the program with VR glasses so people can experience it.
2. To build a simulation of the landscape in Dungeness in real physical space, combining the prototypes with projections on the wall. 3. To record peoples’ action in both spaces and transmit them between each other. 4. To construct a communication system in which two spaces could affect each other.
inExistence
10
Context
Design Research Context Field of Work
Work by others
The field of research is placed In this VR project (see fig.3&4), on the uncanny experience in Barr explore several different architecture and interior design. forms of the existence of the room, some of them are so surreal The uncanny feeling is one of that people would get a totally people’s psyche. In terms of uncanny experience in it. architecture and interior space, it can be brought out by some eerie, This THEORIZ’s project uses unfamiliar or uncomfortable projectors, house tracking scenes. system (Augmenta) and Vive VR tracking technologies with From the material, visual effects real time video and projection to the illusory atmosphere, some mapping in space (see fig.5&6). It artists who have worked with the blurs the boundary between real concept of the ‘uncanny’ include and unreal in the physical space. Barr, Bürgy, Wilson and the THRORIZ team.
Fig.3&4 Game stills of Pippin Barr’s ‘v r 1’ (2016)
Fig.5&6 Video still of THEORIZ’s Mixed Reality Project (2017)
11
inExistence
12
Methods
13
Design Methodologies 1.
How to construct an installations are painted white uncanny space in virtual and all the surface textures are space? performed by projectors.
Working towards the aim of blurring the boundaries between real and unreal. I start with site visit to Dungeness, capturing photos and videos of its distinctive landscape and atmosphere (see fig.7&8). I simulated three game scenes in Unity, each of them has a shed, a boat and a nuclear power station. Start with these three normal scenes, I added some unfamiliar elements and interactions in each scene, to create another three abnormal scenes relatively. The aim is to provoke an uncanny feeling by switching between different scenes. This explores creating the ‘uncanny’ in the unreal world.
How to combine a real space and a virtual space, and make them work collaboratively? Besides the projection, the technique of capturing real-time frames by webcam, can set the real world into the unreal world (see fig.9).
Fig.7&8 Photographs of landscape in Dungeness
In the exhibition space, when a game player is in the shed, other visitors outside the game can see the game scene projected on the wall through the window of the shed model (see fig.10).
2. How to construct an uncanny space in real space?
Fig.9 Using a certain script attached to the virtual model, this wall turns to a screen which shows the frames captured by webcam of the computer.
There are two physical models in the exhibition space, one is the 1:1 model of an abandoned shed in Dungeness, the other is a model of an old boat. These physical
Critical Design Elements 1.
The interactive VR project which simulates landscape of Dungeness.
2. Coordination between the projection in reality and the physical models which are being projected upon. 3. The interactive system between two spaces.
Fig.10 The testing effect of projections in exhibition space
inExistence
14
15
Methods
Prototyping and testing Unreal World 1.0
Real World 1.0
To test the creation of an uncanny atmosphere in virtual space, I built six game scenes in Unity. Three of them look ‘normal’, the others look slightly abnormal. I tested this ‘abnormality’ by changing the scale, movements and positions of game objects (see fig.11).
Firstly I made a 1:25 paper prototype to test the effect of mapping of projection on it (see fig.16). The frame of projection is a video recorded from the game scene. The physical models are white and have no texture themselves, then they are completed by the projection of The idea also involved testing the virtual scene. the movement between scenes. However the coding script did not Real World 2.0 allow switching scenes randomly By testing the effect of without repetition which may projections on the wall, I found create confusion rather an the scene is interesting when uncanny feeling. people’s action are fusing with the Unreal World 2.0 projections (see fig.14.). So rather This time I chose the regular than putting a video on the wall, sequence of switching scenes. I projected the screens of game To create a virtual world growing when it is running. In the new weirder and weirder, I kept the lecture room, the VR area was models in ‘Unreal World 1.0’ and set at the center of the exhibition gathered them in the same game space (see fig.16&17). Which scene instead of different scenes. means when visitors interacting First I tested the movements with with the game, their physical keyboard and mouse (see fig.12), movements, their shadows on I then changed the interactive the wall and the projections of system onto handle controllers of the game compose another kind HTC Vive (see fig.13). of spatial interaction (see fig.18).
04d
04b
04e
And a wood frame with white cotten sticked on it was used to test the effect of receiving projection (see fig.19)
Fig.11 The story board of the VR game: Unreal World 1.0 04f
16
inExistence
17
Fig.12 The opeartion interface when I building the VR game: Unreal World 2.0, player moved by ketboard.
Fig.14 The interesting scene which fuse people and virtual game together, captured in a spacial angle.
Fig.13 The opeartion interface when I building the VR game: Unreal World 2.0, player moved by physical movement and interact by handle controllers.
Fig.15 1:25 prototype
18
inExistence
19
Fig.16 The plan of the arrangement in exhibition space.
Fig.18 Testing of projection
Fig.17 The rendering of the arrangement in exhibition space.
Fig.19 Making the wood frame wall
inExistence
20
Methods
Fabrication Techniques The virtual models are made in 3DMAX and the VR game is built in Unity. Basiclly the interactions are based on an assest named ‘Virtual Reality Toolkit’, it can add various actions to a game object such as grabbing or moving (see. fig.20-1).
this scene, they can see the sign saying ‘no exit’ in a huge mirror. And they cannot find the teleport pad to go back, which means here is the end of the game (see fig.25).
In the exhibition space, two sensors set on the ceiling can Animations are used to perform construct a virtual play area in the some uncanny changes in this game. (see fig.27). ‘virtual world’ (see fig.20-2). For The webcamera set at the example: corner of the exhibition space -1. The boat by the seaside also can capture player’s actions and can be grabbed and thrown into transmit the frame back to the the sea, and the whole game will game (see fig.28). be reloaded (see fig.21). The physical model of the shed -2. When people try to pick up is built with white foam boards the glowing fireballs and ‘feed’ so it can receive the projection the withered plant, it will grow up (see fig.26). But restricted to the to a huge size (see fig.22). condition of the exhibion space, the real shed only standing half -3. The nuclear power station in out of the wall (see fig.29). Dungeness was transformed into floating boxes and can be hit by a flashlight in the game (see fig.23).
Fig.20-1 The VR plug-in and scripts of interactions added on game objects
-4. The door of the shed can be opend. When people walk into the shed, they can see their image in a frame captured by a webcamera that is set in exhibition space (see fig.24). -5. Behind those boxes there is a flashing sign, but the words are mirrored. When people walk into
Fig.20-2 The game object has animation and trigger script on it
21
22
inExistence
23
Fig.21-1
Fig.21-3
Fig.21-2
Fig.21-4 Another perspective when player is playing the game.
24
inExistence
25
Fig.22-1
Fig.22-3
Fig.22-2
Fig.22-4
26
inExistence
27
Fig.23-1
Fig.23-3
Fig.23-2
Fig.23-4
09e
09c
09f
28
inExistence
29
Fig.24-1
Fig.24-3
Fig.24-2
Fig.24-4
30
Fig.25-1 Fig.25-2
inExistence
31
Fig.25-3
32
inExistence
33
Fig.26
Fig.28
Fig.27
Fig.29
inExistence
34
Control Systems Unity and HTC Vive Unity is a software which is used to develop video games and simulations for computers. With different plugins, games can be controlled by VR devices such as HTC Vive (see fig.30-1). On the headset of Vive, there are 32 sensors which can detect the realtime position of players. So they can interact with objects of the game and moving in a physical space, which bring peoples a better experience.
BENQ Projectors When two projectors are connected with the computer, they can project two more expland screen on the wall (see fig.30-2). So I created another two editor windows in the game and set them in different perspectives.
35
Methods
Occupation and Interaction The installation is located in the Kirk lecture room. Figure 32 shows the interactive instructions in the game. Figures 33-1 and 33-2 show the different routes and experiences when occupying the installation, both virtually in the game (figure 33-1) and physically in the room (figire 332). The photo of the exhbition
Vive also provides two handle-like controller for various interactions from dragging to shooting things.
Fig.30-1 HTC Vive Headset and Controllers
Fig.30-2 BENQ Projector
Fig.31 The simulated player’s pathway in the game.
space itself, shows two people occupying the installation and the relationship between real ad virtual, and thus the creation of uncanny moments (see fig.34).
36
inExistence
37
Fig.32 Players can see the instruction of interactions when they firstly enter into the game.
Fig.33-2 When one player is playing the game, others can experience a scene which has virtual frames and physical actions exist simultaneously.
Fig.33-1 Screenshots from the view of the headset when someone is playing the game.
Fig.34 When another spectator comes into the shooting range of the webcamera, both the player and the spectator can see themselves.
inExistence
38
39
VR Headset
Webcamera
VR Headset
Screen 2
Fig.35 Drawing of the whole interactive system
Screen 1
Screen 3
40
Review of Outcomes
inExistence
41
Dissemination and Future Work This work will be published on personal website: https://aesahe. tumblr.com/. This project will be keep deveoped on concept and prototypes. The complex layers of real and unreal gave visitors a special experience. So how to explore and expand the experience such as sound or smell is the future topic. Maybe when visitors try to detach from the whole installation and walk to the border of the exhibition space, they will hear a whisper of raindrops on the umbrella, which means they come to another part of my installation and still being evolved (see fig.36).
Fig.36 Drawing of the future development on sound testing
42
inExistence
Appendix
43
Appendix Annotated Code Excerpts
Materials and Suppliers List
#pragma strict var webcam : WebCamTexture;
Timber (from UCA wood workshop) 45mm x 45mm x 1700mm x 2 45mm x 45mm x 1100mm x 2 (timbers in other lengths are not used in final installation)
function Start () { webcam = new WebCamTexture(); GetComponent.<Renderer>().material.mainTexture = webcam; webcam.Play(); } function Update () { }
Foam boards (from Wilcox, Canterbury) 594mm x 841mm x 8 Webcamera (from Amazon UK) VR device set x 1, projector x 2, speaker x 1 (from UCA Equipment Store) Other sundries:
var ani : Animator; function Start () { ani.enabled = false; } function OnTriggerEnter (other : Collider) { if (other.CompareTag (“Player”)){ } }
ani.enabled = true;
Zinc-Plated Steel Chipboard Screw (Dia)5mm (L)80mm, Pack of 25, B&Q UK Zinc Effect Mild Steel Corner Brace Bracket, Pack of 8, B&Q UK High-Speed Male to Female HDMI Extension Cable - 1.8 m, Amazon UK High-Speed HDMI 2.0 Cable - 3m, Pack of 2, Amazon UK
44
Appendix
Bibliography Johnson, L.R. (2010) Aesthetic Anxiety : Uncanny Symptoms in German Literature and Culture, 1st edn, Rodopi, Amsterdam. Vidler, A. (1999) The Architectural Uncanny: Essays in the Modern Unhomely, Cambridge, Mass: MIT Press
inExistence
45
Credits MA Architecture Course Leader: Sam McElhinney MA Architecture Design Tutor:
Image Credits All figures are copyright the author unless noted as follows: Figure 3&4.Game stills of Pippin Barr’s ‘v r 1’ . (2016) From: v r 1 [Game Still] Canada: Pippin Barr. At:https://killscreen.com/articles/time-confrontuncanny-potential-virtual-architecture/ (Accessed in 09.05.17) Figure 5&6. THEORIZ’s Mixed Reality Project. (2017) From: Mixed Reality [Film still] France: THEORIZ. At: https://vimeo.com/220883711 (Accessed on 20.06.17)
David Di Duca Lucy Jones Special Thanks: Anthropologist Lee
inExistence
46
University for the Creative Arts
Canterbury Epsom Farnham Rochester