VIEWING TO CHANGE
Section title
by Komal Mittal
1
2
Vie wing to Change
3
CSA Research Report
Project Details Project Lead:
Komal Mittal
Design Participants:
Lucy Jones
Title:
Viewing to Change
Location:
Herbert Read Gallery UCA Canterbury Campus
Project Dates:
31 May 2018 - 31 July 2018 Project Design 1 August 2018 - 23 August 2018 Building and testing on site 24 August 2018 - 31 August 2018 Installation open to public
Design Period:
June 2018 - August 2018
Budget:
ÂŁ250
Scale:
3.65 m x 3.65 m
Support:
University for the Creative Arts
4
Vie wing to Change
Section title
5
Vie wing to Change
6
Research Statement
Research Agenda and Process Overview
Significance and Contribution
Climate change has led to areas not previously under flood risk also becoming prone to flooding. Such increased risk has led to a great level of uncertainty and the need for drastic change to provide safety and sustainability. While there are countries which have recognised the immediate need for change by using resilient design practices at micro and macro level ( for example, Netherlands), this awareness is still not widely present. For everyone to work on this problem, creating awareness about its magnitude is the first step.
Hyper-sensory experience of being a spatial witness to a flood.
To witness an environment of destruction spatially can be a way of confronting this awareness of a need to change. This could be created by a participatory experience for a person. A participatory experience will allow the user to witness an environment of a natural disaster and come to terms with its enormity. By creating a virtual experience of a flood, my aim is to make people understand how uncertain and devastating a flood can be. The outcome of this experience will hopefully make people more perceptive towards victims of flooding and also force them to make changes in their life which would make them less vulnerable in a similar situation. An experience like this could be used in any condition and might act as the final force used to make the change in regard to safety against flood risk.
Research Questions 1.
In what ways can a ‘spatial witness’ approach develop uses of virtual reality in design research?
2. How can virtual experiences be used to increase understanding of flood risk? 3. How can a virtual experience go beyond the visual? 4. How real can hyper-sensory experiences be?
Fig.01 (previous page) 3D scanned point cloud of Greyfriars Gardens
The magnitude of the threat of climate change can make its impact hard to conceive. When hearing about natural disasters all over the world, the reaction or sympathy is now short lived. People are no longer moved by news. A new dimension is needed to engage people with climate change. An experience close to reality could awaken the reaction not being provoked by the news alone. Virtual Reality has become a platform for exploring new environments, testing limits to an experience and bringing that experience as close to reality as possible. Consequently, I decided to explore how this technology could be used to design the experience of a disaster and how this experience could be enhanced. By using the 3D scanned data as a point cloud, the recreated virtual space is very close to reality. The enhancement is done by making it an hyper-sensory experience which includes senses of hearing and touch along with the virtual experience making the entire experience more immersive for the user.
Methodologies 1.
Digital design of flood simulations in a virtual environment using the 3D laser scanned point cloud of Greyfriars Gardens.
2. Design an installation in which the senses of hearing and touch enhance the visual experience. 3. To display the image and sound effects of the user’s experience to other people around the installation.
7
Vie wing to Change
8
Section title
Design Proposal View to Change is a virtual reality immersive experience of a flood created by making the user a spatial witness to a flood. This event is created in the Greyfriars Gardens, Canterbury. The entire space is laser scanned in a series of points and this point cloud is bought in a virtual environment. As the point cloud is more of an aesthetic experience rather than an editable object, the interaction possible was limited. The installation is primarily focused on how real a first person flood experience can get and to understand its effect on the user.
The installation includes a 3 m x 3 m surface of grass and pebbles which will help the user connect with the visual unfolding of points in the form of the garden space. The user can also hear the sound of the river flowing along with the rainfall, thunder and wind. This is experienced along with the water level rising in the virtual space. These visuals are projected on a TV screen in the real space for other people to see what the user with the headset is experiencing.
Fig.02 3D laser scan extract Greyfriars Gardens
Key technological outcomes of proposal 1.
Using the 3D scanned point cloud data of Greyfriars Gardens and making it available in a virtual environment using Unity which would be accessible by wearing the virtual reality headset.
Fig.03 3D laser scan extract Greyfriars Gardens
2. Adding water(river) to the point cloud data and controlling its movement. 3. Getting the point cloud to interact with water when the space is flooded. 4. Adding a surface of grass, pebbles and sound effects of rain and flood while a person is in the virtual environment. 5. Projecting the same scene on a TV screen with speakers for other people to see what the user with the headset is experiencing.
Fig.04 3D laser scan extract Greyfriars Gardens
9
10
Vie wing to Change
Proposal & Context
Section title
Design Research Context Field of Work
Work by others
The design research is based on increasing flood risk, 3D scanned point cloud data, recreation of a first person flood experience in a virtual space and its effect on the user.
Hyper-sensory experiences have highlighted senses of sound (For example: In the eyes of the animal, Odyssey) and senses of touch (For example: ScanLAB Projects)
Virtual reality was fundamentally used as a tool for gaming, but is now being applied in various educational and training sectors. This helps the user understand a concept by making it into a reality that a person can explore.
The Fig 02 shows a 3D scanned point cloud of Venice done by ScanLAB Projects to understand the depths of the city’s underground and its historic aspects which have not changed over the period of time. They create visual experiences for users using only 3D scanned data. These unique experiences allow the audience to navigate the same data sets captured on location by the scanning team and used to make the original show graphics.
This recreation of experience can effect people differently and then these effects can be used as a medium for tackling the increased flood risk. Virtual Reality is being proved as a vital tool for educational and technological development which can also create enhanced experiences making them into hyper-sensory experiences. There is a lot of work being done around the world where such immersive hyper sensory experiences are created for the user by including senses other than just vision.
Fig.05 Venice - ScanLAB Projects, 2017
The Fig 03 is a still from a project called ‘IMMERSED’ developed by FEMA as a disaster relief training tool. This project places people in a situation after a flood and gives people the opportunity to make decisions to ensure that all the participants have moved to safety after a flood. This project helped increase better decision making abilities with the people and also on how to work together in such situations.
Fig.06 A still from IMMERSED, 2018
11
Vie wing to Change
12
Design Methodologies The project is a virtual reality spatial experience created using different modes of technology like 3D laser scanning, virtual reality combined with photographs, with the involvement of other senses to enhance the user’s experience.
along with the sound of rainfall, lighting and wind in the virtual space. The next step was adding water into this scene and getting it to interact with the point cloud. As part of testing water to interact with the point cloud, I added different types of water bodies available in Unity allowing a variety of control systems. (Water basic daytime, Water pro advanced). After trying with these methods I found out that the water cannot directly interact with the point cloud and would need a medium.
This experience is created on the 3D laser scan of Greyfriars Gardens, Canterbury. The scan of the site resulted in a point cloud which originally comprised of over a billion points. This point cloud was then processed down to a smaller size of around 15 million points and then bought into a virtual space using Unity. So I started tracing out the point cloud as a 3D model which would Enabling ‘sensory transportation’ be used for interaction but would for the user by adding not be visible to the user. The environmental factors in the water was made a part of a mesh virtual space. This includes the built using different points but creation of rain, fog and clouds was still a 2D surface.
Critical Design Elements 1.
Using the 3D scanned point cloud of the Greyfriars Gardens in a virtual space.
2. Getting objects and landscapes to interact with the point cloud. 3. Creating an interactive environment for a spatial witness. 4. Customizing water by controlling its movement in a virtual environment.
13
Process & Methods
Prototyping and testing The project starting with getting the point cloud into a virtual space, then trying to understand the creation of different environments in a virtual space using Unity. To do that I started building distinct landscapes around the scanned site in an attempt to allow the user to encounter different seasons around the same space. This was also a way to understand and work with the technology. After trying to work my way around the technology, I thought it would be better to experiment only with the point cloud and not the environments around it. As the point cloud is just a large set of points in space produced by 3d scanners, getting the point cloud to interact with other objects in Unity was the biggest challenge. The point cloud is primarily supposed to be an aesthetically pleasing experience for the viewer. Hence, there are not a lot of things that can be done to change or edit it. Also, to keep the experience as real as possible
I tried not to add a lot of things to the point cloud. The idea was to use the scan data and manipulate areas around it. However, since the point cloud forms clusters of points at random, the amount of change was limited. To understand the progress made at every step, the testing was done by repeatedly experiencing the virtual space myself. Every test would uncover issues like changes in ground level, placement of background images, the river flow being too fast or too slow and if the experience felt immersive or not. Different elements of the scene were built separately and then integrated together as part of the final scene. As part of testing and building, it was necessary to understand how others felt about the experience as well. This was tested by receiving feedback for changes needed in details. This made it easier to understand how the whole experience was perceived by the user.
Vie wing to Change
14
Fig.07 3D laser scanning Greyfriars Gardens
Fig.09 Greyfriars Gardens in a virtual space
15
Process & Methods
Fig.08 Greyfriars Gardens in a virtual space
Fig.15 Creation of Clouds
Fig.16 Creation of Rain
Fig.17 Virtual Reality Toolkit Setup 1
Fig.18 Virtual Reality Toolkit Setup 2
Fig.19 Virtual Reality Toolkit Setup 3
Fig.20 Virtual Reality Toolkit Setup 4
Fig.10 Greyfriars Gardens in a virtual space
Fig.11 Building landscape 1
Fig.12 Checking for details in virtual space
Fig.13 Building landscape 2
Fig.14 Building landscape 3
Vie wing to Change
16
Process & Methods
Fig.22 Building Staircase 2
Fig.21 Building Staircase 1
Fig.23 Building Staircase 3
Fig.24 Building Inner Walls 1
Fig.25 Building Inner Walls 2
Fig.26 Building Inner Walls 3
Fig.27 Building Inner Walls 4
Fig.28 Setting Up Navigation 1
Fig.29 Setting Up Navigation 2
17
Vie wing to Change
18
19
Process & Methods
Fabrication Techniques a. Real Space Building a 3 m x 3 m wide surface of grass and pebbles with an antistatic black mat beneath it to avoid any spillage which extends its size to 3.65 m x 3.65 m
This limitation of space was done due to the movement restrictions c. Water surface and movement of the motion sensors, which is The water surface available in 4 m diagonally. unity is a 2D surface. The idea was to add water in the scene The black mat was placed on and then raise its level and create the ground directly after cutting an underwater effect beneath the it into required sizes and joining risen surface as the water in unity the pieces using tape. The is not volumetric. artificial grass was then fixed on these black mats using adhesive To customize the water in the and the pebbles were scattered scene I started with an asset around the surface. available in Unity called Water Basic Daytime. This water b. Virtual Space had limitations relating to customization. The next type of The 3D laser scan provided water was Water Advanced Pro with an accurate replica of which had a variety of options for the Greyfriars Gardens which customization which allowed me was bought in Unity. Around to control the colour, the flow and it I added the background by the reflection of the water. This photographing the same as it water texture was then added to is around the site. In addition to a custom mesh created to fit the that, different environmental shape of the river in the space. factors were added. The river
Fig.30 Setting Up Background with photographs
Fig.31 Photographs of background 1
that flows through the centre of the garden space was added as a 2D surface of water texture. The terrain and different ground levels were added to the point cloud to enable movement in the scene.
Fig.32 Photographs of background 2
Fig.35 Surface detail diagram Fig.33 Photographs of background 3
Fig.34 Photographs of background 4
Vie wing to Change
20
21
Process & Methods
Fig.40 Setting up gallery space 1
Fig.41 Setting up gallery space 2
Fig.42 Setting up gallery space 3
Fig.43 Setting up gallery space 4
Fig.44 Setting up gallery space 5
Fig.45 Set up complete in gallery space
Fig.36 Proposal View for Installation
Fig.37 Surface Detail - Artificial Grass
Fig.38 Surface Detail - Anti Static Mat
Fig.37.2 Surface Detail - Pebbles
Fig.39 Exhibition Space
Vie wing to Change
22
23
Process & Methods
Control Systems Unity and HTC Vive:
Fig.46 Water Test 1
Fig.47 Water Test 2
To create a virtual reality experience, I used the software Unity which is essentially used for game design. The software is combined with a steamVR setup which involves a virtual reality headset along with two hand controllers and two base stations which act as motion sensors. The headset controls the direction of vision, the controllers help in the movement inside the virtual space and the base stations/ motion sensors provide with invisible boundaries in the real space restricting movement for the user with the headset on. Sound System: To add the sense of hearing in the scene, I added a sound system which enabled and the headphones play the audio added in the scene for the user. This sound system was created in
connection with the rainfall and lightning in the scene. This audio system also includes the sound of the river flowing. This sound system is connected to the scene and the user can use headphones while wearing the VR headset to enable the sound while walking in the virtual space. This sound is also set up in the real spaced using speaker systems. Water System: The biggest challenge while working with the point cloud was to get the water to interact with it into the scene. This was one by adding water into the scene and then raising its height which then created an underwater effect below the surface of the risen water. The rise in water level works on a trigger that is activated by the user in the scene which will lead to a flood unfolding in front of their eyes.
Fig.48 Water Test 3
Fig.49 Water Test 4
Fig.50 HTC Vive Control System
Fig.51 Audio System Setup
Vie wing to Change
24
Review of Outcomes
Occupation and Interaction The project is a virtual reality installation set up in the Herbert Read Gallery. It also integrates the real space into the virtual space because the user is provided with a surface for him to walk on while wearing the VR headset. The experience created is a first person flood experience elevated using the senses of hearing and touch thus creating a hyper sensory experience for the user.
can be used to increase understanding of the increasing levels of flood risk. To create awareness on the subject it was essential to make the experience as real as possible and this was done by making it a hypersensory experience and using the data from the 3D laser scan. These technological aspects ensured that the user becomes a spatial witness to a flood after entering the virtual space.
The main aim of this project is to find out how virtual experiences Fig.53 Before the flood 1
Fig.52 Setting up terrain mesh from point cloud
Fig.54 After the flood 1
25
Vie wing to Change
26
Section title
Fig.55 Before the flood 2
Fig.57 Before the flood 3
Fig.56 After the flood 2
Fig.58 After the flood 3
27
28
Vie wing to Change
Review of Outcomes
Dissemination and Future Work This project will be published on my personal website along with the research details and also the process of development: https://komalmittal62.wixsite.com/website. The next step is to understand how much a point cloud can be manipulated and test if there are ways of scanning water into a space. The interaction of point cloud with objects can lead to setting up a system where in real time scanned data can be used for any simulation works to enhance visual quality of the project. Also, to find out how hyper sensory effects can be made more powerful and if there are ways in which the involvement of these senses can be heightened to an extent where they would control the entire project. Fig.59 Before the flood 4
Fig.60 After the flood 4
29
30
Fig.61 After the flood 5
Vie wing to Change
31
Appendix
Appendix Annotated Code Excerpts Object to Terrain: using UnityEngine; using UnityEditor; public class Object2Terrain : EditorWindow { dow () {
[MenuItem(“Terrain/Object to Terrain”, false, 2000)] static void OpenWin-
EditorWindow.GetWindow<Object2Terrain>(true); } Down”};
private int resolution = 512; private Vector3 addTerrain; int bottomTopRadioSelected = 0; static string[] bottomTopRadio = new string[] { “Bottom Up”, “Top private float shiftHeight = 0f;
Water Base: using System; using UnityEngine; namespace UnityStandardAssets.Water { public enum WaterQuality { High = 2, Medium = 1, Low = 0, } [ExecuteInEditMode] public class WaterBase : MonoBehaviour { public Material sharedMaterial; public WaterQuality waterQuality = WaterQuality.High; public bool edgeBlend = true; public void UpdateShader() { if (waterQuality > WaterQuality.Medium) { sharedMaterial.shader.maximumLOD = 501; }
Vie wing to Change
32
Materials and Suppliers List Artificial Grass 3 m x 3 m (from B&M Stores, Canterbury) Pebbles and Stones 1 kg (from Homebase, Canterbury) Anti-Static Black Mat 3.65 m x 3.65 m (from UCA Equipment Store) Timber 250 mm x 1000 mm x 1200 mm (from UCA Wood Workshop) HTC Vive VR Set 1 (from UCA Equipment Store) Speaker Set 1 (from UCA Equipment Store) TV Screen 96 cm x 60 cm (UCA Equipment Store) Other sundries: River Material Asset (from Unity Asset Store) Ambient Sound Pack Asset (from Unity Asset Store) Particle System Effect Asset (from Unity Asset Store)
33
Appendix
Bibliography About | ScanLAB Projects (s.d.) At: https://scanlabprojects.co.uk/ about/ Cazenave, A. and Nerem, R.S. (2004) ‘Present‐day sea level change: Observations and causes’ In: Reviews of Geophysics 42 (3) [online] At: https://agupubs.onlinelibrary.wiley.com/doi/ abs/10.1029/2003RG000139 Farra, S. et al. (2013) ‘Improved Training for Disasters Using 3-D Virtual Reality Simulation’ In: Western Journal of Nursing Research 35 (5) pp.655–671. IMMERSED: A VR Experience About Flood & Resilience | FEMA.gov (s.d.) At: https://www.fema.gov/immersed Klein, N. (2014) This changes everything: capitalism vs. the climate. (s.l.): (s.n.). Watson, D. and Adams, M. (2011) Design for flooding: architecture, landscape, and urban design for resilience to flooding and climate change. Hoboken, N.J: John Wiley & Sons.
USB 2.0 A-Male to A-Female Extension Cable 3 m/9.8 Feet 2 (from Amazon) High-Speed HDMI 2.0 Cable - 3 m/10 Feet 1 (from Amazon)
Image Credits All figures are copyright the author unless noted as follows: Figure 05: Venice – ScanLAB Projects (2017) [Poster] At: https://scanlabprojects.co.uk/work/italys-invisible-cities/ Figure 06: A still from Immersed (2018) [Poster] At: https://www.fema. gov/immersed
Vie wing to Change
34
Credits MA Interior Design Course Leader: Lucy Alice Jones MA Interior Design Tutors: Lucy Alice Jones Owian Caruana Davies Special Thanks: JJ Brophy Anna Holder Kim Trogal Chris Settle
Section title
35
36
Vie wing to Change