INSTITUTE FOR ADVANCED ARCHITECTURE OF CATALONIA IAAC
MASTERS IN ADVANCED ARCHITECTURE 2018 STUDIO STUDENTS
LEAD FACULTY FACULTY ASSISTANT COMPUTATIONAL ADVISOR
DIGITAL MATTER AND INTELLIGENT CONSTRUCTION NIKOL KIROVA | HAYDER MAHDI | SHRUTI JALODIA | EWALD JOOSTE
ARETI MARKOPOULOU DAVID ANDRÉS LEÓN, RAIMUND KRENMÜLLER ANGELOS CHRONIS
SYNAPSE An interactive medium for spatiotemporal behavioral Data collection and Analysis
CONTENTS 01 Author’s note
1
02 Introduction
3/7
| Research Topic | Abstract
03 Research | Current state of art
| Graphene
| Graphene Inks
| Substrates
| Patterns
04 System
| System
8 / 23
24 / 47
| Input
| Capacitive Sensing | Heating system | Footstep Tracing | Continous Undermat | Modular system
05 Prototypes | Matrix prototypes | Gesture recognition | Localized heating with gesture activation | Tile system | Data gathering prototype
48 /77
06 Applications | Architectural | Urban 07 Case study | Superilla 08 Sensor data analysis | Dynamic space configuration
86 / 91
92 / 99
1
AUTHOR’S NOTE Architecture is morphing, as its becoming more and more data informed. Sunlight, Radiation, wind, energy efficiency, water and material efficiency - almost every aspect of building fitness is quantifiable. In fact, not just architecture, but every aspect of our lives today is quantifiable‌ calories, miles, health, online behavior. All this data , combined with increasing power of computing in the age is algorithms and machine learning, is enabling a seamless user experience. Our built environment is no longer isolated to this experience. On one hand, solutions to make building more intuitive and responsive to the users are being developed, and on the other, ideas like open Data - where multiple agents in a city can be analyzed together, are enabling us to understand our cities better. With this paradigm shift, we are also faced with unprecedented problems. such as personal privacy for both physical and virtual identities. However, development of extraordinary materials, technologies and evidence based -data driven processes - allows us a unique opportunity to overcome these issues and develop systems which are powerful, ethical and intuitive. As we developed our project in this context, it became clear to us that there are two main focuses that we are dealing with - Active surfaces and Adaptive spatiotemporal strategies. And that they are not only intrinsically connected to each other, but together, provide a novel and powerful opportunity for architecture and urban design.
2
introduction
4
5
RESEARCH TOPIC How can we develope an interactive medium for spatiotemporal control and analysis, which allows an enhanced architectural performance and user experience ?
6
Architecture has always been in the process of transformation, responding to the technological, social and cultural changes. In recent years, acceleration in digital media technologies has had a very significant impact on multiple facets of architecture ranging from design and modeling, communication, fabrication, and automation in buildings. This transformation has allowed architects to approach design in a data-centric and algorithemic logic. However, means of reliable data collection are restricted to devices and sensor which are not an intrinsic part of architectural materiality. This research aims at developing SYNAPSE, a system which can be a constitutional part of a variety of architecture interfaces like floors, walls, facade systems and even furniture. This capacitive sensing based system is adaptable to different scales, shapes as well as materials and is programmable to suit different applications and scenarios. Graphene, a carbon-based nanomaterial, is being used as a key constituent of the system, for its extreme conductive properties. This paper first provides a detailed description of Synapse system composition followed by a description of each element that constitutes the system and various factors that affect its performance.Also included are the detail on ollected data analysis and user interaction methods using the system. The paper closes with an examination of benefit synapse could provide in enhancing architectural performance and user experience.
7
ABSTRACT The absence of accurate, real-time information about individual’s spatial behavior leaves designers and decision makers without a real space performance metric and may consequently lead to poor judgment and speculations. This work-in-progress research and experimentations aim to provide designers and decision makers with an information gathering tool for users’ spatial behavioral patterns towards a better evidence-based, contextual, data-driven process and behavioral research. Using surface embedded Graphene Nanoplatelets, capacitive sensing system along with machine learning processes, this research presents an analytical and interactive medium for user’s presence sensing, position tracing, and gesture recognition. Moreover, we present several cases in which this system acts as an actuator when paired with various active layers for applications such as localized heating and haptic feedback.
8
research
10
11
CURRENT STATE OF ART The principal application of the SYNAPSE system is a sensory interface for architectural elements. The sensing mechanism has been developed as a construction of emergent graphene technology. As a system, this material together with its application are without precedence in architecture and related industries. Simple sensing mechanisms take a passive binary form with the sole function of triggering a signal in the presence of an object, such as those implemented in Passive Infrared sensors, light barriers and pressure mats. In the case of capacitive sensing, recent developments have demonstrated more advanced architectural applications with the capacity for:
| multiple presence sensing modalities | position tracing, tagging, and gesture recognition | activation of responsive layers | temporisation data processing and accompanying Machine Learning
12
SensFloor®
Wall++
SPACIAL ANALYSIS
CONTEXT - AWARE
13
IKEA Smart Kitchen Table
Local Warming MIT SENSEable City Lab
AUGMENTED REALITY
LOCAL RESPONSE
14
STRONG
LIGHT
HIGH BOILING TEMP
CHEMICALLY INERT
TRANSPARENT
BRITTLE
HEAT CONDUCTIVITY
ELECTRICAL CONDUCTIVITY
PROGRAMABLE
EASY MATERIAL INTEGRATION
Figure 01: Properties of graphene
15
GRAPHENE Since graphene was isolated and characterised in 2004, intensive research efforts have been dedicated to pairing it’s unparalleled material properties with novel architectural applications. Recent success in production of graphene using existing platforms of advanced manufacturing provide access for architectural-scale testing and development. However, to fully leverage the advantages of graphene during this process, a threshold must be crossed from handling the material at the level of particle physics to that of building fabric. The development of graphene as a digital matter medium in design and fabrication systems common to architecture could progress the integration of information technologies. These have the potential to shape not only interaction between elements of the built environment, but interaction between building elements and their inhabitants. Some of the beneficial properties of graphene are represented in figure01. In nanoplatelet form,which is used for this research, graphene presents scope for development of surface-embedded patterning, with low barriers to entry in cost and open-source access.
16
Figure 02: System structural and functional layers
17
GRAPHENE INKS Custom-Made conductive ink is used to deposit electrodes on a given substrate. Graphene is the critical ingredient, and being hydrophobic, it needs a binder to obtain a uniform dispersion in water. As a first stage of investigation we got familiar with graphene inks. A research into the variety of binders and mix proportions took place (figure2). The Ratio of the binder to graphene nanoplatelets is the decisive factor in achieving a required conductivity. We can program the mix such that it provides a high conductivity while using for electrodes and slightly lowering conductivity and increasing the resistivity to an optimal level to be used for graphene heating pads. The final ink mixture and process of fabrication was perfected before we went into further investigation (figure3 & figure 4).
18
GRAPHENE
Figure 03: Graphene ink | ingrediants
WATER
BINDER
19
Figure 04: Graphene ink | making
20
Methyl Methacrylate
Glass
PETG
Figure 05: Substrates
Figure 06: Graphene resistance depending on substrates
Dotted Plyethylene
Cork
21
SUBSTRATES The sensing matrix is supported by a substrate that forms the base layer of each system construction. Over the evolution of the design process a series of materials were trialled for compatibility with conductive ink. Evaluation of results eliminated a range of typical architectural materials from practical application. The remainder provided a matrix of prospective materials for further investigation. Performance of sensing system is dependent on substrate. The key indicator of performance was resistance measured for an applied current over a controlled surface area and lineal distance. Sampled materials and significant results are provided in Figures 05 & 06
22
Figure 07: Pattern exploration on methacrylic plates
23
PATTERNS Ink line widths and geometrical patterns had an impact on ink’s conductivity. We tested different widths using the same ink and width of the electrode is directly proportional to its surface resistance. 30mm wide lines yielded resistance values that are optimal for this research. Another test was conducted using different geometrical patterns of the same ink surface area. Different patterns had different sensing range due to their distinctive projected electrical fields (Figure 7). Until a detailed study of the different electrical fields is conducted, we are implementing a grid of strips as a pattern for the different prototypes. For this test, we used self-capacitance system and we have planned to use mutual capacitance system to test different patterns in the near future. We also observed that a pattern with non-uniform width is ineffective, as the conductivity between line endpoints is governed (among other parameters) by the minimum width across the section.
24
system
25
26
Figure 08: System structural and functional layers
27
SYSTEM The assembly is a composite of sensing and active layers (figure 08). The sensing layers consist of a matrix composed of X and Y electrodes, together acting as location sensor to taglessly locate a person coming in proximity with the matrix. The system utilizes the principle of mutual capacitive sensing, that is widely used in touchscreen technologies and can be embedded in various surfaces ranging from walls, facade, furniture, floor and even warped on curved surface. The sensing system can be coupled with a variety of active layers that can gain from presence sensing, and that does not interfere with the coupled capacitance of the electrodes. The active layers can vary from heating, shading with thermochromic materials, inductive charging and haptic feedback.
28
Position
Pressure passive active
[]
System Gesture
Figure 09: System performance
{
Performance Control
Spatial Observations
29
INPUT This system allows us to accurately determine the position or proximity of an object on the floor, at any given time. Reading position as a passive input from the user, we can determine the direction of motion of the user or any conductive object, their size, speed, and quantity. Using the same sensor, we are also use it in gait analysis and potentially for user identification applications. Using the same assembly, we are also able to program it to recognize user gestures, as active inputs, which can be used for feedback from the user or to actuate a specific action. The design of the capacitive grid, including the size and spacing of the electrodes, has an impact on its sensing capabilities. Computational and Machine Learning methods can be employed to derive the grid design, as described later in chapter 08.
30
31
CAPACITIVE SENSING Capacitive sensing is a proximity sensing technology. Capacitive sensors work by generating an electric field, and detecting nearby objects by sensing whether this field has been disrupted. Capacitive sensors can detect anything that is conductive or that has a significantly different permittivity than air, like a human body or hand. The superior conductive properties of graphene leverage itself effectively to this cause. We investigated a range of patterns and how they refer to the distance that can be achieved with capacitive proximity sensors (figure 10). Later on we cheched the relation between the type of ink with the distance (figure 11);
32
30mm
Figure 10: Capacitive sensors | pattern
60mm
33
80mm
110mm
34
Figure 11: Capacitive sensors | ink
35
36
Figure 12: Heating range
Figure 13: Resistance vs. Temperature
37
HEATING SYSTEM As building heating accounts for a substantial amount of energy, there lies a great potential to save energy by using dynamically controlled, localized heating (Ratti and Claudel 2015). We tested the use of Graphene Nanoplatelets as heating pads to be used in conjunction with the sensory surface to provide localized heating. By changing the mix of graphene ink, we can achieve an ink mixture that can be rapidly heated to very high temperatures with relatively low voltage (Figure 12). We also observed that as the temperature goes higher, the resistance goes lower. (Figure 13). Another test was conducted in which presence sensing was used to actuate different heating pads. This way, only pads with great proximity to the user are activated (Figure 14).
38
Figure 14: Programable heating
39
40
Figure 15: Detection of different phases of a gait cycle through 75mm thick concrete.
Figure 16: Capacitive sensing through concrete
41
FOOTSTEP TRACING By embedding graphene electrodes in concrete tiles and using mutual capacitive sensing, the system was able to detect a footstep through 75mm thick concrete substrate through shoes (figure 15). In a comparision between 45 mm tile and 70mm tiles, there was negligible different in sensing performance. (figure 16) Further, by using a high-resolution grid and analyzing the non-concurrent incoming signals, we are able to distinguish between different phases of a walk cycle. This feature is used to determine the walking direction as well as the stance phase speed. One of the methods we use for analysing this behavior is dynamic time warping. By representing the capacitive effects as a wave we concluded that this method is beneficial for concluding a direction and pace of pedestrians. (figure 17)
42
Figure 17: Analyzing non-concurrent signals to determine walking direction.
43
Figure 18: A large sensory surface
44
Figure 19: Ink deposition on continuous rolls
Figure 20: Sensor sheets embeded in floor and wall
45
CONTINUOUS UNDERMAT The ‘sensing’ assembly can be embedded in the floor or wall surfaces, either under existing finish or in the form of custom made ‘active tiles’. Both methods can be manufactured on a large scale. This research proposes a system that includes two perpendicular electrodes, on either side of a PETG sheet, followed by a protective layer on both the sides. The sheets can be used in the form of a continuous roll (figure19) that can be embedded on a variety of buildups (figure 20).
46
Figure 21: Capacitive sensing electrodes in modular tiles.
Figure 22: Modular system assembly
47
MODULAR SYSTEM Alternatively, graphene electrodes can be miniaturized to be embedded in modular systems such as concrete or wood tiles (Figure 21 & 22).
48
prototypes
49
50
51
CAPACITIVE SENSOR MATRIX Two main focuses of our experimentation and prototyping were, firstly to test, understand and work with a large scale capacitve sensor and secondly, to create large scale capacitive sensor using Graphene conductive ink. In the early stages of the reserch, a copper based capacitive sensor was developed which allowed us to developed computational tools for location sensing (figure 23). This prototype was soon replaced with graphene based sensors which not only allowed location sensing but also pressure sensing, (figure 24)
52
Figure 23: Lacalization on copper matrix
53
54
Figure 24: Pressure and localization on graphene matrix
55
56
Figure 25: Trained gestures for recognition
57
GESTURE RECOGNITION Hand gestures provide natural and intuitive computer-human interface that eliminates the need for switches. Capitalizing on Mutual Capacitance, we examined the use of gestures as an input mechanism. We used an open source, crossplatform, machine learning library for real-time gesture recognition. We utilized the Gesture Recognition Toolkit [Nick Gillian] (figure 26) along with the Example-based Sensor Prediction system [Mellis] to train a set of 8 different hand gestures. (figure 25) We used the Dynamic Time Warping (DTW) algorithm to classify the temporal signals we receive from the sensors. DTW algorithm creates different time series templates for different gestures, then warps the real-time sensor signals to find the best match [N Gillian 2]. (figure 27)After preparing sensor calibration samples, we calibrate the real-time incoming signal and fine tune them as required. We then preprocess incoming signal by using the Moving Average Filter module to remove some amount of high-frequency noise from our 16-dimensional signal. We then compute the first derivative of the signal using the Derivative pre-processing module. For feature extraction, we use the Time Domain Features method to get the signal amplitude with time.
58
Figure 26: Dynamic time warping signals
59
60 200
200
200
100
100
100
50
50
50
0
0
100
200
0
0
100
200
0
200
200
200
100
100
100
50
50
50
0
0
100
200
0
0
100
200
0
200
200
200
100
100
100
50
50
50
0
0
100
200
0
0
Figure 27: Different time series templates for different gestures
100
200
0
0
1
0
1
0
1
61 200
200
100
100
50
50
0
0
100
200
0
200
200
100
100
50
50
0
0
100
200
0
200
200
100
100
50
50
0
0
100
200
0
0
100
200
0
100
200
0
100
200
62
Figure 28: Prototype exploded
63
PROTOTYPE Nยบ1 This prototype was developed at the culmination of first out of two semesters devoted to this research. We are testing sensing, gesture activated control and heating properties of our system. The prototype consists of three layers, two sensing layers topped with a active localised heating layer. The substrate used is 3mm Poly(methyl methacrylate). We trained the system for localized heating activation. When a user circles a specific area on the prototype, graphene heating pads in that area get activated. The system can be further developed to allow for automatic localized heating based on user presence in space.
64
Figure 29: Prototype
65
Figure 30: Prototype | heating | Gesture input
66
67
PROTOTYPE Nยบ2 Active Urban tiles are developed as a replacement of existing, non sensing Urban tiles. These tiles could provide accurate, anonyms, large-scale spatiotemporal information that puts the citizen in the urban design feedback loop. These tiles are developed keeping in mind high traffic movement in urban areas and required material strength. These tiles will not only allow sensing capabilities, but will also allow easy and accurate detection of cracks or deformation in urban space, leading to effective maintainence.
68
Figure 31: Concrete tile fabrication
69
70
71
PROTOTYPE Nยบ3 This large scale walkable prototype is idea as an undermat which can be paired with any of the existing flooring or wall materials. The design of the elctrodes is such that it allows for maximum sensing area and hence a higher resolution of data. With this sample, we are able to collect realtime data of pedestrain tracking, space occupancy and walking direction. The same system can be easily trained to recognize gestures as well. A 0.5 mm thick flexible PETG sheet is used as substrate and the fabrication involved stenciling, ink deposition and lamination. Connection between graphene electrodes and end cables proved to tricky and a conductive tape is used for the same.
72
Figure 32: Stenciling on PETG
73
74
Figure 33: Ink deposition on PETG
75
76
Figure 34: Data analysis.
77
DATA GATHERING Data collected from the large sample is used to generate occupancy maps, which allow users and decision maker to read interpret this data with ease. Occupancy maps in figure 43 show two information sets : 1) Activation - number of pedestrains passing through a specific area is denoted by height of the mesh. Heighst peak stands for the most visited space. 2) engagemnet - the colors denote user engagement of the space i.e. time spent in a specific space. Warm colors represent more engagemnet. Activation and engagement, together give a holistic index of occupancy in a space.
78
applications
80
Figure 35: Public Space
81
Architectural In indoor spaces, active sensory surfaces help predicting spatial performance through diverse behavioral data of similar spaces. Moreover, real-time data can enhance the performance and user experience. By having clear and accurate understanding of the spatial user behavior, a space can consequently be adapted according to the specific needs of the inhabitant. Dynamic space configuration can be achieved as a result of behavioral analysis. By understanding how people use spaces in an unbiased way, evidence-based conclusions can be drawn and spaces can adapt better to the different user’s spatial requirements. Some of the spaces where such system can be of great benefit to are public and civic buildings as well as working areas such as hospitals, exhibition spaces, theatres, educational institutions as well as retail venues.
82
Figure 36 : Residential
83
Figure 37 : Theatre
84
Figure 41: Vision for urban system application 1) Footfall Count 2) Emergency route 3) Unusual activity moniter
85
URBAN VISION Synapse could provide accurate, anonyms, large-scale spatiotemporal information that puts the citizen in the urban design feedback loop. By recording traces of footsteps, the system can provide dynamic occupancy maps, trend prediction, and historical pedestrian information. Such information provides relevant authorities in urban areas with accurate, real-time metrics of user engagement, public spaces performance, put strategies for site and crowd management, perform risk evaluation, predict maintenance works, and deploy sufficient emergency services. By analyzing public pedestrian traffic data, Synapse’s density maps could illustrate multi-layer information:
| The occupancy rate at different time periods | Most and least occupied areas | Pedestrian traffic bottlenecks | Crowd gathering and dispersion routes
Moreover, by performing microanalysis on the electrodelevel data, we can obtain information about:
| Pedestrian engagement rate with crowd attractors | Walking patterns: gait and direction | Pedestrian type: group or individual | Age group analysis: based on pace & step size | Walking paths
86
superilla case study
88
Figure 38: Superblock of Poblenou selected for on-site investigation.
89
ON-SITE INVESTIGATION The Superblock of Poblenou is a pilot project where pedestrain and vehicular spaces are physically redifeined to included Public space, play zones, sport zones and urban renewal locations. The goal of this onsite testing was to investigte whether these redefined spaces are being used as expected by the urban designers. This on-site testing of prototype was performed at one of streets where the former vehicular area has been reassigned to pedestrians. (figure 38) The findings showcase that the policy change has not reflected in the desired outcome as pedestrians still prefer to walk on the side pavement rather than in the center. (figure 39).
90
Figure 39: Selected streets for investigation
91
Figure 40: Collected date on site
92
COMPUTATION | DATA ANALYSIS
94
95
DYNAMIC SPACE CONFIGURATION We have developed a spatiotemporal localization sensor, with which we can map occupancy and behavioral patterns of inhabitants in a space. The occupancy pattern, when overlapped with other spatial information such as Sunlight, Temperature and visual connectivity, can give a cohesive and holistic spatial performance overview. Based on this information many spatial issues can be revealed like pedestrian bottlenecks, spatial congestion, ignored/ disdained spaces as well as accident-prone areas. One of the solutions to some of these problems is ‘dynamic space configuration’; where the spatial arrangement of any item in a space is computationally decided, based on the aforementioned parameters. For this computational exercise, the factors taken into considerations are: sunlight, visual connections and frequent pedestrian paths.
96
97
98
99
100
Refrences Cooper, Daniel R., Benjamin D’Anjou, Nageswara Ghattamaneni, Benjamin Harack, Michael Hilke, Alexandre Horth, Norberto Majlis, et al. 2011. “Experimental Review of Graphene” 2012. https://doi. org/10.5402/2012/501686. Hong, Seung Wan, Yehuda E. Kalay, and Davide Schaumann. “The effects of human behavior simulation on architectural design education.” (2015). Gong, Nan-Wei, Steve Hodges, and Joseph A. Paradiso. 2011. “Leveraging Conductive Inkjet Technology to Build a Scalable and Versatile Surface for Ubiquitous Sensing.” Proceedings of the 13th International Conference on Ubiquitous Computing - UbiComp ’11, 45. https://doi.org/10.1145/2030112.2030120. Gillian, Nicholas, and Joseph a Paradiso. 2014. “The Gesture Recognition Toolkit.” Journal of Machine Learning Research 15:3483–87. https://doi.org/10.13140/2.1.4216.2886. Gillian, Nicholas, R Benjamin Knapp, and Sile O ’modhrain. 2011. “Recognition Of Multivariate Temporal Musical Gestures Using N-Dimensional Dynamic Time Warping.” Nime, 337–42. Mellis, David A, Ben Zhang, and Audrey Leung. 2017. “Machine Learning for Makers : Interactive Sensor Data Classification Based on Augmented Code Examples.” Designing Interactive Systems2:1213–25. https://doi.org/10.1145/3064663.3064735. Ramezani Akhmareh, Alireza; Lazarescu, Mihai T.; Bin Tariq, Osama; Lavagno, Luciano. 2016. “A Tagless Indoor Localization System Based on Capacitive Sensing Technology.” Sensors 16, no. 9: 1448. Ratti, C., Claudel, M. (2015) . “Local Warming”. ARQ, 89, 42-49. Shu Wan,Hengchang Bi,Yilong Zhou,Xiao Xie,Shi Su,Kuibo Yin,Litao Sun, 2017 “Graphene oxide as highperformance dielectric materials for capacitive pressure sensors” https://doi.org/10.1016/j.carbon.2016.12.023 Yang Zhang, Chouchang(Jack) Yang, Scott E. Hudson, Chris Harrison, Alanson Sample (CHI 2018) “Wall++: Room-Scale Interactive and Context-Aware Sensing” Hiroshi Ishii, 2008, “Tangible Bits: Beyond Pixels” Danil Nagy, Damon Lau, John Locke, Jim Stoddart, Lorenzo Villaggi, Ray Wang, Dale Zhao and David Benjamin, “Project Discover: An Application of Generative Design for Architectural Space Planning”
101
INSTITUTE FOR ADVANCED ARCHITECTURE OF CATALONIA IAAC
MASTERS IN ADVANCED ARCHITECTURE 2018 STUDIO STUDENTS
DIGITAL MATTER AND INTELLIGENT CONSTRUCTION
NIKOL KIROVA, HAYDER MAHDI, SHRUTI JALODIA, EWALD JOOSTE
LEAD FACULTY
ARETI MARKOPOULOU FACULTY ASSISTANT COMPUTATIONAL ADVISOR
DAVID ANDRÉS LEÓN, RAIMUND KRENMÜLLER ANGELOS CHRONIS