Living Gesture

LAB 1244 Living Gesture Arina Paula Cazan

Ein Entwurf im Wintersemester 2018/19 am:

Institut für Leichtbau Entwerfen und Konstruieren Universität Stuttgart Prof. Dr.-Ing. Dr.-Ing. E.h. Dr. h.c. Werner Sobek Prof. Dr.-Ing. Balthasar Novák Projektleitung: Dipl.-Ing. Architektin und Stadtplanerin Irina Auernhammer Dr.-Ing. Nicu Toader Dipl.-Ing. Christian Kelleter Fotos: © Arina Paula Cazan Satz und Layout: Arina Paula Cazan Kooperation mit:

2

Living Gesture Content 04..................................

06..................................

08..................................

10..................................

12..................................

16..................................

22..................................

24..................................

Introduction Communication Relationship The UNIT Actuators Sensor & Gestures Softness List of Figures and Literature

3

4

Living Gesture [new forms of communicating and creatively interacting with the built environment of tomorrow] Imagine an interactive and adaptive built environment. What would that be and feel like and most importantly how could we learn to communicate with it in an effective and satisfying way? The concept of communication referred to in this case will be further described later on.

The aim of this project is to explore new possibilities of communication and creativity that arise out of this interaction and at the same time to make a rich, easily understood, aesthetic and most of all satisfying interaction experience possible.

Living Gesture is a project involving an activated textile that responds to human gestures.

This very basic and intuitive form of non-verbal communication

through hand gestures is now being imprinted into a new medium, prolonging its effect.

Encoding information into materiality and thus giving ‚life‘ to the not before long static artificial world [1] around us can be quite exciting.

At the same time, it is crucial to question our motives for wanting to create and control an interactive surrounding and ask ourselves if this is something that we will really want or need in the future.

Why would a human show any interest in interacting with something he created for – let‘s assume – very practical reasons?

Do we wish to be able to give commands more efficiently and expect our built environment to execute them or are we instead

looking for a deeper meaning and genuine ways to express ourselves and experience new forms of creativity?

This project aims particularly at enabling us to dig deeper into our understanding and perception of the man-made world of tomorrow and at offering a testing field for all of these questions with the help of an experimental unit.

In the end, this should give every individual the chance to imprint a unique pattern into its surroundings while enjoying the effect his gestures generate.

1 Composition of gestures activating a textile 5

6

Communication According to the Merriam Webster Dictionary, communication can be defined as „a process by which information is exchanged between individuals through a common system of symbols, signs, or behavior“. For this project, the definition of this process has been slightly adapted to incorporate the UNIT as a participant.

The communication system tested in this case is rather simple and comprises a human/ several humans who generate and encode data through a non-verbal form of communication – hand gestures – that is picked up by the gesture sensor and transmitted to the Arduino microcontroller as digital input. This data is then being decoded and translated

into the activation of several individual or series of actuators connected to the back of the fabric that in turn generate the movement of the textile. The interpretation of the data transmitted by the hand gesture is now visible in the foldings of the fabric. When the human realizes the effect of his gestures and reacts accordingly, his reaction closes the cycle of communication.

At this stage, I would like to mention that this is by no means meant to replace the more traditional forms of interaction/conversation between people and should have no direct impact on the relationship between human beings.

2 Hand gesture signaling a movement upwards 7

8

Relationship The relationship we are used to having with our built surroundings is very likely to change in the next few decades. As technology and particularly the development of artificial intelligence – AI – forms evolve, it is unlikely that these changes will evade the building industry for much longer.

Two major types of relationships can be formulated when it comes to humans and the experimental unit – stands for the built world.

The first one is something we are very familiar with: human-human relationship with the unit being used as a mere tool in order to facilitate communication and interaction between people. The second type would be the human-unit relationship and would

emphasize the unit as a subject of interest. During this project, we will look into the second type, the human-unit relationship.

So how could that work? When we think about communication we automatically think about response, so what are the possible responses the unit could formulate? The simplest response will always be the most predictable one. The textile could mimic the presence of the human that is interacting with it and generate a type of ‚mirror effect‘ or simply execute a given command like a movement to the left or to the right for example. The opposite of that would be the textile being able to process the information it registers from the human gesture

and generate an unpredictable movement which would require some form of AI in order to be original and truly unexpected. As AI research is not fully developed yet, it is hard to imagine this as a viable option at the moment except as a nice prospect for the future. Between these extremes there lies a 3rd option: a response that starts out as being predictable and through the combination of different gestures and the help of the parameter ‚time‘ develops into something more, an original combination of movements that each person interacting with the unit can influence and personalize.

3 Hand gesture signaling a movement to the right 9

10

The UNIT The unit can be seen as the visual component of the interaction, an entity that is capable of registering hand gestures that take place in its immediate proximity and formulate a response. What exactly this response could be and whether the unit could truly become a participant in the communication are two of the main topics of this work.

One possible materialization of the concept is the activation of a light-weight textile with the use of shape memory alloys (SMA).

Movement in the proximity of the unit is being sensed by the integrated sensor, interpreted by a microcontroller and transformed into electric stimuli that set the actuators – NiTiCu wires – in mo-

tion.

The main components of the unit comprise a 60×60 cm satin fabric – 97% Polyester, 3% Elastane; 98g/m2 –, a T-shaped aluminium frame to hold the fabric in a vertical position, a Plexiglas base, a pattern of SMA wires connected to the back of the fabric in an arrangement based on a 5×5 cm grid, conductive thread plus copper wires linking the actuators together, a gesture sensor – APDS 9960 –, Arduino UNO microcontroller, 6 MOSFET transistors, a breadboard and a DC power supply.

4 Electrical and digital components 11

12

Actuators While researching different types of conventional and unconventional actuators, the benefits of Shape Memory Materials (SMM) and in particular of Shape Memory Alloys like Nitinol played an important role. Lightweight, compactness, high power to weight ration and low operation voltage are among the advantages of this type of actuator [2].

NiTi is a metal alloy of Nickel – Ni~ 55% – and Titanium – Ti~45%.

NOL stands for Naval Ordnance Laboratory, where it was discovered in 1962 [3]. SMAs react by changing into a prescribed shape – in which they were annealed at approx.500 °C – starting at a certain temperature – austenite start temperature As – with the transformation being completed at the

austenite finish temperature (Af).

The ability to ‚memorize‘ a certain shape is due to nitinol special crystalline structure with its 2 main phases: austenite and martensite + the annealing phase [1] [4]. When the metal is heated up, it tends to change into the previously imprinted shape and when it cools down it slowly relaxes.

This reversible transformation between austenite – high temp. – and martensite – low temp. – generates the shape memory effect.

For this project, I have tested out two different thicknesses of NiTiCu – Nickel + Titanium + Copper – wires: 0,1 mm and 0,25 mm.

Due to a small proportion of 5% nickel being replaced by copper, these wires have a lower transition temp. and are therefore more

efficient than normal NiTi wires.

The austenite finish temperature for both wires is set at 60 °C making them mid-temperature range nitinols and thus easy to work with in combination with textiles.

While the thinner wire turned out to be good for lifting very light fabrics, the advantages of the thicker one – 0,25 mm – proved to be significant especially in terms of heating pull force and reliability.

• ø= 0,25 mm Af= 60 °C (mid-temp. range) R= 18,5 Ω/m I= 1050 mA (1s contraction) Fheat= 891 g Fcool= 356 g [5] 5 NiTiCu 0,25 mm wire heated into shape 13

14

In order to heat up the wires and control the temperature, electricity is being used. According to the thickness and specifications of the SMA actuators, the necessary amount of current that needs to pass through them has been calculated in order to not overheat the wires or damage the fabric.

By doing this one can control the speed and the duration of the activation and therefore the movement of the textile.

actuator main grid secondary grid fabric edge

For the activation of the SMA wires, an extra power supply is needed to generate the necessary voltage of 5-12/15 V. The Arduino board itself is not powerful enough for this task and requires a series of MOSFET transistors that act as switches in order to

control the flow of current from the power supply – only through the actuators and not the microcontroller. 6 transistors were used for the individual control of 6 series of NiTiCu wires that determine the possible movements.

Depending on how the wires are arranged, different patterns can be achieved. The pattern of actuators used for this project is illustrated in the pictogram. Due to multiple intersections, special care needed to be taken in order to isolate the wires and to facilitate the necessary freedom of motion.

6 Back side of the fabric. Active + passive wires 15

16

Sensor and gestures At first, a basic PIR motion sensor was used in order to begin controlling the activation of the SMA wires according to nearby movement. Due to the wide range of this sensor of ~6 m, the necessity for a more precise device soon became clear. The APDS9960 RGB and gesture sensor – compatible with Arduino – offers touchless gesture sensing and makes controlling the fabric with a simple swipe of the hand possible. The detection range of the 6 possible gestures that can be perceived, out of which 4 were assigned commands in this case, varies between 10 and 20 cm [6]. actuator main grid secondary grid fabric edge

A swipe up, left or right generates the basis of the interaction and constitutes the predictable type

of response mentioned earlier.

Only when the user starts composing the gestures, a unique response can be formulated. Due to the slower deactivation of the NiTiCu wires, overlapping movements are possible.

The parameter TIME plays an essential role in achieving this. By programming the activation period, the duration between activations and the duration of the overlap, one can control the motion effect. The intensity of the current is the parameter that influences the speed of the activation, between predefined boundaries.

7 LEFT gesture 17

8 RIGHT gesture 18

actuator main grid secondary grid fabric edge

actuator main grid secondary grid fabric edge

9 UP gesture 19

20

The near gesture is also part of a more complex response that requires 3 individual series of actuators that transmit the motion from one to the other in a wavelike manner. The combination with the 3 basic gestures is possible as well.

One pair of actuators begins to fold the fabric and from that starting point, a ripple effect is created that transmits the movement forward to the next series of actuator-pairs and so on, until the motion is complete. While one actuator pair is being activated, other pairs start deactivating, making a smooth transition possible. actuator main grid secondary grid fabric edge

As the artist Julien PrÊvieux noticed: „Gestures used to be fleeting and to not leave any evidence

that they ever happened. Which meant that if a gesture was made and either no response or the wrong response was received, there was little information available to help understand why“ [7].

In this case, even after the momentary hand gesturing has passed, the textile keeps performing for a little while longer, enabling the user to stand back and experience and analyse the response.

10 NEAR gesture 21

22

Softness By moving and changing shape according to human input, the unit actually connects physically with the user in a social manner.

The final form is no longer designed, an initial state is defined, a set of rules is introduced and then the form is being created in real time [8] through the activation of the fabric.

The textile itself becomes a soft 3D user interface, allowing a ‚no touch‘ interaction and thus becoming the link between software, hardware, and humans [9].

According to Hiroshi Ishii, director of the Tangible Media Group of MIT, it is necessary to think “beyond the screen” and find more efficient and direct ways to let people interact with technology [1].

As an experiment, Living Gesture

offers the possibility of exploring future interactions via a present experience. The softness of the satin invites the potential user to approach the unit and take a closer look at it. Once the person gets nearer and starts reaching with its hand towards the fabric, first contact has been initiated.

The ripple effect created could have a startling effect on the user/users but once he/she gets used to the idea, an interaction can take place and one begins to play with the possible gestures and responses.

Thus, it can be said that the project explores territories beyond functionality and encourages selfquestioning and expression.

11 Ripple effect in motion 23

24

List of Figures 1: Composition of gestures activating a textile © Arina Paula Cazan 2: Hand gesture signaling a movement upwards © Arina Paula Cazan 3: Hand gesture signaling a movement to the right © Arina Paula Cazan 4: Electrical and digital components © Arina Paula Cazan 5: NiTiCu 0,25 mm wire heated into shape © Arina Paula Cazan 6: Back side of the fabric. Active + passive wires © Arina Paula Cazan 7: LEFT gesture © Arina Paula Cazan 8: Right gesture © Arina Paula Cazan 9: UP gesture © Arina Paula Cazan

10: NEAR gesture © Arina Paula Cazan 11: Ripple effect in motion © Arina Paula Cazan P. 24: Photo collage © Arina Paula Cazan P. 26-27: Code © Arina Paula Cazan Cover/back cover: Textile in motion. Ripple effect © Arina Paula Cazan Literature References [1] M. Bengisu und M. Ferrara, Materials that move, Smart Materials, Intelligent Design, New York, NY: Springer Berlin Heidelberg, 2018.

[2] M. Mehrpouya und H. C. Bidsorkhi, „MEMS Applications of NiTi Based Shape Memory Alloys: A Review“, Micro and Nanosystems, Bd. 8, Nr. 2, S.

79–91, Juni 2017.

[3] M. Kretzer, Information materials, Smart Materials for Adaptive Architecture, New York, NY: Springer Berlin Heidelberg, 2016.

[4] A. S. Lah, „Preparation of Shape Memory NiTiNOL Filaments for Smart Textiles“, Tekstilec, Bd. 59, Nr. 2, S. 168–174, Mai 2016.

[5] http://www.dynalloy.com/ tech_data_wire.php, Last access: 21.02.2018.

[6] https://www.sparkfun.com, Last access: 21.02.2018.

[7] Video „What shall we do next?“ (Sequence #2) by Julien Prévieux, 2014.

[8] B. F. Bouzanjani, N. Leach, und A. Huang, „THE DESIGN OF AN INTERACTIVE TENSEGRITY STRUCTURE“, 2013.

[9] Broschuere „Open Codes.

Living in digital worlds“, 2017.

25