Master in Advanced Architecture 2013-2014
PROTOTYPE GINGA
THE DANCER BECOMES MUSIC ADVANCED INTERACTION STUDIO. Sebastiรกn Alvarado Grugiel Costa Rica June 2014
Master in Advanced Architecture 2013-2014
PROTOTYPE GINGA
THE DANCER BECOMES MUSIC ADVANCED INTERACTION STUDIO. STUDENT Sebastiรกn Alvarado Grugiel Costa Rica.
FACULTY Carlos Gรณmez Xavi Gonzรกlez
‘ ‘I would only believe in a God who could dance.’’ Friedrich Nietzsche
INDEX Introduction 1
Pre-study
7 9
Advanced Architecture Sound, Color and Interaction 2
Interaction
17
Input/Output - Looking for Loops About Humans Synesthesia Designing Behaviors for New Humans 3
Prototype GINGA
28
Context - Capoeira Reading the Body Sensor Setup Turning Data into Sound 4
Exhibitions
36
SONAR 2014 IAAC References
42
Special thanks
44
Introduction
The dancer becomes music. Ginga is a wearable technology that turns body movement into sound. Our body is the result of thousands of generations of intense evolution. When we use modern technologies our bodies are reduced to a chair and a finger click. Design creates behaviours. A new approach is needed to develop the technologies of the future, starting from our physical reality, giving emergent technologies a human shape. Prototype Ginga is an experiment conceived in Capoeira, the Brazilian dance/ martial arts.
IaaC
MAA_01
2013-2014
1
PRE-STUDY
Advanced Architecture Now a days different disciplines of knowledge are finding points of coincidence as they develop and get more specific. There is no more an expert in architecture, but an architect who is an expert in some area of the field, within asymmetric groups of experts. Disciplines mix as they get complex and creativity and intelligence result in hybrid new fields of knowledge, linking and understanding in new ways, as sciences look for answers inside each other. The first personal interest in interaction came from an interest in music, trying to find a link between personal passions. Architecture in evolving incredibly fast now, and is demanding of new links between knowledge. If we compare the evolution of architecture and the evolution of life, we see that architecture have evolved in form, while life evolves in form and intelligence. Will architecture become intelligent?... Advanced Architecture is intelligent, through interaction and communication. 2013-2014
MAA_01
IaaC
11
Prototype GINGA
12
IaaC
MAA_01
2013-2014
2013-2014
MAA_01
IaaC
13
Prototype GINGA
Sound, Color and Interaction
Sound is essential for the interaction of the most complex kind of life. Language is one of the most important elements of human interaction and intelligence. It could have evolved from music, as exposed by Rousseau, Herder, Humboldt, and Charles Darwin. Complex organisms like humans use also color/light as information to interact. Can intelligent Architecture use color/light and sound to interact and communicate? Color and sound can be measured, and turned into bits of information. Bits can be transmitted and under a certain code, they can become messages for interaction. This way, color becomes sound and sound becomes color. They are both just channels for information. This can be used to develop Intelligence in Architecture designing new lightscapes and soundscapes.
14
IaaC
MAA_01
2013-2014
2013-2014
MAA_01
IaaC
15
Prototype GINGA
Experiment 1 Using seven basic colours and assigning them to each musical note, we can trans-late sound information into colour information. How would that change how we experience music, space, atmospheres?
16
IaaC
MAA_01
2013-2014
2013-2014
MAA_01
IaaC
17
Prototype GINGA
MORE information at https://www.youtube.com/watch?v=C-jtWJiO6SI
18
IaaC
MAA_01
2013-2014
2
INTERACTION
Input/Output - Looking for Loops
Interaction is the action that occurs as two or more objects have an effect upon one another. Object 1 sends an output to Object 2, which receives it as an input. Object 2 reacts sending an output that Object 1 receives as input. This modifies the behavior of Object 1 and its next output. Interaction sets a loop of inputs and outputs that modify the behavior between objects. Interaction Design focuses on the way people interact with each other using external objects. Interaction is as good as the experiences that people can share. Technology is a tool to make this interaction more complex and intriguing.
2013-2014
MAA_01
IaaC
19
Prototype GINGA
Kids playing in a square. The metallic pyramid is nothing particularly special, but the people’s interaction gives it an important meaning. The interaction between people, its complexity, and the experiences we learn and share, is what makes interaction meaningful.
20
IaaC
MAA_01
2013-2014
Input
A loop of inputs and outputs.
2013-2014
MAA_01
IaaC
21
About Humans Interaction is as good as what people share and learn from each other. People should be the target of any interaction design. Technology improved how we communicate today and proved to be a great tool for exploring new ways of interaction, making it more complex by linking new interests. Modern humans have been existing for 200 000 years. This means that there are around 6000 generations of humans between us and our first ancestros. We are the result of such evolution, resulting in the biological technologic marvel that is our body. We are capable of incredible things: we play music, dance, sing, paint, build, swim, glide, surf, you name it; and our brains backup all our creativity and understanding of the world.
22
IaaC
MAA_01
2013-2014
The human body is a powerful result of evolution.
2013-2014
MAA_01
IaaC
23
Synesthesia “This music crept by me upon the waters” From “The Tempest” by William Shakespeare. This project of interaction was based on how to add complexity to the human natural behavior. The main interest was in the human factor of interaction. We define reality using several channels, we call senses, to register information from what we call the world around us.
24
IaaC
MAA_01
2013-2014
The data from the environment is picked up by receptors that are specialized to respond to a specific physical stimulus. The data is transferred to the brain as signals with the same amplitude but different frequencies. The brain doesn’t determine the type of stimuli since all arrive in the same form, but from the place the stimuli came from. This means that any stimuli received by the photoreceptor will be decoded in the visual center of the brain as light information; if by any chance it came to the sound center, it would be interpreted as sound. This ability/weakness of the brain to mix sensory signals some times lead to SYNESTHESIA.
Understanding interaction through the human experience.
2013-2014
MAA_01
IaaC
25
Prototype GINGA
Synesthesia is a neurological phenomenon that occurs when the stimulation of one sensory or cognitive pathway is experienced in a second sensory or cognitive pathway. People with synesthesia may see each letter in a particular color when they are actually black and white and even if they are blind, experiment taste in their mouth when they listen to a particular word, or count by arranging number in space and color. Studies on synesthets reveal that our brain establish sensory associations during our childhood, all the mixed sense experience of synesthesia is defined when we are very young. Synesthets appear to have connections between different areas of the brain that are typically completely separate. Although synesthesia is present in a minority of people and with very diverse manifestations, it might help to explain the complexity of human intelligence. Everyday we use synesthetic principles to understand complex concepts, like number sequences. Synesthesia helps to explain another critical skill of the human condition: creativity.
26
IaaC
MAA_01
2013-2014
Genetic studies on synesthesia have found that synesthetic experiences are 8 times more common among artists, poets and novelists. This increased creativity could be caused by the perception of a sense disguised as another and the ability to perceive hidden information. The basis of creativity is finding unexpected links. The use of metaphors in language evidence our natural ability to link our senses to create and describe concepts. We might say things as loud green, sharp cheese, bright sound, where we mix sound, color, shape and physical properties together. Synesthetic processes could even explain the origins of human language, according to several researchers. In a way, we are all syn-
Some kinds of synesthesia allow people to see letters in color, even thought they are printed in black and white.
2013-2014
MAA_01
IaaC
27
Behaviors for new humans Design makes behavior. Designers have to be conscious that every time we design a product, we are also design how people are going to behave around it. This is a huge responsibility, challenge and opportunity. Modern technologies, even though powerful, tend to constrain the body into not natural behaviors. Modern humans suffer from diseases related to the lack of movement and forced sedentary live styles. Future technologies will take a human form so that instead of limiting its potential, it will amplify it. How should technology attach to the body? The picture shows the work of Heather Hansen and how she creates several drawings based on the body movement and proportions. New technologies can improve the human interaction by finding new ways of understanding the relationship between the human reality and the technological interfaces.
28
IaaC
MAA_01
2013-2014
Physical Movement Translated into Symmetrical Charcoal. Emptied Gestures by Heather Hansen.
Neil Harbisson uses this color sensor that detects color frequency and produces a sound in his head. This way he can listen to color. The paintings shown are his interpretation for F端r Elise by Beethoven and Triumphal March from Aida by Giuseppe Verdi, from left to right
2013-2014
MAA_01
IaaC
29
3
PROTOTYPE GINGA
Choosing a context - Capoeira Based on the previous research, this experiment was run to clarify the viability for technology to take a human form, reading its natural behavior to develop new interfaces between humans and machines/computers. A context was chosen in order to define limits and direct the project. There was a special interest since the beginning of the studio in music for interaction. This was added to the interest in the human experience and its physical reality. The perfect context would be that in which the body could express freely and creatively in relation to music, using all the body potential. Several contexts were considered specially contemporary dance, break dance and martial arts. The perfect combination was found in Capoeira, a Brazilian dance/martial art. GINGA is the name of the main movement in the Capoeira dance/martial art. 30
IaaC
MAA_01
2013-2014
Capoeira is a Brazilian martial art that combines elements of dance, acrobatics and music. It originated in Brazil in the 16th century. When Portugal reclaimed Brazil, they brought slaves from Africa to work the lands in inhumane conditions. Slaves developed several fighting techniques hoping to fight their way to freedom. They trained with music trying to disguise their fighting techniques as a dance or a game, in order to avoid punishments. Modern Capoeira is practiced as a performance dance, using the traditional instruments and songs. Even though it still uses several fighting movements, it is not a combat martial art. It is a dance that improves your body abilities and reinforce social relations. 2013-2014
MAA_01
IaaC
31
Reading the body The project developed a set of sensors capable of reading the body movement, in this case, the Capoeira dance. The traditional music of Capoeira will play and the ‘‘players’’ will start dancing. Each player will have a set of sensors. As they move following the music, the sensor will read their movements and translate them into data that will alter the base music, adding sounds or modifying it. The resulting music and sound will alter again the behavior of the dancer, which will react to it with new movements that will again alter the base music. In this way, there is a loop of action-sound-reaction in which the dancers not only listens to the music but make it. The set of sensors amplify the human output adding complexity to the experience. This results in an interaction that adds complexity to the experience. The device amplifies the human power. The dancer listens to the music and dances, the sensors turn the dance into data, the data is transmitted to a computer that uses it to modify the music. The new music is reinterpreted by the dancer, who adapts the movement to the new sounds. The loop is closed and flowing. 32
IaaC
MAA_01
2013-2014
Human
Flex sensor Accelerometer
Sound/Music
Wireless micro controller. Arduino Fio + Xbee
Computer + MAX/MSP Music programming
Flowing loop of interaction.
2013-2014
MAA_01
IaaC
33
Sensor Setup The device works by attaching a flex sensor in the elbow to detect the extension of the arm and the act of opening the body. An accelerometer in the wrist reads the start and end of any wrist displacement in space. It reads in x,y,z axis. The sensors send those inputs to an Arduino Fio board and turns it into numbers. The data is sent wirelessly via Xbee radios to a computer. The numbers are put into a programming language for music called MAX/ MSP, which creates several frequencies with each input. The real achievement of the prototype was to turn the body movement into data that can be used, remapped and modified as parameters in different computer applications. This means that the device can be used in several fields such as music composition, music performance, dance performance, medical uses, virtual reality interfaces, robotics and sports. All the technologies used to develop this prototype are open source.
34
IaaC
MAA_01
2013-2014
First sketch.
Accelerometer Flex sensor
Arduino FIO + LIPO battery
Prototype GINGA
2013-2014
MAA_01
IaaC
35
Turning movement/data into sound Arduino Fio+Xbee Radio signal to computer
Flex Sensor
Accelerometer X, Y, Z axis
Numbers are information that can be used to do anything. Movement is turned into data using a sensor setup. The numbers are used to produce sound and modify music, through a programming language for music (max/msp).
36
IaaC
MAA_01
2013-2014
GINGA - Basic movement in Capoeira
X
Y
Z Flexion
Data received in four channels. Shown as a wave.
2013-2014
MAA_01
IaaC
37
Exhibitions As part of the study programme for the Interaction Studio we had the chance to present our project in SONAR+D, a international conference for creativity and technology, designed to promote talent and business opportunities in the digital culture environment. It was also shown in the final presentation of the master in IAAC. People had the chance to try out GINGA and the feedback got from them was of great value. The sound of GINGA is very experimental, so as people move they change basic frequencies. People immediately recognized the sound changing as they moved, which proved that when technology turns into human shape we can instinctively adapt it to our natural behavior. Several people described the sound as robotic, and suggested different kinds of sound like nature sounds of water, rain, or more recognizable sounds. Even though, the reaction was generally very positive and many of the visitors really engaged into the experience of listening to their own body as it moved.
38
IaaC
MAA_01
2013-2014
From the feedback and the experiences learned, this project could grow in several directions such as medical applications for rehabilitation, dance performances, music composition and robotics. It could also complement alteration to traditional musical instrument or even become one by itself.
MORE information at:
instagram.com/gingasonar2014
facebook.com/gingasonar2014
2013-2014
MAA_01
IaaC
39
Prototype GINGA
SONAR 2014
40
IaaC
MAA_01
2013-2014
2013-2014
MAA_01
IaaC
41
Prototype GINGA
IAAC Final Presentation
42
IaaC
MAA_01
2013-2014
2013-2014
MAA_01
IaaC
43
REFERENCES Connecting – Trends in
The 4 ways sound affects us
UI, Interaction, & Experience
Ted Talk by Julian Treasure
Design
ted.com
by Bassett & Partners
2009
youtube.com 2014
What color is Tuesday?
When Senses Collide
-Exploring Synesthesia
-Synesthesia Origins
ted.com
BBC
2013
2008
I Listen to color
Software (as) art
Ted Talk by Neil Harbisson
Ted Talk by Golan Levin
ted.com
ted.com
2012
2004
How architecture helped music evolve Ted Talk by David Byrne ted.com 2010
44
IaaC
MAA_01
2013-2014
IMAGES REFERENCE
Ear by Jerry Wang
Synesthesia
Page 25
Page 28
The Noun Project
wikipedia.org
Eye by Jean-Philippe Caba-
Emptied Gestures
roc
By Heather Hansen
Page 25
Page 30
The Noun Project
heatherhansen.net
Nose by Rachel Healey
I Listen to color
Page 25
By Neil Harbisson
The Noun Project
Page 30 ted.com
Hand by Maico Amorim
Capoeira
Page 25
Page 32
The Noun Project
By Jacek DylÄ…g
Mouth by Sergi Delgado Page 25 The Noun Project
2013-2014
MAA_01
IaaC
45
Special thanks Carlos G贸mez Xavi Gonz谩lez Guillem Camprodon Apostolos Mouzakopoulos John Giraldo Giombattista Areddia Rodolfo Parolin Pablo Marcet Raphael Libonati Michele Braidy Mary Katherine Heinrich Esther Marquez Carmen Aguilar Daniel Giraldo Marina Castan Rhys Duindam Tomico Plasencia 46
IaaC
MAA_01
2013-2014
valldaura S elf S ufficient Lab
www.iaac.net Accredited by:
www.upc.edu
fablabbcn.org
www.valldaura.net
The dancer becomes music. Ginga is a wearable technology that turns body movement into sound. Our body is the result of thousands of generations of intense evolution. When we use modern technologies our bodies are reduced to a chair and a finger click. Design creates behaviours. A new approach is needed to develop the technologies of the future, starting from our physical reality, giving emergent technologies a human shape. Prototype Ginga is an experiment conceived in Capoeira, the Brazilian dance/martial arts.