1
Fruitmotion
by Tsun Chun Chen
2
fr uitmotion
3
CSA Research Report
Project Details Project Lead:
Lucy Jones, MA course Leader
Design Participants:
UCA MA Architecture Students UCA MA Interior Design Students David Di Duca, BAT Studio
Title:
Fruitmotion
Type:
Interactive motion installation
Location:
UCA Canterbury campus CSA Entrance Lobby
Project Dates:
07 August 2017 Final Portfolio Submission 14 August 2017 Research Report Submission 31 July - 23 August 2017 MA Show Preparation 25 August - 01 September MA Final Exhibition
Design Period:
05 June - 18 August 2017
Budget:
£ 400
Scale:
3.6m x 2.4m x 0.6m
Support:
University for the Creative Arts, Canterbury
4
Fig.01 motions, trails, spaces are all involved in 'purchase behaviour' to present interaction among customers, commodities and retail spaces.
fr uitmotion
5
Fig.01 The interactivity explores the possible movements of people and animated objects, such as the shelf and fruits. As the motion of the participants are detected by the webcam camera, the shelf will react and behave in different ways to interact with the participants paradoxically.
fr uitmotion
6
Research Agenda and Process Overview The aim of this installation is to explore possibilities of interaction between people and objects. In particular, I am interested in examining how animating objects elicits different emotional responses. This work has emerged from my studies on the use of interior design in retail spaces, and how design can express different emotions in order to promote sales. This project tries to reverse so called ‘purchase behaviour’ by experimenting with the animation of fruit and shelf. It uses emotion, movement and idea of paradox as design concepts. The final prototype does create specific feelings for participants as they gaze at the shelf on the wall. The movements of the animated shelf and reaction of participants illustrate spatial languages and different emotions. Research Questions 1.
How does the animation of objects elicit different emotional responses in human participants?
2. Equally, how can an installation become animated by its participants’ responses? 3. How do the interactions of people and objects in this context change spatial relationships?
Research Statement
Significance and Contribution This project takes a different approach to both ‘purchase behaviour’ and interaction design. It draws on the emotional aspects involved in the design of commercial spaces to question the inert nature or display systems and explore emotional dimensions of interaction design. The installation encourages participants to address everyday objects in a way that questions their essence and properties in a space. It builds knowledge by focusing on factors involved in the process of shopping such as the shopper’s gaze and how they think about objects of consumption or desire.
Methodologies 1.
To prototype and make a series of live tests that experiment with how we can animate objects. This will be through observation in retail spaces, and through model making and human involvement.
2. To prototype ways in which participants respond to, and in turn animate the object. Those prototypes will be triggered by sensors and arduino programmes. 3. Observation and analytical drawings of the installation in use via enquires and photographs and video stills of participants,focusing on gaze, movement and interaction. 4. Critical reflection on findings, and development of final piece.
7
fr uitmotion
8
Design Proposal Reflecting on the concept of animated objects and the use of interior design in retail spaces, I have recognised display shelving serves an important role in displaying commodities to customers. Interactions among shelves, customers and aisles elicit spatial meanings that express relationships of people, objects and spaces. This project combines three main factors together: participants, objects and space. The object will be the display shelving for fruit in a supermarket; the space will be a corridor, akin to an aisle,
as experienced by customers while they are shopping. The interaction occurs when customers approach the shelves and try to pick up the fruit. Participants’ movements will be detected, and response to this the shelving and fruit will be animated. The idea of the interactive process is related to emotion-involved ‘purchase behaviour’ that illustrates several factors will have an impact on people’s shopping experience. These include: form and material of shelving, fixtures, lighting and spatial environment.
Key technological outcomes of proposal 1.
To develop an interactive display device - termed the ‘hanging plotter’.
2. To make a hanging plotter with the best functioning movements of the magnet by gears, stepper motors, mathematics and arduino programme. 3. To for
write and edit functioning programme codes specific movements of the hanging plotter.
4. To connect the processing programme with the arduino system successfully via computer.
9
Proposal & Context
Design Research Context Field of Work
Work by others
The project explores emotional and paradoxical design via autonomous and magnetic systems, discussing the interaction between people and daily objects. It focuses on movements that have influences on people’s emotion and feelings in a space.
Several projects set out by a studio called Interactive Architecture Lab (IAL) show different possibilities of emotionally involved installations with spatial relations.
Emotional design is explored by many researchers who relate human behaviour and people’s memories, which they argue impact people’s perception and activities in a space.
The most notable project that has inspired this project is the: ‘Eye Catcher’ made by IAL. An interactive installation that engages people’s expressions and movements that trigger a robotic system to interact with participants. Emotional design is used in Eye Catcher project to combine human perceptions and vision, creating subtle interactive processes between objects and people’s movements.
Fig.02 "The Eye Catcher project in its conclusion has developed a novel expressive interface where emotion recognition algorithms read audience faces and in-turn trigger the animation of a face formed of ferrofluid" (Lin, 2014).
fr uitmotion
10
Design Methodologies This project is dedicated to four main elements that need to be mastered so as to perfect an interactive installation: 1.
Successful hanging plotter. Correctly sized gears are made to combine with stepper motors and threads to function as a hanging plotter.
2.
 Mathematics and lengths of threads. Calculating lengths of threads in regard to resolutions of stepper motor per degree is an important factor for movements and positions of the magnet.
3. Style and material of shelves. Different kinds of styles and materials of shelves have various weights. The less weights the shelves have, the eaiser it is for the magnet to support the shelves on the wall. 4. The final aspect will be creating a system that will react to people’s presence in space. A webcam camera will detect people’s movements and distances, transmitting data back to processing and arduino programmes. The programme will activate the hanging plotter to move around a surface with the magnet.
Critical Design Elements 1.
Bespoke and functioning hanging plotter
2. Well-calculated distance of threads 3. Shelves with proper weights and appearances 4. Interactive system controlled by arduino, processing, and webcam camera
11
Process & Methods
Prototyping and testing 1.
Hanging plotter prototyping and testing a.prototyping
b.testing
The first element that needed    Using simple arduino codes with button controlling to be understood and resolutions of stepper develped was the hanging motors and measuring plotter. distances, the prototype was made to record different In order to research numbers of lengths of this, a small scale threads and calculate functioning prototype specific positions and was made for testing movements of the magnet. movements and distances. The tests also included Using MDF as material for compatibility of MDF the gears, cotton threads for gears, stepper motors, paths and a small magnet for power supplies and tests, with the structure of combinations of arduino. the prototype constructed from scrap wood boards.
Fig.03 hanging plotter prototyping and testing
fr uitmotion
12
a.prototyping
a.prototyping
a.prototyping
a.prototyping
a.prototyping
a.prototyping
13
Process & Methods
b.testing
b.testing
b.testing
b.testing
b.testing
b.testing
fr uitmotion
14
2. Mathmatics and calculations of threads and distances via Pythagorean Theorem c.Pythagorean theorem
d.grids for distance
Using Pythagorean theorem Papers with grids were made to test the distances to measure distances from of magnet, and the one point to another point process of tests were is a method not only for all recorded as numbers physic movements but also for the above theorem. for writing programme languages. The full code developed will be detailed in the Appendix section.
05a
Fig.04 papers with grids for theorem and records
15
Process & Methods
B
A d.grids for distance
In order to move from point A to B, Pythagorean Theorem was used to measure distances between each point and understand how lengths of threads should be cut for hanging plotter.
B
A c.Pythagorean Theorem
16
fr uitmotion
Process & Methods
Fabrication Techniques 1. Making hanging
a
functioning plotter
a. using ‘Gear Generator’ programme to design matched gears
e. drilling the MDF board for bearings
gears board
f. assembling gears, threaded rods, nuts and bearings on the wood board with motors
c. gluing several layers of gears together with superglue
g. using copper wires and nails to fix threads
d. measuring and marking the position of motors and their support structure on the MDF board
h. drilling the MDF board for magnets and threads
b. laser cutting from MDF
Fig.05 making a functioning hanging plotter
17
Process & Methods
a.programme
b.laser cutting
d.measuring and marking
f.assembling
g.fixing threads with nail
h.’the plotter’
18
Process & Methods
fr uitmotion
2. Calculating and assembling of threads a.
connecting threads with the MDF board mounted magnets through holes
b.
measuring the distances between the MDF board and bearings
c.
tying up threads on bearings
3. Making a shelf with proper weight and size a. collecting a scrap of corrugated cardboard box from a supermarket b.
cutting and assembling the box into a shelf
c. combining the box with metal piece and magnets on the back
Fig.06 calculating and assembling of threads
19
Process & Methods
2a
2a
2b
3a
3b
3c
20
fr uitmotion
Process & Methods
4.
Making timber structure for the wall
a.
measuring and marking required dimensions on timbers
b.
cutting the marked timbers
c.
assembling the timbers with screws and MDF board
Fig.07 making timber structure for the wall
21
4b
4c
4c
4c
4c
4c
22
fr uitmotion
Process & Methods
5. Creating motion and distance detected device This project is a motiondetected and interactive spatial installation. The interactive process is that when people walk through a corridor, they will see a shelf with fruits on the wall, and as they get closer, the shelf will have specific movements related to people’s motions detected by webcam camera.
For this interaction, hanging plotter and processing programme are going to be connected with arduino and power supplies. The code used will be explained in the Appendix section. a. diagram of the set up used b. the components used
stepper motor SM-42BYGH011-25 webcam camera USB
laptop
arduino cable
I2C
power supply
arduino board
motor driver _
+
12V shifting power supply
220v socket
5a. diagram of the set up used
23
5b
5b
5b
5b
5b
5b
24
fr uitmotion
Process & Methods
Control Systems In order for all elements to function appropriately in this project, the following control systems were introduced to the work undertaken. a.hanging plotter Using hanging plotter to create movements of the shelf that can interact and react with people in front of the wall is combined with magnet and shelf rather than pen. The advantage of the hanging plotter is its spontaneous positions in the process of movements that create different feelings for the audience. b.shelf The shelf is a symbol of the animated object in retail spaces, and it also provides several displays for customers in spatial environments. The shelf represents the connection between customers and commodities, and the relationship between space, humans and unanimated objects.
c.lengths of threads Calculated by Pythagorean Theorem, the threads were measured carefully to ensure proper movements between resolutions of motors and distances. d.timber frame structure In order to create a surface for the shelf to move on, the timber frame structure was made to support the freestanding wall. Owing to the large width and height of the wall, the timber structure was separated into four sections and then assembled in place. e.arduino The arduino codes had several revisions to control the movements of the shelf, and explore the possibilities of the programme language. f.webcam camera processing
and
The most important role in this project is the processing and webcam camera that detects participants’ movements. The video shot by the webcam camera will send the data back to the computer and activate the arduino to guide the motors.
25
Process & Methods
Occupation and Interaction This project aims to set up relationships between participants and shelf, which is an interactive process showing motions of people and emotionalized objects. The shelf is supposed to react to people’s motions detected by the webcam camera and make different movements.
move the shelf in a linear path. The second is that as the participants approach the shelf, it will create movements with different directions. The third is a ‘far distant’ scenario that people stand far away from the shelf, and the shelf will stay in the same position.
The interactions between participants and this installation The final one is that people have four different situations: just cross the corridor, and the camera will not detect enough The first one is that when people area in video, so the shelf will not walk through the corridor, move. the camera will detect the same pixel area in video and
05
02-3
02-2
02-1
02
01-3
01-2
01-1
03
01-1
02
01-2 01-3
03
04
01-1_participant out of range of camera 01-2_participant walking through the corridor 01-3_participant noticing movements of shelf 02_participant getting closer to the shelf 03_participant standing too far away from the shelf 04_other people crossing the corridor 05_webcam camera mounted on the top of the wall
26
Review of Outcomes
fr uitmotion
Dissemination and Future Work A possible venue to present the work could be the RIBA Regent Street Windows, which is a collaboration of retailers and architects that creates stunning window displays and installation. The project could also be relevant for the upcoming annual Retail Design Expo (London) May 2018. This event includes a conference bringing together architects, designers and retailers, and provides a platform for sellers who can promote their brands and products. In terms of the interactivity of this project, a possible way to develop this research is to collaborate with supermarket retailers, and promote their sales via showing their commodities in the RIBA Regent Street Windows project and Retail Design Expo. Customers may be attracted by the movements of the commodities in front of window displays.
This project could develope further by researching into programme languages, such as Arduino and Processing. One of the important point of this research is to elicit real movements of objects from digital programme languages. Owing to the reason that this project combines the concepts of motion, purchase and emotion, a further improvement could be installing an emotion or expression-detecting device (rather than just larger, physical body movement) that can observe participants’ face and interact with them. Another critical element in this project which could be developed is the technological to animate the objects. In this case, hanging plotter that controls the movements of the shelf. And one important component of the hanging plotter is the motor which can guide the gears and threads. If the motor could have higher torque that can move more objects on the wall, participants would be able to interact with more objects. However, the budget limitations would not allow this to happen.
27
Appendix
Appendix Annotated Arduino Code Excerpts #include <Grove_I2C_Motor_Driver.h> #define I2C_ADDRESS_A 0x0f //stepper driver one #define I2C_ADDRESS_B 0x0a //stepper driver two int currentX; int currentY; int valX; int valY; const int oneCM = -7; //number of steps to pull the string 1cm in or out- calibrate by changing this const int nailDist = 50; void setup() { Serial.begin(9600); while (!Serial) { ; // wait for serial port to connect. Needed for native USB port only } currentX = 0; // starting an coordinate system from 0,0 currentY = 0; //pinMode(buttonPin, INPUT); } void loop() { //buttonState = digitalRead(buttonPin); talk(); int moveX = valX; int moveY = valY; moveStepper_01(moveX, moveY); moveStepper_02(moveX, moveY); Serial.print(moveX); // all for debug Serial.print(moveY); Serial.print(currentX); Serial.print(currentY); Serial.print(‘\n’); currentX = moveX ; currentY = moveY ; } // end of loop void talk() { if (Serial.available() > 0) { if (Serial.find(“a”)) { valX = Serial.parseInt(); valY = Serial.parseInt(); } } }
fr uitmotion
28
void moveStepper_01(int x, int y) { int newX = x; int newY = y; float lengthA = sqrt(sq(currentX) + sq(nailDist - currentY)); float lengthB = sqrt(sq(newX) + sq(nailDist - newX)); int lengthChange = oneCM * (int(lengthA - lengthB)); Motor.begin(I2C_ADDRESS_B); Motor.StepperRun(lengthChange);
} void moveStepper_02(int x, int y) { int newX = x; int newY = y;
float lengthA = sqrt(sq(currentX) + sq(currentY)); float lengthB = sqrt(sq(newX) + sq(newX)); int lengthChange = oneCM * (int(lengthA - lengthB));
}
Motor.begin(I2C_ADDRESS_A); Motor.StepperRun(lengthChange);
29
Annotated Processing Code Excerpts import processing.serial.*; import processing.video.*; Capture video; PImage prev; float threshold = 100; float motionX = 0; float motionY = 0; float lerpX = 0; float lerpY = 0; int posX = 0; int posY = 0; Serial myPort; // Create object from Serial class int val; // Data received from the serial port boolean mouseState = false; void setup() { size(640, 360); String[] cameras = Capture.list(); printArray(cameras); video = new Capture(this, cameras[30]); video.start(); prev = createImage(640, 360, RGB); //String portName = Serial.list()[0]; myPort = new Serial(this, â&#x20AC;&#x153;COM7â&#x20AC;?, 9600); printArray(Serial.list()); } void captureEvent(Capture video) { prev.copy(video, 0, 0, video.width, video.height, 0, 0, prev.width, prev.height); prev.updatePixels(); video.read(); } void draw() { video.loadPixels(); prev.loadPixels(); image(video, 0, 0); threshold = map(mouseX, 0, width, 0, 100); //threshold = 50; int count = 0; float avgX = 0; float avgY = 0; loadPixels(); // Begin loop to walk through every pixel for (int x = 0; x < video.width; x++ ) { for (int y = 0; y < video.height; y++ ) { int loc = x + y * video.width; // What is current color
fr uitmotion
30
color currentColor = video.pixels[loc]; float r1 = red(currentColor); float g1 = green(currentColor); float b1 = blue(currentColor); color prevColor = prev.pixels[loc]; float r2 = red(prevColor); float g2 = green(prevColor); float b2 = blue(prevColor); float d = distSq(r1, g1, b1, r2, g2, b2); if (d > threshold*threshold) { //stroke(255); //strokeWeight(1); //point(x, y); avgX += x; avgY += y; count++; pixels[loc] = color(255); } else { pixels[loc] = color(0); }
} } updatePixels();
// We only consider the color found if its color distance is less than 10. // This threshold of 10 is arbitrary and you can adjust this number depending on how accurate you require the tracking to be. if (count > 200) { motionX = avgX / count; motionY = avgY / count; // Draw a circle at the tracked pixel } lerpX = lerp(lerpX, motionX, 0.1); lerpY = lerp(lerpY, motionY, 0.1); posX = int(lerpX); posY = int(lerpY); fill(255, 0, 255); strokeWeight(2.0); stroke(0); ellipse(lerpX, lerpY, 36, 36); //image(video, 0, 0, 100, 100); //image(prev, 100, 0, 100, 100); println(lerpX, lerpY);
31
if (mouseState == true) { //println(“a”,mouseX,”, “,mouseY,”\n”); myPort.write(“a” + lerpX + “,” + lerpY +”\n”); } } float distSq(float x1, float y1, float z1, float x2, float y2, float z2) { float d = (x2-x1)*(x2-x1) + (y2-y1)*(y2-y1) +(z2-z1)*(z2-z1); return d; } void mousePressed() { if (mouseState == false) { mouseState = true; } else { mouseState = false; } }
fr uitmotion
32
Materials and Suppliers List Fabrication: MDF board (from workshop UCA, Canterbury) 90x240x0.6/1.2cm Timber (from workshop UCA, Canterbury) 3x3x4000cm Magnets (from Amazon,UK www.amazon.co.uk) Thread (from Amazon,UK www.amazon.co.uk) Bearings (EBay seller) Technical System: Arduino UNO R3 (from Amazon,UK www.amazon.co.uk) Grove I2C Motor Driver (from Amazon,UK www.amazon.co.uk) Webcam Camera (from Amazon,UK www.amazon.co.uk) 9V Power Source (from Amazon,UK www.amazon.co.uk) Other sundries: Wood screw 5 x 80mm, UCA workshop Wood screw 3.5 x 40mm, UCA workshop Corner Plate, Wilcoâ&#x20AC;&#x2122;s UK Threaded rod, Amazon M8 nuts, B&Q Canterbury Black Fabric 75x300cm, C&H Canterbury Rubber Belt, 4D Model Shop
33
Appendix
Bibliography Norman, D.A. (2003) Emotional design: Why we love (or hate) everyday things. 3rd ed. New York: Basic Books. Lindstrom, M. (2010) Buy ology: Truth and lies about why we buy. New York: Crown Publishing Group. Underhill, P. (2008) Why we buy: The science of shopping: Updated and revised for the Internet, the global consumer, and beyond. New York: Simon & Schuster Adult Publishing Group. Morgan, T. (2008) Visual merchandising: Window and in-store displays for retail. London: Laurence King Publishers.
Image Credits All figures are copyright the author unless noted as follows: Figure 02: Lin, C.Z (2014) Eye Catcher. At: http://www.interactivearchitecture.org/lab-projects/eye-catcher (Access on 18 June 2017)
fr uitmotion
34
Credits MA Interior Design Course Leader: Lucy Johns MA Interior Design Design Tutor: David Di Duca Visiting Critics 2016/17: Will Alsop, All Design Hanif Kara, AKT II Jonty Craig - BAT Studio Gem Barton - University of Brighton Jon Hodges - Bare Conductive Guy Woodhouse - Piercy & Co. Charlotte Bocci - Ian Chalk Architects David Lomax - Waugh Thistleton Architects Fiona Zisch - University of Westminster Clemens Plank - University of Innsbruck James Whitaker - Whitaker Studio Kevin Kelly - Pringle Richards Sharrat Tetsuro Nagata - Nissen Richards Studio Elizabeth Upham - MUD Architecture Ruth Lang - Studio ARG Shumi Bose - Blueprint Verity Jane Keefe - The Mobile Museum
35
36
fr uitmotion