Facilitating Interspecies Play // Interaction Design

FACILITATING INTERSPECIES PLAY Physical Computing, DIGF-1006-001

http://panopticamx.blogspot. com/2009/03/nick-veasey.html

Mandeep Mangat mandeep.mangat.s@gmail.com mandeepmangat.format.com

Introduction FOR THIS PROJECT I WAS TASKED WITH BUILDING AN EMPATHY MACHINE - AN INTERACTIVE INSTALLATION THAT GENERATES THE EXPERIENCE OF BOTH EMOTIONAL EXCHANGE AND EMOTIONAL UNDERSTANDING OF ANOTHER IN THE PRIMARY USER. APPLYING A MULTI-SPECIES PERSPECTIVE, I INTERPRETED THIS PROJECT THROUGH CREATING A CO-DESIGNED, FLUCTUATING AND PLAYFUL EXPERIENCE BETWEEN THE USER AND ANOTHER SPECIES. http://www.santafenewmexican.com/pasatiempo/art/exhibitionism/bryan-whitney-at-david-richard-gallery/article_4718ff7e-aba6-53cb-ad59-b525317c6728.html

AS WE WERE REQUIRED TO USE INPUT (SENSORS) AND OUTPUT (ACTUATORS) DATA TO MAKE THE “MACHINE” COME TO LIFE, I DECIDED ON A TOUCH SENSOR (TO SENSE THE INTERSPECIES INTERACTIONS) AND MIDI OUTPUT TO A DIGITAL SYNTHESIZER AS MY ACTUATOR (ENABLING A SENSE OF PLAY THROUGH MUSIC).

02

TABLE OF CONTENTS Content

Page

Introduction

01

Concept

04

The Final Piece

05

Prototype Development

07

Shelf

07

Electronics

13

Software Development

15

Circuit Diagram

17

Digital Synthesizer

18

Conclusion

25

References

27

http://allthatsinteresting.com/bryan-whitney-x-ray-art

“IN THE ‘JOINT DANCE OF BEING’ CRITTERS ARE ‘BOUNDED IN SIGNIFICANT OTHERNESS’ - A RELATIONAL CONSTITUTION OF IDENTITY WHERE BEINGS ‘ARE NEITHER WHOLE NOR PARTS’ BUT RATHER ARE MADE IN THE FLESHLY SPACE OF ENCOUNTER.” (Katherine Wright)

04

Concept Statement The empathy machine that I designed is inspired by the emergence of a multi-species perspective - a discourse that re-examines the role between Homo sapiens and nonhuman lifeforms.

In the past three decades, there has been an increasing accumulation of scientific evidence that often re-frames non-human species as having human-like qualities, such as: intelligence; living within social fabrics, and complex emotional capabilities.

A multi-species perspective challenges the boundaries between the concept of the human and the non-human condition. It reconsiders the framework of the existing ethics within our modern day society.

This piece encourages a playfully symbiotic interaction between species. The application of touch (from the user) stimulates the otherwise inhibited senses of the indoor plants, and in response (through the touch capacitor), the user is rewarded with audio sounds.

The Empathy Machine The frosted white acrylic houses the plants along with the arduino leonardo, and capacitive touch shield.

06

Parts Used: Arduino Leonardo, Capacitive Touch Shield, 22 AWG Hookup Wire, 4 x Biomedical Pads, 3/8� Frosted Acrylic

Prototype Development: Shelf I n order to both avoid short circuits and allow the touch sensor to play different sounds, I needed to keep the plants separate from one another.

08

Measuring the size of the pots that the plants came in, I was able to determine a safe distance between the plants.

By using these measurements, I then sketched out the design for the shelving unit and purchased acrylic sheeting.

Prototype Development: Shelf In order to make the acrylic sheeting more manageable, I reduced the size of it using the table-saw, followed by the band-saw.

10

Afterwards, I finalized the measurements of the cut pieces using the wet sander.

To make the holes (that would contain the planter) in the acrylic sheeting, I made a file on adobe illustrator and had the sheeting cut with a laser cutter.

Prototype Development: Shelf To prepare, I secured the corresponding joints together using masking tape and then secured them the solvent “res-n-bond�. Once dried, I would move onto another edge.

12

Once the exterior of the shelf was constructed, I placed two small acrylic panels inside the shelf to help stabilize the micro-controller and sensor board.

In order to acquire accurate data from the plants and avoid damaging them, I wrapped wire around biomedical pads which would then stick onto individual plant leaves.

14

Prototype Development: Electronics The wires would be connected to biomedical pads on one end, and on another, the capacitive touch shield.

Software Development: Circuit Diagram The header pins allowed me to snap the capacitive touch shied directly on top of the arduino leonardo micro-controller.

The output sent from the plants > touch sensor > microcontroller > laptop is in the form of MIDI data (Musical Instrument Digital Interface). This MIDI data is read and responded to by a digital synthesizer software.

16

Software Development: Digital Synthesizer Leveraging collaborative reviews from various media artists, Ableton was frequently cited as a preferred digital synthesizer software.

I downloaded a free trial and began by exploring the interface. The version I installed came with a step-by-step instructions that introduced the different features of the software.

18

Software Development: Digital Synthesizer This required me to manually map the MIDI input from each biosensor to my chosen tracks.

While Ableton is compatible with MIDI input, the software does not recognize the Arduino platforms input channels.

20

Software Development: Digital Synthesizer To set-up the software to respond to the MIDI input, I created a Drum Rack Track (see right side of screen shot).

Next, I selected the Drum Rack, track 8 for more information to appear at the bottom of the screen.

I searched the pre-existing audio samples that the software comes with (top-left of screen shot) for gentle, meditative sounds.

Once I found a sound file that I liked, I would click on it and drag the file into an empty slot into the drum rack below.

22

Software Development: Digital Synthesizer Once I had my audio files that I wanted to play, I had to match them up with the appropriate plant sensor.

Selecting the “MIDI” option on the topright of Ableton’s interface allowed me to enter a new screen for manually mapping each sensor to an audio sound on my drum rack.

To do this, I would trigger the sensor by touching the plant. Once done, the touched sensor would appear as a channel on the “MIDI Mappings” tab.

From this tab, I would select the note that I would want to be triggered in the Drum Rack.

For instance, the first biosensor appeared as Channel 1, correlates to Note D1 (Note D1 is selected in the Drum Rack, as “Freeze, 6-FM Sitar”).

24

Conclusion This interactive installation is intended as an inquiry into interspecies relations, with an emphasis on technological interventions to augment human to plant communication through both a tactile interface and audio feedback loop. In consideration of this, the experiences that emerge through the designed interactions are considered a success, but as an iterative process there is always room for further development.

While touching the plants, the users immediately developed a greater awareness of their hands and movements in relation to the plants, due to the real-time feedback of the digital synthesizer. For instance: touch triggered a track; continued touch would result in the volume of the particular track escalating; and ending the tactile interaction would result in the track fading out and ending. The interactions and sounds selected resulted in a feeling of wonder. Users reported feeling connected to the plants through the co-designed process of livemusic creation.

26

Unfortunately, the implementation of technology was overly apparent - and deterring from my intention of an organic moment between species. Particularly, the reliance on an external digital synthesizer on a laptop drew the users attention away from the plant installation, and served as a reminder of the technosphere. The next prototype should emphasize greater integration of electronics, and remove the dependence on a laptop altogether. For instance, an audio board (that is compatible with arduino and accepts MIDI input to trigger audio output) and speakers could be included in the plant shelf.

Through my research, I discovered that different species of plants respond differently to touch (IE: stimulated, indifferent, or even harmed). In future iterations, species of plants that respond well to touch to include are: Potentilla recta, Arabidsopsis thaliana, and Linaria vulgaris.

Lastly, larger themes to consider are that of the role of technology to intervene in experiences and to prompt synesthesia. As well, throughout this project I have been reflecting on the ethical responsibilities a designer must navigate in the process of representing and granting a “voice� to a species unable to participate in the dialogue.

References Due to the scope of this project within a limited time-frame, I relied on existing tutorials to inform my process and development.

Adding electronics to plants, especially in audio form (IE. Giving non-speaking species a voice) turned out to be a popular topic.

As well, within the same theme, I found some very interesting resources on: fauna bio-electricity; biosensing for human to computer interactions; creating electronic actuators that respond to galvanic skin response; and touch sensors that create MIDI output.

“Moss Power.” Tumblr, 2013-2016, mosspower.tumblr.com/.

Accessed on 3 Feb. 2018.

“When Plants Jam with Synths: Leslie Garcia’s Open Project Lets Plants Talk with Sound.” Create Digital Media, 27 March 2013, cdm.link/2013/03/when-plants-jam-with-synths-lesliegarcias-open-project-lets-plants-talk-with-sound/. Accessed on 3 Feb. 2018.

Christie, Thomas. “Plant Sounds.” Youtube, 21 April 2015, www.youtube.com/watch?v=VvWPT4VhKTk. Accessed on 5 Feb. 2018.

DangerousTim. “Arduino Tutorial: Capacitive Touch Sensors.”

Instructables, 13 Sep. 2014, www.instructables.com/id/How-ToUse-Touch-Sensors-With-Arduino/. Accessed on 12 Feb. 2018.

Electronics, A2D. “Easy DIY Arduino Touch Sensor Piano.”

Instructables, 25 Jul. 2017, www.instructables.com/id/EasyDIY-Arduino-Touch-Sensor-Mini-Piano/. Accessed on 11 Feb. 2018.

28 03

Hess, Der. “Biosensing for Human Computer Interactions.”

Der Hess, am medialen werkeln, 14 June 2014, blog. derhess.de/2014/06/14/biosensing-for-human-computerinteractions/. Accessed on 7 Feb. 2018.

Hess, Der. “Can Plants Be A Musical Instrument.” Der Hess, am medialen werkeln, 14 June 2014, blog.derhess. de/2013/10/08/can-plants-be-a-music-instrument/.

Accessed on 7 Feb. 2018.

Howse, Martin. “Biologic Sensing/Systems Research.”

Micro_Research, 28 Mar. 2013, www.1010.co.uk/org/ biologic.html. Accessed on 12 Feb. 2018.

Madshobye. “Singing Plant. Make Your Plant Sing With Arduino, Touche and A Gameduino.” Instructables, 4 Oct. 2012, www.instructables.com/id/Singing-plant-Make-yourplant-sing-with-Arduino-/. Accessed on 5 Feb. 2018.

Patel, Abhilash. “DIY: Arduino Based Continuous Touch Piano.” Instructables, 11 Feb. 2017, www.instructables. com/id/DIY-Arduino-Based-Continuous-Touch-Piano/.

Accessed on 11 Feb. 2018.

Straaten, Bas. “Ardunio Touch Capacitive MIDI Keyboard.”

Instructables, 9 Feb. 2014, www.instructables.com/id/ Arduino-Touch-capacitive-Midi-keyboard/. Accessed on 11 Feb 2018.

Straaten, Bas. “How to Make a Cheap Arduino MIDI Controller.” Instructables, 7 Dec. 2016, www.instructables. com/id/How-to-Make-a-Cheap-Arduino-MIDI-Controller/.

Accessed on 11 Feb. 2018.