BEYOND SCREENS
Beyond Screens Design space for embedded displays of personal walking information
ACKNOWLEDGEMENTS We would like to thank Srishti School of Art, Design and Technology for the opportunity to work on this project, IT - University of Copenhagen for funding the project, and S.Labs for the space and technology facilities. We would also like to thank Naveen Bagalkot and Tomas Sokoler for their encouragement and guidance through the project. We would like to acknowledge our friends at Srishti school of Art, Design and Technology for participating as actors during the filming of the project videos, and for patiently lending their time and technical equipment for the successful completion of our project. We would also like to acknowledge the printing place which printed and bound this folio for us.
TABLE OF CONTENTS Introduction
1
Easy Step
7
Fibulus Hoop
41
Posture Patch
73
INTRODUCTION Naveen Bagalkot and Tomas Sokoler The field of personal informatics has made advances in developing embedded sensors that collect information about a person’s physiology and other aspects of health. This information is then displayed on the person’s mobile or tablet device for self-reflection on how they are managing their health. The underlying vision fueling these advances is to make the technologies of personal informatics part of the everyday life of people. However, there has not been much foray into exploring the various forms the displays can take so that they become part of the everyday life of people. Barring certain seminal artistic explorations1 there have not been many works that push the understanding of ‘information display’ away from the traditional screens and graphical displays, and explore representing information that is perceivable by all human senses. Meanwhile, research shows that people constantly manipulate and configure, reconfigure their social and physical environments as resources for performing health related activities such as walking. In particular, over time, people get adept at using specific objects not only as resources to perform an activity, but also as props for reflection, memory, storytelling, etc. In other words, habituated objects mark people’s activities. Hence, it seems to be promising to embrace such habituated objects2 while designing for integration of digitally recorded information as part of everyday life.
1
For e.g. Marble Answering Machine by Durrel Bishop http://vimeo.com/19930744 Margot Brereton. 2013. Habituated objects: everyday tangibles that foster the independent living of an elderly woman. interactions 20, 4 (July 2013), 20-24. DOI=10.1145/2486227.2486233 http://doi.acm.org/10.1145/2486227.2486233
2
1
INTRODUCTION
The studio project aimed to develop a design space of the various forms displays of personal informatics can take as they become embedded in a person’s social and physical environments. The intention was to explore and push the boundaries of what could be possible beyond the screen dominated world. The studio was driven by six students at Srishti Labs, who focused on displaying the information about the activity of walking as it unfolds in urban India. While the sensor data was simulated, the students explored the forms the display of these data can take, by considering the following questions as starting points: • How can a walking stick inform the walker about how he/she is walking? • How can the walking stick inform the walker about his/her peers walking nearby? • How can the wooden carving on the walking stick hold and reveal historical information? • How can a wear on a shoe indicate the distance walked, the force exerted?
2
Background
The project was the first venture of a collaborative relation between the Interaction Design Lab at the IT University of Copenhagen and Srishti Labs at the Srishti School of Art, Design and Technology, Bangalore. Following an understanding of common interests both the institutions have signed a Memorandum of Agreement with the aim of exploring the overall theme of designing novel interactive pervasive and mobile ICT based products in support of physical rehabilitation as experienced by citizens and professional healthcare practitioners across India and Denmark. In particular, taking the conditions of everyday life, rather than conditions at the clinic, as the point of departure we are guided by a common interest in how pervasive and mobile ICT can be designed to further the integration of rehabilitation processes with everyday rehabilitee living. This in order to help address the societal demand expressed in both India and Denmark for more successful out-of-clinic rehabilitation processes.
3
Results
Over a duration of 4 months, the students focused on opening opportunities for post-activity reflection rather than immediate feedback while walking. Immediate feedback tends to focus on quantified feedback about how a person is walking recording data such as the pace, the posture, the stance, physiological changes, etc. However, by focusing on post-activity reflection the project explored more qualitative possibilities of engagement with the recorded data. In particular, the project explored how by integrating information as part of a person’s everyday activities and objects one may open up opportunities for new encounters with the recorded data, which in turn may invite the person to reflect and serendipitously discover some new aspect of their own activity and life. Three teams with two members each explored three avenues of presenting information about walking as an integral part of a person’s everyday life. This exploration resulted in the articulation of three design spaces. These are as follows: 1) Easy Step: This design space explores the presentation of recorded data about walking for the purpose of self-reflection over a varying timeline. The timeline spans from immediate feedback (getting the number of calories burnt at an hourly basis), daily feedback to more leisurely monthly or quarterly feedback. 2) Fibulus Hoop: This design space explores facilitating everyday post-walk reflection through exploring the timing of data transfer along the dimension of synchronous and asynchronous data transfer.
4
3) Posture Patch: This design space explores supporting collaborative exploration of self-monitored information through varying degrees of proximal interaction based on the level of personal interaction between the partner and the user. The scale moves from most intimate to least intimate. Finally, we see this project and its resulting design spaces as a step towards realizing the intent of the agreement between the two institutions. In particular, we hope that the videos and the report will inspire the more technically sound students at the ITU’s IxD lab to explore the concepts in the Danish setting of everyday walking.
5
POSTURE PATCH
POSTURE PATCH 7
POSTURE PATCH
CONCEPT Designing for collaborative exploration of selfmonitored information through proximal interaction. The user would receive immediate feedback to correct their posture via the silicone patch embedded with skin-mountable electronics (sensors and vibrators). The posture change would be tracked every 30 seconds and this data would be transmitted to the partner’s portable display device (e.g. key-chain) in a cumulative manner. The cumulative colour change would monitor the posture progress by the minute. The pair could then share their reflections through 4 of the possible interaction scenarios.
Logo design for posture patch 8
POSTURE PATCH
Prototype for posture patch
Prototype for posture patch - worn to depict approximate size and placement 9
POSTURE PATCH
DESIGN SPACE Experience factors: • Tracking physiological change • Immediate haptic feedback to the user, and visual data sharing with the motivational partner • Creating an interpersonal interaction while opening a dialogue between user and partner
10
POSTURE PATCH
11
POSTURE PATCH
DESIGN SPACE Dimension: Level of personal interaction between the partner and the user, moving from most intimate to least intimate. The four design scenarios are based on the scale of level of personal interaction between the partner and the user. The scale moves from most intimate to least intimate. The user and the partner must be in close proximity for the patch, the key-chain and the display system to function for any of the following scenarios.
• Mirroring • Posture clock • Progress clock • Texture pad
12
POSTURE PATCH
Most intimate Least intimate
13
POSTURE PATCH
INSPIRATION
AND
RESEARCH
Our inspiration stemmed from wanting to track a physiological change. The initial idea was to have a patch worn on the back that would vibrate, suggesting that the user needed to correct his/her current walking posture. One such existing device is called ‘Lumo Back’, which functions in the same way as our aforementioned initial idea. We integrated a motivating factor to the concept of ‘Lumo Back’. The modified patch would sense muscle movement in four different directions and give relevant haptic feedback. This idea evolved into a system that included a partner device to share information about posture change. It thus enabled the user to be more conscious about his/her posture through motivation by the partner. This led us to a better understanding of how daily walking with the Posture Patch could enhance our everyday interactions. We determined that posture correction is an issue that would be effectively improved through constant physiological reflection and motivation. Hence, our focus became: • To track a physiological change (posture) while walking • To share this data with a motivating partner • To map and view the changes in order to improve posture during and post walk
14
POSTURE PATCH
Tracking a physiological change
Existing posture change tracking device - LUMOBACK
Existing posture change tracking device - LUMOBACK http://abcnews.go.com/Technology/lumoback-review-posture-improving-belt-replace-nagging-mom/story?id=20106207 15
POSTURE PATCH
PROGRAMMING WORKSHOP Programming workshop with Tomas Sokoler: The workshop with Tomas introduced us to the world of programming. We learnt a few basics about the Processing software and were fascinated by how tangibly you could change any kind of physical input into a digital output – be it sound, colours, or data mapping. This was very helpful to understand the code language required to write an Arduino program. With significant help from Tomas we were able to explore the possibilities of using a wifi-enabled micro-processor. It was fascinating not only to make LEDs blink under desired conditions but also to make the same micro-processors ‘tweet’ for us at intervals. The potential application of such technologies in our everyday communications is unfathomably vast. The workshop helped us understand how easily we might be able to translate prior ideas to reality. It also opened a whole new set of possible ideas and concepts to work on.
16
POSTURE PATCH
Bezier curve made using processing
Making a Knightrider LED connection using Arduino 17
Making an LED blink using Arduino
POSTURE PATCH
PROCESS After speaking briefly with Tomas, and narrowing down our area of inspiration, we started the process of creating the system. This involved research, quick and dirty prototyping, and enactment to imagine a-day-in-the-life-of-a-walker and figure out what type of communication and display system would create the interaction that we intended. The system included three key components: 1. A patch worn on the upper back to provide immediate feedback for posture correction 2. A portable device to be kept with the partner to receive data about the user 3. An interactive display system in their home to provide them with cumulative data on the posture progress, hence creating room for negotiation
Form and Material Exploration: We explored various forms and materials for the wearable patch, the mobile device as well as the display system at home. Smith & Nephew’s CICA CARE GEL1 embedded with skin mounted electronics (sensors, vibrators, and flexible batteries) were chosen as ideal materials for the Posture Patch.
1
http://www.smith-nephew.com/key-products/advanced-wound-management/other-wound-care-products/cicacare/
18
POSTURE PATCH
Form exploration
Form exploration in particular material
Form exploration in different materials 19
POSTURE PATCH
PROCESS Iterations for display systems: The display system was integrated into habituated objects2 , thus creating a mix of augmented and augment-able object displays. We envisioned colour changing textiles, dedicated digital displays, and data projecting mirrors as tentative object displays.
Optical fibres woven into fabrics (ambient)
Used as table cloth/centre piece (ambient)
Used as wall hanging (ambient)
Used as bedside mat (ambient)
2
http://interactions.acm.org/archive/view/july-august-2013/habituated-objects
20
POSTURE PATCH
Clock mounted with colour changing silicon sheet (ambient)
Mirror with LCD display (ambient)
Pushpins to indicate posture progress to partner
Slow distribution of colour to indicate posture progress to partner
21
Radial colour change to indicate posture progress to partner
Partner’s portable device can be used as magnet, phone charm, bag accessory, keychain or necklace.
POSTURE PATCH
FEEDBACK FROM TOMAS After reviewing our ideas with Tomas, we decided to create a system to include the user and the motivating partner in a more intimate interaction rather than an ambient one. We focused on proximity between the two devices – the patch and the keychain, to effectively increase interpersonal (face-to-face) interaction. This led to our final concept and design space.
From:
22
POSTURE PATCH
To:
Most intimate 23
Least intimate
POSTURE PATCH
FUNCTIONING The Posture Patch comprises of four main components: 1. Adhesive clear silicone gel to enable flexibility and convenience 2. Skin mounted electronics – 4 muscle movement sensors, 4 vibrators (one in each direction) 3. A plastic encasing of two parts – a base embedded in the silicone gel, and 4. A cover to enclose the electric connections (battery, charger slot, circuit board etc.) The Keychain comprises of three main components: 1. Lightweight plastic body to encase electronics 2. Electronics (LEDs, charger slot, circuit board) 3. Key ring to enable portability
Plastic Encasing
Silicone gel embedded with electronics 24
CICA CARE GEL
Skin-mounted electronics
POSTURE PATCH
Sketches to understand ergonomics
Sketches to understand ergonomics
Final Prototypes - [MDF wood, CICACARE GEL, and paint] - Keychain; Posture Patch (Left to Right)
Posture Patch 25
Keychain
POSTURE PATCH
USER SCENARIO 1. The patch is worn on the back at the intersection of the nape and shoulders. 2. Through the day, the sensors sense muscle movement in four directions [East-West-North-South]. 3. The user receives haptic feedback about the posture change through subtle vibrations. 4. This data is cumulatively displayed on the partner’s portable device. (changes every 30 seconds) 5. This opens room for discussion between the partner and user. 6. The interactive system at home is activated when the three devices (the patch, the keychain, and the display system) are in close proximity. 7. The user and the partner together interact with one of the four possible display devices to better understand the user’s daily posture changes.
26
POSTURE PATCH
SCENARIO VARIATIONS
27
POSTURE PATCH
SCENARIO VARIATION 1. Mirroring: The user and the partner face each other a foot away. The partner wears the user’s patch. The posture data is compiled in the patch to observe the major areas of error. The partner then receives this data through haptic feedback. He/She then conveys the stress area/area of error to the user through actions. They both work together to fix the area.
28
POSTURE PATCH
29
POSTURE PATCH
SCENARIO VARIATION 2. Posture Clock: The user turns his/her back to the partner. The patch is embedded with directional sensors to detect the location of the keychain in relation to itself. The two devices work in sync while the keychain is moved around on the user’s back. . The colour displayed on the keychain would change when it is placed on a major error area.
30
POSTURE PATCH
31
POSTURE PATCH
SCENARIO VARIATION 3. Progress Clock: The patch and the keychain are kept on a flat table. The directional sensor in the patch would detect the location of the keychain on the table with respect to itself. The two would work in sync to display change in posture by the minute at any given time. The keychain would function similar to the hands of a clock. The colour change would represent the posture progress at the ‘time’ at which it is kept.
32
POSTURE PATCH
33
POSTURE PATCH
SCENARIO VARIATION 4. Texture Pad The texture pad would be activated when the keychain and patch are in close proximity. The keychain would wirelessly transmit the posture change data of the day to the texture pad. The latter would then display the data through change in texture on an hourly timeline.
34
POSTURE PATCH
35
POSTURE PATCH
FUTURE WORK We see the possibilities of this project in two parts,one to improve usability of the patch for posture correction while walking and other to make it adaptable across scopes. It could be implemented for everyday posture correction. We envision that the posture patch can also be used to improve athletic performance. It could be adapted to different postures in different sports so as to track changes and improve the game one step at a time. This system would also improve team spirit by bringing in the interaction to a larger group of people.
http://bhls.wordpress.com/2011/06/15/how-to-improve-posture-posture-exercises-to-correct-bad-posture/ 36
POSTURE PATCH
Google images 37
POSTURE PATCH
REFLECTION Chaitrali Bhide This project has been a great learning experience for me. I have been fascinated with the process of innovation design and was glad I got a chance to be part of the behind-the-curtains crew for the Beyond Screens project. I opted for this program with a clear mind, eager to know how one could naturally enhance a person’s daily routine. The fact that this course was going to be more about designing an experience rather than a product or system was a pleasant surprise, and added to my learning experience. This project has given me the opportunity to immerse myself fully in the process without knowing the final outcome. I learned how to generate fresh ideas each time the design space was modified Right from researching, to brainstorming (where several ‘Post-it’ lives were lost), to dirty prototyping and finding the suitable materials for the final iteration, this has been a great journey to understand the thought process behind making good design great design.
38
POSTURE PATCH
Sravya Bhavirisetty S.Labs has been a great learning experience in terms of understanding holistic design approach and also it opened to me a whole new world of using programming for prototyping. Something that was completely new to me was ideating through brainstorming, though initially I felt that all the crazy ideas would take us nowhere and some were even to bizarre to be true, but I saw how they can be narrowed to innovative realistic ideas. Through the course of the project, I learnt to let go of ideas, as difficult as it was to do it, looking back at it think it helped us get to much creative ideas and concepts. I learnt that as much important it is to keep in mind what the user wants, its equally important to know what you want the user to experience and feel when they are using your product.
39