Exploring Big Data The Ethical Side of a New Trend
Job Huberts Coach: Mathias Funk Assessor: Ronald van Tienhoven
Introduction Chapter 1
Index
3 Introduction Index Introduction Vision on design
FMP Proposal
From vision to project The design opportunity Context of public domain Orientation on Big Data Ethics in Big Data Exploring Big Data Curiosity as a tool Direction Methodology of project Explanatory movie Feedback & Reflection
Workshop China Goal of Workshop Co-reflection Workshop Influence on vision
Improving proposal
Review proposal Expert meeting with Joep Frens Reflection on expert meeting Expert meeting with Dirk van Mulligen Specified direction
Concept iteration
Scenario of Use Impact n society Where does the data come from? Communication structure Ethical side of concept Expert meeting with Berry Eggen Perception of traces Physical controls, why? Implementation of physical controls Suggestion system Form of concept Partners Video Exhibition
Closing
M2.2 steps Referencesw
Introduction “In order to bring back traditional social cultural values, I want to design interfering experiences in the public domain that remind people of their surroundings.�
4 This is a quote from my vision on design and society. During my Final Bachelor Project and the first semester of my master course I started to develop this vision in accordance to my identity as a designer that likes to explore the world. In my Final Master Project I wanted to come up with a service or product that aids people in exploring their world in order to find back social cultural values. In this report I describe the process from vision, via FMP proposal to concept. I start by describing how I envision the role of design in society and how this leads to the focus of my project: big data in the public domain. Following this I will discuss the current status of the context and topic of big data. In the first iteration I explored the potential of big data and how I can use this. The second iteration was performed in a workshop in China. The outcomes of these first two iterations were combined in the third and fourth iterations, leading to a concept that will be elaborated on in my M2.2 semester.
Vision on Design When looking at the society of today I am fascinated by the way our society changes and how it adapts to changes. Western cultures change due to technologic developments and an omnipresent fear for unknown threats. 5 Eastern countries are becoming a major competitor in the field of technical knowledge and production facilities since they are getting richer in knowledge and resources. Less developed, third world countries are influenced by this, which can be noticed when looking at their prosperity, but even they are developing at a rapid pace. In this global stream of developments, I see old (meaningful) values move to the background or change. Just dropping by for a cup of coffee is becoming less customary. Values like caring about each other, trust and social connections are interpreted in a moderner (digital) way. As I see it, relations get more screen based, flat. One of the causes of this is the rise of social media. I think that social media brings people closer together, but also pushes them apart at the same time since the communication becomes remote. The fact that people start to live alongside each other, can be seen in the increase of problems caused by social friction (Jensen, 2004). I, as designer, want people to leave their screens and start exploring outside of their own values and heritage.
To achieve this explorative behavior, I think the bodily skills of a person are of great importance. We experience the world through our body. Our bodily skills are more a criterion to experience the world than a tool (Sennett, 2008). This means that I see the body as a way of valuing the world and create value out of this. This connects to the theory that states that we learn to understand the world through our body (Merleau-Ponty, 1958). If we recover the value that our body has, we eventually will recover who we are ourselves. I interpret this world as the environment and society we currently live in. To achieve explorative behavior, I envision the design of interfering experiences that can work on both a personal and collective level. For this I see a role in open system design that provides the opportunity to create connections between people and systems. Open systems refer to systems and services that can adapt to changes in the devices connected to it. I hope to achieve this on different levels, from the domestic to the public domain. In the public domain I want to establish a personalized interaction that is part of the bigger environment. e.g. a one-on-one interaction that perceives and anticipates on the environment.
FMp Proposal Chapter 2
From Vision to Project For my FMP, I wanted to put my vision into practice in order to test it in a realistic context. I envisioned to create a service or product that has market potential, and thus can be developed further after my graduation.
7 In the first orientation phase of my FMP I came across the Growing System project in Out of Control. This project emphasizes inter alia on the potential of urban areas and how this can be explored with the help of big data. It hints in the direction of big data and the application of phenomenology, two concepts that appeal to me as they focus on the changing world we live in and how this world develops. In this project I wanted to come up with a concept that shows the potential of urban space to people in a physical way, e.g. discovering deeper layers of society. As a focus group I wanted to use the behavior of young people as a reference. Are they aware of the big amount of data that is gathered and do they understand the potential of the public area? In this I wanted to emphasize on cultural characteristics and technical developments. I hope to create a societal relevant system that is integrated in existing social structures.
The Design Opportunity Data is an interesting phenomenon, in fact, everything is data, but has a different appearance. The use of the term big data is a big trend. The buzz word big data does not refer to one particular concept.
8 It can refer to a data set that is too big to manage with regular computers. Or it is seen as not focussing on the quantity of data, but on the tools that can show patterns. Another definition states that big data is the process that helps the planet grow a nervous system. For me, big data is somewhere in between the first two definitions. Big data is a large amount of data that gives us the opportunity to discover new patterns in our existence and that of the planet. Well known sets of big data are the databases of Facebook and Twitter. Here, almost every detail of your life can be stored, and will be stored. Companies can use it to target advertisement better, governments can use it to gauge the mood of their people. But big data can also be the information about (data) traffic around the world or a record of all rainy clouds above the Netherlands. Combining these big data sets can lead to new insights. Insights in information that nobody has ever contributed directly and/ or explicitely.
This is were the principles of ethics become evident. People provide personal information to services they use online or offline. When used only for this service, the data is pretty harmless, but when combined with other sets of big data, new associations may pop up. The current generation of teenagers is a generation that always has been and will be in contact with computers. This generation is also related to as generation Z (R.C.W. Boender, 2011). In my opinion there are big opportunities in showing people from this generation the ins and outs of big data. I want to go past the stage of awareness and change peoples life. How? I will explain this during my first iteration.
Context of Public Domain As stated already in the Growing Systems project brief, the city around us is coming to life in the digital world. First examples of this can be seen in the use of applications like Layar and FourSquare. People start to use their geographical location as input for screen based interactions. 9 I want to go further in this. The public domain houses way more information than just two coordinates. In the public domain, people are living together, they influence each other and their environment. By moving through public space, all kinds of data are gathered. I see potential in using this data to create curiosity among young people in order to awake explorative behavior.
Orientation on Big Data Big data is a buzz word, due to new computing technologies the applications of big data are becoming more diverse. When recapitulating on the various definitions of big data I can say that big data is in fact a new way to measure and sense life. 10 Big data is as much about recognizing relations and patterns as about data (Gardner, 2012). The patterns arising from big data can give us insight in human behavior and the development of society. We are making a shift from a form of science where we make assumptions to a form of science where we “know” things (Enriquez, 2012). The things we “know” are abstracted from the vast amount of numbers that makes big data big. These patterns still have a quite abstract character and are hard to interpret for laymen. If we want to prove that big data is becoming our world wide nervous system (Dumbill, 2013) we first have to translate it to a level where people will understand what the influence of big data is. Here I see an opportunity to teach people the influence and potential of big data by providing examples and experiences in everyday life.
Representing data in daily life can be done is several ways, be it digital or physical. Services that are provided already try to visualize data mainly in a digital form. There are applications that give you an overview of your energy use, your health and social connections (e.g. Toon by Eneco, Nike+ and MyHeritage. com). But all of these are digital, on screen, representations. Translating data sets to physical experiences requires a clear correlation between the data and the way its physicalized. Currently, most of these data visualizations take the form of an interactive piece of art. Examples include the Next Nature data fountain and the sculptures by WHITEvoid. These installations tend to focus on one particular data set, just like most 2D data visualizations. I want to create a system that uses multiple data sets and correlates those to the user personally. This makes it more than a physicalized data visualization, it makes one experience the impact of big data sets.
Ethics in Big Data When discussing the personalization of big data sets, the ethical side becomes increasingly important. By combining two different data sets, a third data set is created. This set can contain data that was not available before, created out of parameters from the two different data sets. 11 For example, recently researchers showed that based on digital traces (in this case Facebook likes) intimate personal attributes like sexual orientation, religion or IQ could be determined(Kosinski, 2013). Information that we did not know or share before, becomes public without our specific permission. People are mostly unaware of this way of correlating data sets to acquire information. “While people might know that information is being collected about them, knowing this and knowing that the data will be analyzed for various purposes are two different things (LaBossiere, 2012). LaBossiere points out that privacy does not have to be violated; “If I get the secret ingredient in someone’s recipe by sneaking a look at the recipe, then I have acted wrongly. However, if I infer the secret ingredient by tasting the food when I am invited to dinner, then I have not acted wrongly. “ This is the point that I think people should be conscious of.
Exploring Big Data At this point I had two goals in this project; make young people aware of the ethics around big data, and as stated in the beginning, make people explore the world. These goals are visualized in the info graphic on the right page (figure 1.1). 12 I see two main development opportunities for this, one is a service (e.g. learning method or campaign) the second a physical intervention (e.g. a personal device or an interactive installation in public space). The underlying layer contains a theoretical framework of phenomenology and ethics, on which the concept will be based. This is visualized in the info graphic. Besides that the graphic shows how data is transferred from user to the cloud, which can also be accessed by companies. Based on this graphic, ideas arose covering the two mentioned goals. The conclusion of this was that I wanted to create a tool service for generation Z that lets them explore their environment, in order to gather data and assimilate these data. By letting youth quantify their living environment, explorative behavior can be promoted and children learn to value data.
13 GOAL Bring back traditional values by physicalyzing ethics of data
Learning method
Service
Government Campaign
Generation Z Cloud
Physical
Interactive Installation
Personal Device
Framework
Theoretical Framework _ Phenomenology _ Big Data _ Ethics
Adaptation
Open System Market
Society figure 2.1: possible directions of FMP
Public Domain
Curiosity as a Tool The trigger for this explorative behavior can be curiosity I think. Youngsters’ desire to learn is triggered by curiosity (Perry, 2000). The pleasure received from discovering and exploring, fuels this curiosity. 14 Since curiosity is the mean in this process, exploration becomes the final goal. Sharing discoveries can lead to admiration from external parties. The curiosity cycle (figure 1.3) of Jonathan Mugan (Mugan, 2012) shows this in a more abstract way. Children individuate concepts from the environment, and they use these concepts to build models. By testing these models they can see how well they predicted the environment. Through this iterative process, children will learn new models and concepts. In the graphic to the right (figure 1.2) the process of passing the stage of awareness in order to reach the goal is visualized. Curiosity is the catalyst in this.
Values _ privacy _ social connections _ understanding
Awareness
Curiosity Goal _ change the world _ make people explore _ develop values
figure 2.2: going from value to goal
15
Concept Individualisation
Model Testing
figure 2.3: Mugan’s curiosity cycle
Model Building
Direction The idea that I want to focus on in the next iterations is a service tool for youth of generation Z that awakes their curiosity to explore their living environment by gathering data. This can be established in different forms, it can be a toolkit with multiple recorders or a looking glass that shows digital traces. 16
figure 2.4: sketches of toolkit
Methodology project The first iteration round was characterized by research in literature and expert meetings. In the next iteration rounds I want to adapt a more explorative approach. I want to involve people from the target group in cooperative workshops. 17 The first session I will do during the trip to China. In this session I want to investigate the view of other cultures on the rise of big data. The Chinese digital infrastructure is significantly different from our Western infrastructure. In Coreflection sessions with Chinese students I want to confront them with my views on the subject and try to extract ideas out of their point of view.
Explanatory Movie During the theme days there was the opportunity to get feedback. Presentations were done by showing a 2 minute movie. For this occasion, an animation was created to explain the direction of my FMP. This animation illustrates the process from interest in big data vs. the public domain to the envisioned direction. 18
Growing Systems
Vision On
Design
Design oppor tunity
Reflection & Feedback During the FMP approval meeting, the before mentioned vision on my FMP project was discussed. The independent approver, dr. ir. Jean Bernard Martens, expressed his concerns about the broad topic of big data. He advised to find critical sparring partners for the project and involve them in the process. 19 This iteration round was mainly used to define the boundaries of my FMP. In the next weeks I want to specify the exact direction within the field of big data traces in the public domain.
Workshop China Chapter 3
Goal of Workshop To be able to step away from the project and acquire a new sight on cultural influences on design I decided to join the Growing Systems workshop in Taicang, Shanghai.
21 Second to that, the subject of the workshop was close related to my Final Master Project (FMP), bringing underlying layers of public space more to the surface. The proposed tool for this was projection mapping. Projection mapping is a technique that can be used to turn objects in video projection screens, often creating illusions around the movement of the actually static object.
Co-Reflection In order to get insights from students with a blank mind on the subject, a Co-reflection session was done with Chinese students in the workshop. First the initial direction was explained with the movie created previously. Information was only given about the direction and field of interest, not yet about concepts. 22 During the second step, the Chinese students were asked to come up with ideas concerning big data and how to create awareness around this. First striking point was that in Chinese culture it does not appear to be an issue that bulks of information are stored in the cloud. The students indicated that for them it was quite normal that the government did know about their whereabouts. When asked to come up with ideas that used this knowledge, ideas came to the table involving the use of their personal information. Public space was used as a projection canvas for projecting pictures when people passed by. Interaction with these projections was created by bringing in game elements of traditional Chinese games. Third step was to present the concepts created in the first iteration round. The main idea in this was to make people perceive the digital traces they leave in the physical world. In the following brainstorm, this idea was combined with the ideas of projecting personal information on buildings. What came out where ideas of projecting personal information on the floor when one passes by. In this way, traces of data would be created.
Workshop The goal of the workshop was to create an “interactive patina of cultures�. In this, patina refers to an interactive object that changes over time due to use. Both in appearance as in interaction. The context of use would be the public space. 23 Tools to realize this patina were to be sought in the use of projection mapping. Projection mapping provides the opportunity to show characteristics of an object that otherwise could not be seen with the human eye. In the brainstorms ideas from the coreflection session were used as starting point. While brainstorming, the context was narrowed down to parks where people come to hang out. The goal was to create an installation that provides a playful interaction while passing through public space. During early presentations, concepts were presented that involved traditional Chinese children games combined with projection mapping. People had to look for particular colors, projected on trees, benches and bins. By tagging those objects, their color would become dominant. People would be triggered to gather as much objects as possible, making an exercise activity out of this game.
24 During following iterations, the concept of tagging was developed. A change was made from traditional children games to a traditional religious object, the praying wheel. Praying wheels are commonly integrated in for instance public parks. By adding a layer of projected visuals on this, the interaction could be given an extra dimension. By touching the wheels, people will influence the projections. Wheels surrounding the touched wheel will accordingly light up to suggest which wheel can be touched next. During feedback sessions, the remark came that it can be interpreted as impolite to approach the concept of the praying wheels in this way, since it is a worshiped object. To finalize the concept, the interactive patina was evaluated. The direction of giving new values to existing traditional objects was still considered interesting, however the object itself had to be changed. While visiting a museum about traditional China, we got intrigued by old Chinese musical instruments and the fact that they are hardly used anymore. One of those intruments, the biānzhōng, jumped out due to its size, but also opportunities to use as projection canvas. The biānzhōng used to be an exclusive instrument for the upper class. This provided an interesting opportunity to bring it to the lower classes
in a new interactive appearance. A benefit of the biānzhōng was that it was made of bronze, a material that wears nicely when used over years, creating a patina. The principle of an interactive patina is that not only the material wears, but that the interaction wears too over time. For this, the interaction of hitting bells had to be slightly adapted. A look back was taken towards the praying wheels, that send prayers in the air when rotated. This idea was combined with the characteristics of the bells in the biānzhōng. When one played a melody on the biānzhōng, this melody would be recorded. Over time, this melody would be played when the wind played through the bells, but could also be used to suggest melodies for people that had less musical capabilities. These suggestions would be projected on the bells when one of the bells is hit. To explain and test the concept, a life size model was built. When one hit a bell, the bell would ring and a pattern would be suggested to create a melody. The model used computer generated sounds for the bells instead of real bronze bells. Projected suggestions were based on the twelve sings from the Chinese constellations. This was done to give people an image to relate to.
25
Chimes
An interactive patina of cultures
Influence on Vision The two week workshop did influence my vision on how design influences our daily lives. In Western civilization, the threshold to adapt new technologies is higher than in China. In Chinese daily life it could be seen that technologies are adapted at a faster pace, causing the rapid speed in which China is developing. 26 On the other hand, old traditional values still played an important role in Chinese public life. Respect to older people, helping each other and having dinner together were cherished more than in our Western culture. Chinese people have a bigger tendency to live in public space. I think this has an influence on how they live together and threat each other. This more traditional approach to life, combined with the continuously developing technologies creates a society that adapts more efficient to technologies in some fields. While focusing on music and how it plays a role in society, I got triggered to start looking at the potentials of music in public space. Sound in general is almost everywhere and influences peoples behavior. Music is an art form that plays a role in the lives of lots of people. Several data sets are created and exist concerning music.
Reflecting on this workshop, I can say that it helped me well in developing a concrete direction in my FMP. Working on a concept that involved music, I experienced the universal language of music. This inspired me to put the emphasis on music and start to analyze the role music plays in the public domain. People listen to music everywhere, leaving trace of music as they travel. In the next iteration this direction of left music traces will be elaborated.
27
Improving Proposal Chapter 4
Review Proposal In my FMP proposal I stated that I wanted to create a tool service for generation Z that lets them explore their environment, in order to gather data and assimilate these data.
29 By letting youth quantify their living environment, explorative behavior can be promoted and children learn to value data. Besides this, I see an opportunity in promoting societal collaboration in a personal way with this service. According to the feedback, the direction has to be specified and sparring partners had to be found. First step to specify my direction was to find a sparring partner or client. For this I contacted Joep Frens, one of the proposers of the Out of Control Growing Systems project, to discuss with him the envisioned direction of the project. Besides Joep Frens, I contacted Dirk van Mulligen again. He is an author and speaker on the human factor in technological developments. He has experience in setting up programs around the handling of big data and the ethical side of this.
Expert meeting with Joep Frens The second step was to meet experts. In China, I discussed the Growing Systems project with Joep Frens and how this relates to my project. 30 We discussed the parallel between digital traces and physical traces that I tried to draw. Joep Frens stated:“Digital traces differ from physical ones in the sense that they do not automatically vanish and are easier traceable.“ This difference in characteristics has to be taken into account. To create awareness around big data with help of a tool kit does not imply that a change in action is envisioned. In our discussion it became vital that the project should aim at changing behavior on a long lasting scale. The aimed change in behavior makes people take care of their data in future events. Generation Z should realize that they do not want their information spread all over the internet, instead of being told that they should not want in. I hope people will act on their awareness and so a comfortable environment for changing behavior will be created. In this way, people are not covering up their traces from the past, but will consciously change the way they create traces.
“In order to facilitate this change in behavior, a difference has to be made between explicitly and implicitly created traces. For each one, a different strategy is required in order to establish a change in behavior. You should supply guidelines for people to hold on to in order to change their behavior.” As said by Joep Frens. I see an opportunity in creating a system that is open enough to anticipate on these changes. The direction that I see at this point is to offer people the opportunity to discover new music in public space. I want to give people the opportunity to follow tracks of music, left by other people. The traces are left implicitly by people listening to music while commuting. The moment people decide to follow a musical trace and start exploring, the actions become explicit. In this way people come in contact with music and geographical places that do not have to fit their initial taste. This differs from how music is suggested at the moment, based on what people already like.
Reflection on expert meeting As mentioned before, the focus should be put on creating a change in behavior instead of covering up actions from the past. 31 A toolkit may be not the right way to do this, since it implies that there is room for own interpretation. In a discussion with my coach Mathias Funk, I concluded that the data sets needed for this are the databases of social networks. They possess personal data, the data that I want to use. These data sets are accessible and can be used to pull conclusions. In this meeting it also came forth that it would be wise to focus on my personal interests to find a direction in the social media databases. For me, music is an interesting subject. I see opportunities in making people discover new musical styles and music related places in their environment that are out of their comfort zone. I experienced this during the workshop in China, where music formed a mean to communicate cultural differences, but also was a way to deploy explorative behavior in an interactive installation.
Expert Meeting with Dirk van Mulligen In a meeting with Dirk van Mulligen, the before mentioned direction was discussed.
32 Dirk van Mulligen agreed on the fact that suggesting new paths in ones musical taste will trigger curiosity. By converging suggestions, like is happening at the moment, explorative behavior is cut off. He sees this approach in a broader context where it could also be applied to other markets like online shopping and holiday destinations. The unexpected element in this interaction offers leaps forward in getting to know new subjects. Dirk van Mulligen sees potential for this system to avoid polarization. Dirk van Mulligen indicated that he wants to be involved as a sparring partner and expert on the role of data in our society, for my FMP.
Specified Direction The initial connection between physical and digital traces has developed into a connection between digital traces left in the physical world and discovering this physical world. Music is the connection I see in this.
33 The initial connection between physical and digital traces has developed into a connection between digital traces left in the physical world and discovering this physical world. Music is the connection I see in this. The idea to visualize traces flying through the air adapts to this, the traces will be bits of music tracks that remain behind when people pass by. Imagine, virtual beams of light floating in the air. To discover new music, one can simply tune in to a different trace, follow it, make it stronger and leave it. Throughout the whole city a network of songs will be created which provide paths to explore. As stated before, the traces will be left implicitly, the following of the traces will be an explicit act. Data will be gathered
from services like Spotify, Last.fm and Facebook, combined with GPS data. With this data it can be tracked what music is listened on which routes. To acquire this data, I want to contact the relevant parties to see of they can aid me in getting the right data. The values of privacy and ethics changed to values of sharing and exploring. Although, the underlying layer still concerns the ethical side of the way big data is made open. The change in action can be found in people consciously changing their behavior by starting to follow new traces and exploring public space. In the next iteration, this concept of following music traces has to be developed and specified.
Concept Iteration Chapter 5
Developing Concept The current concept consists of musical traces, figuratively drawn in the air. People can tune in to these traces. By following traces, people discover new music, but also new places. Curiosity of people is triggered since the destination of the trace is unclear and could lead them anywhere. 35 People start making assumptions about the trace, curiosity will trigger them to start validate this assumption (Mugan, 2012). Leading to a more explorative way of moving through public space. In fact, this service, brings a currently invisible layer to the surface of the city. The soundtrack of the city covers the existing structure of buildings, roads and parks. Music venues can use it to communicate their program, when one passes a venue, a cloud of upcoming artists can intervene. This kind of radio station can also emerge at different spots. If a specific type of music is played significantly more than others, a music cloud can form in that neighborhood. These are just examples of the opportunities of making the audible layer of an urban area observable. To be able to work out this concept, some ambiguities have to be solved first. How are the traces visualized? Can they be seen, heard or felt? How does the user navigate between traces? Which data sets are involved and how are they combined? Which partners would be involved with this and what would be their role? These, and more, questions had to be answered in this final iteration round of this semester.
Scenarios of Use In order to be able to place the use of this concept in context, scenarios were created that illustrate the use in daily life.
36 #1 One is traveling to school, on his way he can discover new music by switching tracks that he comes across. new routes can be discovered by following traces from others. At the same time, he will leave a trace when listening his own music and enforce a trace when following a trace. #2 One is curious to explore its surroundings, the threshold to do this can be high without a trigger. By starting following traces, you are lead through the city in a way that you might discover new places. #3 Not everyone is aware of events that are going on, by creating clouds over the traces one can discover this by just passing by the location. The music the person is listening to and thus creating a trace with is temporarily overruled by the broadcasted music.
Impact on Society Taste for music is different for each person. It can be perceived as a small stand alone biotope. Music taste is influenced by several factors, and differ per person.
37 Some people do not have the habit to actively look for new music, their biotope is only influenced by external factors like radio and television. Other people are more active in discovering new music. Their taste is more influenced by external factors, like other people, recommendation services besides radio and television. With this concept, the biotopes of people will be influenced constantly in an implicit way. People will be able to discover music outside their comfort zone in a physical explorative manner. The creation of traces will be something that is done in an implicit way. By listening to music while traveling from A to B, traces are created and left. The decision to step out of your comfort zone and start expanding your biotope is an explicit decision. People choose consciously to follow a trace and when to leave it.
Where does the Data come from? In order to create and use the digital traces that people leave, several data sets are needed.
38 First a database is needed which records the tracks someone is listening to. A second database is needed that records the GPS location of a person. This GPS database has to store the location of a person, linked to the time. For the music database several music services can be addressed. At the moment there are a few big services that provide streaming music on the go. The one that is most used is Spotify, smaller players are Deezer, Rdio and last.fm. The focus is put on Spotify at the moment. Spotify provides tools to create own applications for both the desktop Spotify app and mobile systems. With this API, playlists can be created in an accessible way. In these playlists the songs people listen to on a route can be stored. If this is combined with GPS data and a timestamp, a route of songs would be created. To document the location of a user, the GPS function of a smartphone has to be used. Since the person already used his smartphone to play the music, it is easiest to use the GPS tracker in this phone. This GPS function has to be called in the application that has to be created. The app would use its own GPS data and a playlist provided by the Spotify application.
39 Playlist Info & GPS Data
Spotify Database
Geotagged Music Playlist
Smartphone Application
GPS Data of person
Smartphone & Music Player figure 5.1: flow of data. GPS data and music data feed the system
Communication structure In the system, people will have two different roles. In the first place they create (creator) traces by listening to music when they are on the go. At the same time they listen (listener) to traces left by other people.
40 But, what follows is a chicken and egg situation. People can not start following traces if there are no traces left yet. This could be solved by making the app work as extra layer over the existing Spotify application. In this way people can listen to their music as they normally would do and would be notified when traces appear. In further development, this strategy can lead to some obstacles. First, an application has to be created that stores Spotify playlists, this has to be done with the Spotify API. Alongside to this, an application has to be created that gets and returns GPS data. This GPS data has to be retrieved from the GPS sensor in the phone and returned to the Spotify playlist application. Critical points can be encountered in communication between applications and communication with Spotify.
41
Listens to Traces
Listener
Gets Data
Creator
Application
Leaves traces
Exchange Data
figure 5.2: a user has both the role of creator and listener
Public Space
Ethical Side of concept As stated in the FMP proposal, the initial goal was to create a change in behavior among young people. This to make them aware of the ethical side of big data.
42 This concept uses the locations and music libraries of people to make the digital traces we leave perceptible. In this way the creation of awareness is not the main purpose anymore, but has become an integrated layer within the act of perceiving surroundings. Looking at the concept, there are several possible issues that could show up concerning ethics. There are several occasions in which privacy is compromised. By leaving traces, it becomes easier to stalk people. This could result in unwanted visitors. A solution for this would be to automatically switch of the app within a certain range of places like home and work. Privacy could also be an issue when looking at the contents of the traces left. Switching on the app would be a conscious decision, however it could occur that music passes that one does not want to be public.
Something else that could show up concerns the commercial exploitation of the service. Since venues can create radio clouds and traces will be recorded, shops can use this in their advantage. By influencing the music played around the store, commercial messages could be streamed in the air round a specific shop. This can be an opportunity to, for instance, communicate special offers. But it could also provide an unwanted layer of noise.
Expert Meeting with Berry Eggen To discuss the concept of following music traces and the influence of sound Berry Eggen was consulted.
43 After discussing the global idea of leaving digital traces and using them to discover both music and locations we started to dive further in the potential of this concept. Going out with getting lost as sole goal is something that won’t happen that much. But emphasizing on the connection between location and music has strong potential according to Berry Eggen. For example, when one walks towards the train station, he can pick up the traces of playlists that other people listened to on this trip. The same could go for riding your bike through the city or on the country side. An interesting opportunity would lie in a system that remembers special songs at certain locations. If one has the tendency to play a special song when he enters the highway for example, the system could automatically switch to this.
The service could also be given extra functionality as an open-ended navigation system that makes you get lost. A specific destination would not be needed, it would be more global, like a skate park, mall or Korean restaurant. By indicating this, the system would compose a route that follows paths of music that suit the destination (e.g. Punk for a skatepark or K-pop for a Korean restaurant). The destination that is chosen by the system is shrouded in ambiguity. Changes are that the system will select a different route next time, but leads to the same destination.
Perception of Traces Key element in this concept is the perception of musical traces. How does one perceive something that maybe does not even exist? The original idea was to make the traces visible, to create traces that magically form and fly through the air. 44 Technologies that would have to be used for this (augmented reality) are or really expensive (Google Glass) or not usable in a continuous way (Layar). The concept of Google Glass however, is interesting for this purpose since it adds extra layers of information on the real world while one is doing his everyday things. For traces of music, it would fit better if the way traces are perceived is integrated in the music playing device and/or headphones. This takes away the glasses like object, but the opportunity to perceive hidden layers is still there. Traces could be perceived through changing sound (panning, volume change) or feeling (vibrations, temperature changes, pressure). At this point, the layer of musical traces that lays over the city is static. It consists of fixed GPS locations with songs attached. The layer can be perceived by the user, but the layer does not perceive the user. If the layer would perceive the user, it could provide a more dynamic and curiosity awakening interaction. Suggestions could be made based on known behavior of the user, and certain traces can be adapted for example. If the user perceives that the system perceives him the connection between user and system will be stronger.
On body Device
Smartphone & Spotify App
Spotify Database figure 5.3: devices in system
45 Creator
Leaves traces
Perceptive Space Of Dynamic Music traces
Create Common History
Listener
Gets Data
Application
Exchange Data
Public Space
Form one Coherent Entity
App + Traces Display Curiosity Awakening Behavior
figure 5.4: layer of perceptual crossing integrated in system
Physical controls, why? When moving through public space, it can be noticed that a lot of people listen to music while walking, riding bike or in a car. This creates multiple digital traces at the same time.
46 It could be that the hall of train station is filled with music traces within minutes, creating a tangle of traces that are not distinguishable anymore. In order to navigate through these traces and select traces to keep playing, controls have to be added. To avoid a screen based interaction, which would distract from the traces, it was decided to investigate the possibilities of a physical control system. As stated before, the body is a tool to perceive the world, that is why the emphasis is put on this way of control. This can vary from gestures around the body to playing with beads attached to the earphone wire. As a placeholder interaction, it was chosen to use movement of the head as controls. In a next iteration this can be reconsidered.
Radio Like knobs on belt
Leap Motion Plus On belt
Turnable device on belt
figure 5.5: concept sketches
47
Moving objects on wire
Turning of head
Moving with wire
tape to scroll through traces
Moving beads on wire
Turn discs on Headphones
Implementation of Physical COntrols In order to be able to control the music one is listening to, several controls have to be provided.
48 One has to be able to select a trace, which has to be confirmed to start listening to it and stick to it. While listening to a trace, one has to be able to deselect the trace and switch back to his own music or an other trace. If an interesting tune is discovered, there should be a possibility to stick to this tune, no matter what there will be suggested. In a co-reflection session with fellow students this interaction was elaborated. Students were asked to act out the previously mentioned controls and asked how they imagined the traces to be like. While 5 out of 6 student immediately referred to Google Glass like applications, they ended up with scenarios where the traces would behave more like tangible walls or tunnels. By breaching through a wall another trace would be entered.
The way they would control these traces is summarized here: _ Discover traces: look around by moving head and thus moving line of sight _ Follow trace: consciously follow a trace, when the trace takes another direction, this will be indicated by a panning noise, that covers the music on the opposite side. _ Leave trace: consciously step away from the trace, ignoring the noise that will be played. _ Stick to tune: a nod of the head to approve tune. As said in previous chapter, this interaction scheme will be subject of change in the next iteration round when the final form of the concept will be defined.
Suggestion System Since there will be a lot of musical traces flying through the air, a division has to be made between traces, as stated before.
49 First step is to give the user the opportunity to select traces. But as the perceptive system gets to know the behavior and whereabouts of the use it could make a pre-selection. This selection will not be based on music the user already likes, this would result in people discovering particularly music that they in fact already know. The selection would be based on other parameters, like speed of travel, routes taken and other decisions made earlier while using the system. In this way traces are recommended that do not per se match the music taste of the user, but maybe some of his other characteristics. Like discussed in the meeting with Dirk van Mulligen, this could also be interesting for other commercial fields. Choices for nutrition, holiday or cars can be based on other characteristics of a person.
Form of Concept The concept as it is now consists of a smartphone application with an add on providing the physical interaction.
50 The smartphone application will be the core of the final product. Other objects like a headphone, smartphone cover or special T-shirt could add value since they supply room for sensors. This is in fact similar to the way Nike+ puts itself in the market. With the app alone, most of the functionality is available, by buying extra products, the experience becomes more intense. In this case this means that with the application it already is possible to navigate through traces (with an on screen interaction), but by adding accessories the experience becomes complete and more functionalities (e.g. motion control) unlock.
Partners As I stated before, I want to develop a market ready product or service during my FMP. In order to get a realistic view on the market possibilities I want to consult partners in my project and hopefully connect to a client. At the moment the following partners play a role in the project: 51 _ Dirk van Mulligen: Founder of Coherent Solutions. I came in contact with him via Joep Frens, based on a previous project that was done together about creating awareness around ethics in data, rQive. I will try to use him as a counselor on the ethical side of my project. _ Berry Eggen: Berry Eggen is an expert in the field of audiology in design, besides that he has an interest in the way music plays a role in our life and public places. Berry Eggen has indicated to be available as sparring partner and expert concerning the use of audio to influence people.
_ Hout & de Wolf: Currently I am founding my own company together with another student, Mitchell Jacobs. The emphasis of this company lies on creating educational methods that teach young people about developing technologies and cultural changes. I want to use this company as development partner during the project and if possible as a company to put it on the market after the project. _ Spotify: Before my M2.2 semester I want to contact the Spotify software development team to see if there is a possibility to develop the concept further in a cooperation. I will look for the appropriate person to apply for this. Besides that, I will use the API provided by Spotify to develop my project.
Video The concept of non visible traces that float through the air can be quite abstract. In order to make the concept better understandable a movie had to be made that makes the concept perceptible. By using real life footage and adding fictive traces of music to it, people could better understand and imagine the concept. 52 A movie was shot in the shopping streets of Eindhoven. This was done with a steady cam to create a first person perspective. The movie was edited later with Adobe After Effects to draw virtual music traces in the air following people that listened to music. Traces were shows mainly in the beginning of the movie to explain how a trace would behave, later
in the movie, these traces are left out, to create a more realistic experience. By panning the music separately on the left and right channel, the direction of the music was indicated. At spots where the first person view look started to stray from the path, the right direction was indicated by panning noise.
Exhibition At the exhibition, the movie was used to explain the concept to visitors. By putting on headphones, the concept could be experienced.
53 These short sessions were used as user validation. It became clear that people did understand the concept when experiencing the movie with stereo headphones on. However, in order to be able to state that it works, this has to be supported by a user evaluation using quantitative and qualitative methods. Questions arose about the physical
interaction, the way it was used now can be made richer. Besides that, people wondered how the creation and decay of traces would take place. What is the time a trace remains and how or when will it possibly evolve into a cloud or radio? These are very relevant questions to focus on in the next iteration, during my M2.2 semester.
Closing Chapter 6
M2.2 Steps My goal was to come up with a concrete concept at the end of my M2.1. This concept would be further developed during my M2.2. Some aspects that were planned to address this semester have to be developed more during next semester. 55 _ Interaction. The way people interact with the service has to be elaborated. This will be done in creative sessions with people from generation Z. The focus will be on the physical interaction between body and service. _ Form. The form of the concept is closely intertwined with the interaction, these two aspects have to be developed parallel. The form can vary from a stand alone application to an application supported by a range of products. _ Working. The exact working of the concept has to be defined. The behavior of the traces has to be fixed. This involves the rise and decay of traces, but also the role of music clouds and radios put in the air by third parties. _ Integration of data sets. To prove the working of the concept, a working prototype has to be developed. This will be used to test the working with people from generation Z. In this prototype, the several data sets have to be combined. _ User validation. With the final prototype, user validation sessions have to be conducted in order to see how the service impacts daily life. Conclusions of this have to be used to develop the final concept.
_ Commercial side of concept. As noted before, there are some interesting business directions attached to the concept. From the start of next semester these will be developed. For this I want to consult Ad van Berlo as an expert. Together with Ad van Berlo I want to roadmap the business model of my concept.
Planning M2.2 To direct the process in my M2.2 semester, I made a week to week planning for the semester. In the planning below the main topics per week can be seen. These topics can still be shifted if some turn out to require more time. 56 36
Review concept + Contact Partners
37
Design Physical Interaction & Co reflection
38
Prototyping for Validation
39
Prototyiping & Testing & Validating
40
Review Tests & Discuss with Partners
41
Finalize concept & Design Prototype
42
Elaborate on Market Opportunities
43
Prototyping & DDW 2013
44
Prototyping
57 45
Programming of Prototype
46
Integrating Software and Hardware
47
Prototyping & setup User Test
48
Testing & Validating in Context
49
Developing Commercial Opportunities
50
Create market ready concept
51
Report & Buffer
52
Preparing Exhibition
1
Assessments
References
58 Boender, R.C.W. (2011) Generatie Z, Bertram + de Leeuw Dumbill, E., (2013) Making Sense of Big Data, Mary Ann Liebert, Inc. Vol. 1 Enriquez, J., (2012) The Human Face of Big Data, page 9, ISBN 978-1-4549-0827-2 Gardner, D., (2012) The Human Face of Big Data, page 5, ISBN 978-1-4549-0827-2 Jensen, R.H., (2004) Designing for social friction: Exploring ubiquitous computing as means of cultural interventions in urban space, IT University of Copenhagen Kosinski, M., (2013) Digital Records could expose intimate details and personality traits of millions, http://www.cam.ac.uk/research/news/digital-records-could-exposeintimate-details-and-personality-traits-of-millions LaBossiere, M., (2012) Big Data & Ethics, Big Data, The Moving Parts: Fast Data, Big Analytics, and Deep Insight Merleau-Ponty, M., (1958) Phenomenology of Perception (C. Smith) Routledge & Kegan Paul Mugan, J., (2012) The Curiosity Cycle: Preparing your Child for the Ongoing Technological Explosion, page 6 Perry, B.D. M.D. Ph.D., (2000) Curiosity: The Fuel of Development, Early Childhood Today Magazine, March 2011 Sennett, R., (2008) The craftsman. Penguin Books, London
59
Discover the sound of the city. Create and follow digital traces of music in public space, left by you and other people. By using bodily skills like gestures to navigate through music tracks, new tunes can be discovered. Get lost in an unconstrained way by exploring the paths of people before you.
Job Huberts s070991 j.y.j.huberts@student.tue.nl www.jobhuberts.com