SJournal

02830 Advanced Project in Digital Media Engineering SJournal - a part of SensibleDTU Ulrik Uhre Brink s081830 mail@ulrikuhrebrink.dk

Kristian Michael Clarkson s072660 the.kc114@gmail.com

ABSTRACT

The main problem that our application will address is, that it can be difficult to remember certain events; such as where the user was last week and who the user talked to last night at work. This project will search to solve the problems concerning remembering places and interactions with other people.

Our application will try to solve the problem while allowing the user to go through the data by giving them a simple and intuitive interface and only present the most relevant data.

Combining gathered data and a great looking interface, we will be able to give the students at DTU a prototype of a mobile application, that will make it easier to gain insight about their places and interactions.

Figure 1. Vision of SJournal described in only 140 characters

Author Keywords

more and more interesting. Cultural trends shows that logging your every move is getting more socially acceptable[8] and with devices like the JawBone Up[15], FitBit[16] and mobile tracking applications like Moves, it gets easier to track yourself and therefore even more popular[9]. Life logging is about digging in to the data gathered and getting an insight of what and when certain events occurred.

Interface; Quantified Self; Android; User Centered Interface (UI); User Experience (UX); Lifelogging INTRODUCTION

The prototype application, SJournal, is part of the SensibleDTU program and in this project try to come up with a improved version of the existing Android application SensibleJournal[4].

Besides our motivation for data and self tracking, we have been working on a project in collaboration with Glostrup Hospital concerning people diagnosed with MCI[7]. These people are having trouble remembering daily tasks, and lifelogging would in a near future be an assistance to them. During interviews they have specifically asked for a way of logging their daily routines and help them remember earlier occurred events.

Our vision for improving on the existing application is to create a whole new way of presenting the same gathered data. In order to improve it, we will look at the market and see how similar applications are presenting their data. The prototype application will allow the users get an easy overview of their collected data and allowing them to gain more insight of the data. Since only students at DTU are allowed access to the data, it would be interesting to allow them to see who they interact with and in what context they interact. SJournal can be described in a tweet (140 characters) which is illustrated in Figure 1.

As we are working under a restricted time frame we will not be able to both design and implement a vertical prototype[11].

We have chosen to go in depth with, what we believe is the most important aspect, the user interface. To get a feeling of what kind of data is available from the SensibleDTU we will also develop a prototype application including a JSON-parser.

This will allow us to gather the most relevant data and test if it is feasible to solve the problem concerning remembering places and interactions with other people.

Our motivation for doing this project is our deep curiosity about gaining self knowledge of our activities through data collected from different sensors. The technology behind lifelogging has evolved the past year and lifelogging is getting

The report is build up around the prototype including related work which includes a market analysis on similar applications, methods on how we did the project, results from our work, discussion and evaluation on our results and a conclusion including future development.

1

RELATED WORK

Before beginning the development of a new application and to improve the existing SensibleJournal application, it is important to gain knowledge on the existing market of similar applications to see how they do it.

The following section will include an analysis of the current available applications and an evaluation on their strengths and main focus. This evaluation makes it possible to create an overview of what our application should include in order to improve on the existing SensibleJournal-application[4].

Related apps on the Android-market

There are currently several different markets for mobile applications under a variety of mobile platforms including iOS (iPhone), Android and Windows Phone. In this project we will be focusing on the Android-market, as this is our preferred markup language.

It is important to have a good knowledge of any competition when a new application is developed, which should take the best and differentiate from the rest. The following is therefore a review of the applications which are considered as the largest competitors in terms of the new application.

Figure 3. The timeline of people interacted with during a given day.

mark life events based on specific locations. In a map-view the user is able to get an overview of each day. The main focus is to track the users whereabouts without disturbing the user. The user is able to extend life-logging with data from i.e. social networks and calls. The timeline of interactions on a given place is illustrated in Figure 4.

Moves [1] allows the user to record and track their current activity i.e. walking, running, cycling and transporting. Moves can create a storyline for each day including the places the user has been and what type of activity. The main focus is tracking the activities between places. An example of the storyline and summary of activities is illustrated in Figure 2.

Figure 4. The timeline of people interacted with on a given place Figure 2. The daily storyline and maps show where, when, and how you move).

SensibleJournal [4] allows the student at DTU to get an overview of visited places each day and how long they have been there. Furthermore, it includes the time used on different activities i.e. stationary, walking and �on a vehicle�. The timeline is illustrated in Figure 5.

Friday [2] allows the user to get a complete log of i.e. places visited, people interacted with, music listened to, calls, emails and texts. All of these data is logged and will help remembering past activities, people talked to and events attended. The main focus is to tracks all possible activities in a lifelogging journal. The timeline of interactions is illustrated in Figure 3.

Summary of related work

Trails [3] is created by the same developers who also developed the Friday-app and allows the user to automatically

Based on our small market analysis of the existing apps we have become aware of what applications we would stand up 2

Use cases

Based on our core users found from the user-stories we have created a set of use cases. The use cases provide crucial information about the context in which the prototype application will be used.

Use case: The student Morten, 23, is really hungover from last night’s event and is trying to figure out what happened and who he talked to during his escapades. An application will quickly give him an overview of where the party started and follow its progress place for place in a chronologically order. On each event he can view who he interacted with both through texts and calls. Morten is now able to bind together a story of what went on last night.

Use case: The Elderly Jane, 62, has been diagnosed with a MCI[7] (Mild cognitive disorder), which causes her to forget new events. This is often apparent when going out shopping on her own and talking to her children on the phone. But after a few hours she forgets who she has spoken to. The children sometimes get irritated when she has phoned them four times in one day. An application will be able to keep track of who she has phoned, how long the conversation was and also where she was when the conversation took place.

Figure 5. The timeline of time spent at places and on activities

against if releasing the application. The following are the features that our application should include as the absolute minimum in order to compete on the market with self knowledge applications.

Use case: The life loggist Jesper, 41, is logging every event of his life and write down places he has visited in his diary.

This is a very time consuming task to do, and he tends to forget to do it every day. The days’ events is piling up and in the end of the week he can not really remember what went on at Monday. An application automatically tracks places he visit and now he can with go back in time with a few clicks.

The only problem is now, that all his old data is not as cool looking as the new collected data.

• a simple and intuitive interface • interactions with other users to some extent • a history of places visited and people interacted with METHODS

Our workload for developing the prototype is divided into several smaller parts. The following section will include the user-stories, use cases and the steps of developing the prototype.

Use case: The consultant Lars, 34, usually spends a lot of time on-site at different customers. Keeping track of where he has been and for how long, is manually written in an elaborate spreadsheet to keep track of billing hours. Just as well with long client calls which are also billable. Keeping track of all of this takes its toll on Lars and mistakes are bound to occur. An application which could log this data for Lars would greatly improve the billing accuracy to his clients, and save him from stressing about remembering to log the data himself.

User stories

The users are a crucial focus area when developing a usercentered application. Our core users needs and values should be taken in consideration when designing the interface and adding features to it, as they would be the end users to the application.

To be able to specify our users, we are making use of user stories and use cases, and based on these we will make sure that the included features in the prototype application will meet their needs and requirements:

Summary of our user stories and use cases

We became aware of our core users and in which situations they would use the application. We have extracted the following focus points. At this point only the students from DTU will be able to use the prototype application.

• As a student I can quickly check my timeline, so I can be reminded what happened and who I have talked to at the party last night.

• Where the user has been divided in days.

• As an elderly person I can see who I have talked to today, since I tend to forget.

• Who the user has interacted with and where he has interacted with them.

• As a life loggist[5] I can go back in time, so I can see earlier visited placed and check which places I visit the most.

• Simple and intuitive display which matches many user types from students, business people, elderly to enthusiasts.

• As a consultant I can check newly visited places, so I can track how much of my time is spend on which customer.

3

The core users have very different backgrounds, incentives and needs. However, they still have a common goal; Quantified Self[5]. Tracking the users movements and calls, displaying the data in a simple way will make it easy for everyone to understand and interpret them.

Prototyping

Throughout the development of the prototype we have made extensive use of user-centered design principles so the final application will satisfy and give the users a more user-friendly user experience - for a final product this will be the make or break on the market.

Mockup

Figure 7. Mockup 01 - CardsUI - ActivityCard

Following the principles of user-centered design we went through several iterations of mockups before finding the final design for our prototype. An iteration consisted of a concept which lead to a mockup, which was then tested and based on the evaluation and feedback we revised the mockup. Each mockup was evaluated on the students from the course and from their feedback we made improvement to the mockups.

The first mockup featured the use of the CardsUI[6] as this would give us opportunity to combine different types of activities. Each card would represent an activity i.e. a text, a phone call, a place visited and so on. The mockup is illustrated in Figure 6 and Figure 7. Our feedback from this mockup was that the interface would easily be cluttered when adding more than a few activities and the list would be very long.

Figure 8. Mockup 02 - ListView Colorcoded

Figure 9. The feedback we got from the focus group was that the mockup lacked a chronological timeline of the activities, but the listview was a great idea.

Figure 6. Mockup 01 - CardsUI

The second mockup featured an offspring of the first one and based on the feedback we got from the first one we decided to change the layout. To solve the cluttering and long timeline, as noted in the first mockup, we introduced the cards in groups and each card activity were assigned an unique color.

The mockup is illustrated in Figure 8. The feedback that we received was, that it was less simple to look at, and the users wanted more content on the screen instead of having to scroll etc.

Figure 9. Mockup 03 - Activities in a listview

In the fourth mockup we tested out a radical change in the layout, showing activity icons with content and a timeline between each activity. This mockup was intended to only test the timeline feature. The mockup is illustrated in Figure 10.

The focus group loved the the timeline between each activity which gave a good separation between activities. One thing they missed, however, was the listview from the last mockup.

In the third mockup we skipped the CardsUI included in the first two mockups. The new mockup introduced a list of the activities including their unique colors and this time also relevant icons. Skipping the CardsUI we made room for more content shown on the screen. The mockup is illustrated in

Based on the feedback we decided to settle for a final design for the prototype based on the third and fourth mockup. These 4

• Removing data points where the user spends less than 15 minutes which indicate that the user is on the move When these steps has been taken, only a handful of locations are presented per day. The SMS and call-log can then be gathered for the specific day and then divided out to the various locations.

RESULTS

Based on the feedback from both the focus groups on our mockups and the json-parser, we have developed two smaller prototypes each with its own purpose. The following section include each of these prototypes, a comparison against more comprehensible SensibleJournal and a user-testing on the prototype application, SJournal.

Figure 10. Mockup 04 - Timeline of activities

two mockups were the ones that the users were the most fond of.

Prototype

Based upon the feedback from the focus group on the early mockups and the needs from our core users, we came up with the final design for our prototype. The following are the features that we have included in our application.

JSON-parser

To make the design, the data has to be collected, digested and presented in a standardized way. The data is collected from an url containing the address, access token and parameters which define what data should be collected. The process of collecting the data from the SensibleDTU-database is visualized in Figure 11.

• a complete summary of the day • a chronological timeline – including places visited – including interactions with other people • a simple and yet intuitive interface with unique icons and colors • a timeline indicator In the top of each day we have added a complete summary of the day. The summary includes the total number of places visited and the number of interactions both on texts and phone calls. Below the summary is the timeline for the entire day.

The timeline includes each visited place throughout the day in a chronological order. Each place is given a unique icon which indicates the type of place. We have included an actual timeline indicating the time arriving at and leaving a given place. We have included the interactions with other people for each place visited, so the user can look up interactions at specific places. The list of every interaction could easily be a very long list and to solve this problem we have combined texts and calls based on the persons who the user has interacted with at that specific place. The timeline and summary in the prototype is illustrated in Figure 12 and the interactions between people for a given day is illustrated in Figure 14.

Figure 11. Gathering the data

To get the relevant data, this process has to be done three times. One for gathering the locations, one for the call-log and one for SMS. With the help of the API, the data is collected back in time when getting locations and only on specific date when retrieving call-log and SMS. Since Sensible DTU collects a lot of GPS locations every day the iteration of retrieving the next url is imperative to collect all of the data per day. For the SMS and call-log this is not a problem, it is our estimate that average people do not log more than a thousand texts or calls a day. The received format is json, which is then easily iterated through to get either next link or the results which are well structured. Every location data point contains an ID, TIMESTAMP and GPS-location that we are interested in. When these are downloaded the process of filtering out duplicate locations is comprised of the following steps:

JSON-Parser

The data from the Sensible DTU server was collected and interpreted with the implemented json parser. As Figure 13 shows, the filtering narrows down to a few 100 locations and later combining so only a few actual locations remains where a user has been for more than 15 minutes. When the calllog and sms were collected, they were manually connected with views from SensibleDTU[10] to see that data could be combined with a location and the timespand at that place of interest.

• Removing zeros from the GPS locations so accuracy is down to the nearest meter. The GPS coordinates have more precise points but with some inaccuracy, so they are not really that accurate, and unimportant this application.

• Removing data points within 10 meters. There will not be many points of interests within 10 meters than can be comprehended.

5

Figure 12. Summary and timeline

Figure 14. Interaction

focusing on locations and not movements, means occurrences like on a vehicle and walking is not in the scope as it is illustrated in Figure 15 from SensibleJournal.

Figure 13. Filtering tha data

Comparing SJournal with SensibleJournal 2013 and others

Figure 15. Interaction

The Initial approach was to improve some of the aspects depicted in SensibleJournals. The areas that have been focused on are locations throughout the whole day and how to make it more digestible for the user. By also adding call and text logs to locations, a more memorable experience will be gained since locations will be connected with interactions with friends or family. An example of this is shown in Figure 14.

Some of the design aspects were inspired by the various apps mentioned earlier. By making a timeline instead of a card per day the user will get a more specific insight to activities during the day.

With the proof of concept accomplished, data can be collected and digested. Static data was used from this to make sure we received consistent data to build the design. In this case the connection with foursquare[14] was not implemented, though tests showed that the GPS coordinates gathered from Sensible DTU, coincided with actual locations on foursquare. Mainly 6

User Testing

In order to test the final prototype we tested the prototype on different users. There exists a wide range of different ways to evaluate mobile applications through user testing. The following is a list of different testing methods which was considered used:

Figure 16. Snippet of Diary

• Benchmarking

Concerning control of the data the control-data from the diaries has been used to pinpoint the most relevant datapoints in the json-parser prototype, but as the main focus was to create a more user-friendly interface, the development of the dataprototype was discontinued.

• Participative evaluation

DISCUSSION

In this case, think-out-loud and heuristic evaluation has been the backbone of testing the design of SJournal. Participative evaluation has been used in the process of eliminating less favorable designs.

During the process of the this project we have designed, tested and implemented two vertical prototypes - one for gathering the data and one for presenting it.

• Recording observations • Collecting usage data

We started with a simple brainstorming and based on our ideas we came up with different mockups. We went through several iterations and for each iteration we had feedback from the other course participants. Based on the feedback from both the course participants and our core users found from the user stories, we combined their needs and requirements with the summary from our market analysis of existing applications. With this in mind we were able to finalize our interface for our prototype while following the user-centered design principles.

One of the test participants for the thinking out loud evaluation, Marianne, 53, is used to computers and tablets, so she has a healthy attitude towards using applications and design needs. A small focus group of MCI-patients[7] at Glostrup hospital has also viewed and commented on SJournal.

Marianne, was asked to open up the application and then asked to give her first opinion about what the different elements in the interface was. Immediately she spotted the timeline and told us that this was the path throughout the day. As we nodded she kept indicating the different elements. Her next assignment was to find out how many texts and calls she received at a specific place. A little thinking out loud later she found the specific place and clicked on it. Hereafter she gave us the result for the question. The test with Marianne was a success and proved to us that the interface was designed in a user-friendly way.

The final prototype was tested and shown to a wide variety of users with different backgrounds. We based our user testing entirely on a think-out-loud-interview and heuristic evaluation. From the interviews we received great feedback and the think-out-loud interview really showed that our interface is very user-friendly and intuitive, as Marianne did not have any real problems solving the tasks given. The heuristic evaluation from the MCI-patients showed that even elderly people could easily use the application, and even mentioned that there was a need for these kinds of applications for them.

Even though the elderly is part of our core users, SJournal is not intended for their purpose in this project. On the other hand we will not entirely decline the opportunity to develop a similar application, which would fit their needs even more, as we can see a great market for these kinds of apps.

The MCI patients were introduced to SJournal as a side project. Since they were also asking for a simple way to see where they had been and who they had interacted with, introducing them to SJournal made sense. They particularly liked the simplicity, since smartphones normally confused them with all the different things happening everywhere. Something ”simple and not too flashy” was their comment, and they liked the connection between places and interactions.

This also helped with understanding real world applications for life logging.

During the process we stopped working on the json-parser, since we had proved that the data could be collected, filtered and displayed. However, the process of downloading the data every time the application was run took a long time, since a lot of data points had to be collected. The data could have been stored locally in files or on the application via a database.

Since this was not the main focus of the prototype, we relied on static data from our journals to illustrate the look and feel of real world data instead.

Throughout this semester other students following the course, has given feedback in the form of peer reviews. This has sometimes helped steer away from bad design ideas and in some cases help guide in the right way.

Controlling the data

Throughout the project a task to write a diary on a daily basis was given. The purpose of the diary served as a control-point when handling the gathered data from the SensibleDTU API. One of the control diaries is included in GoogleDocs[12] and a snippet of it is illustrated in Figure 16. Besides using the manually diary, a control has also kept with Moves[1]. This has given a great view of how the data looks on a timeline, and this may have had an impact on how the data should be presented in SJournal.

The time barrier stopped us from combining the two developed apps. It would have been a great feature to be able to show real world data in the application, so any student enrolled in the SensibleDTU could have viewed their own collected data. For translating the places (latitude/longitude) from the SensibleDTU API we could have made use of the public search API from foursquare[14], so real and userfriendly place-names were shown instead.

7

and tedious for them to remember. This, combined with foursquares location names, will help the user feel a closer connection and easily remember previous interactions at locations.

When creating the user prototype, we have considered the five stages of personal information collection [13].

The preparation stage is a great part of the SensibleDTUproject as the great motivation behind this was to gain insight to how social networking look in real-time.

A dropdown button on the top to change between the various days should also be present so the user can look further back in time to remember past activities. An overview of how long the user has spent at home, work or DTU could also give a nice look back at the last week, and perhaps motivate them to prioritize their time differently.

The collection stage was done by the DataCollector provided by the SensibleDTU, which collected all the relevant data from the sensors in the smartphone and then storing it in the databases on a specific frequency.

The integration stage comprised of downloading data from the Sensible DTU server and filtering out irrelevant data points and only showing actual visited locations in a simple construction.

Individual contributions

During the process of the project we have divided the workload equally and the following is the sections that each of us took responsibility of.

The reflection stage then presents the data in a simple comprehensible way.

Ulrik

• The graphical prototype

The action stage has been overlooked since the projects focus was to present the data and not suggest any future activity based on the presented data.

• Abstract • Introduction

CONCLUSION

• Related work

During the project we have developed two vertical applications which individually have their own benefits and strengths. Combining the prototypes we would end up with a product that could lead to a final Android-application which could be released on the market after some further development.

• Results • Conclusion • Team spirit Kristian

By using methods like user-stories and use-cases we gained knowledge and a greater overview about who our core users was, what features they needed in the application and in what different types of context the applications would be used in.

• The json-parser prototype • Abstract • Methods

We made extensive use of the user-centered design principles in order to get to a final prototype, which we then could test on different test participants. Our tests showed that the hard work on going through several iterations of the design-phase paid off. The test participants liked the idea of being able to get a timeline of who, when and where they have interacted with people in their lives.

• Discussion • Conclusion • Team spirit REFERENCES

The tests also confirmed the potential of these kinds of applications being more and more interesting for the general population. Quantified self is here, and in the next couple of years the devices allowing lifelogging will be more widespread.

1. Moves, Activity Tracker Without Gadgets, http://www.moves-app.com, (02 Dec 2013 at 12:54)

Our vision for this project was to design, test and develop an application that could improve the interface of the current SensibleJournal-application.

3. Trails, Trails mark your life experiences into a map, http://trails.fridayed.com, (02 Dec 2013 at 13:26)

2. Friday, A storyline of your life, http://www.fridayed.com, (02 Dec 2013 at 13:14)

4. SensibleJournal, TThe look of a social network in real time, https://www.sensible.dtu.dk/?lang=en, (02 Dec 2013 at 15:46)

Future Work

By combining the two prototypes users will be able to actually see where they have been. By implementing a database it would be easy to save, structure and maintain data points throughout the day.

5. Quantified Self , A collaboration of users and tool makers who share an interest in self knowledge through self-tracking, http://bit.ly/1bIgu7X, (04 Dec 2013 at 15:36)

The data should be crowd sourced by letting the users edit names of places they have visited. By calling their home for ”home” and DTU library for what it is, instead of addresses on a road, which could be rather impersonal for users

6. Google CardUI, Google Card UI (extracted from Google Now, https://github.com/afollestad/Cards-UI, (04 Dec 2013 at 17:36) 8

12. Sensible Diary, Day to day diary of our whereabouts, http://bit.ly/IR89mC, (05 Dec 2013 at 14:57)

7. MCI, Mild cognitive impairement, http://www.alz.org/dementia/mild-cognitiveimpairment-mci.asp, (05 Dec 2013 at 12:36)

13. Prototype Slides p. 20, Slides from 02827 Mobile Application Prototyping, http://bit.ly/1cnW3eW, (05 Dec 2013 at 16:52)

8. Lifelogging - whats next , Capturing and sharing all daily minutiae will soon get automated. Very soon., http://bit.ly/1eYGQ4j, (05 Dec 2013 at 12:36)

14. foursquare API, The foursquare API gives you access to all of the data used by the foursquare, https://developer.foursquare.com, (10 Dec 2013 at 10:52)

9. The Lifelogging Trend, What Can We Expect From It?, http://doktorspinn.com/2012/08/18/the-lifeloggingtrend-what-can-we-expect/, (05 Dec 2013 at 12:58)

15. Jawbone Up, UP is a wristband and app that tracks how you sleep, move and eatthen helps you use that information to feel your best., https://jawbone.com/up/international, (10 Dec 2013 at 10:13)

10. SensibleDTU, a large interdisciplinary research project, involving researchers from several departments at DTU, http://bit.ly/sensibledtu, (05 Dec 2013 at 13:15) 11. Prototype Slides p. 20, Slides from 02827 Mobile Application Prototyping, http://bit.ly/1g9wfGP, (05 Dec 2013 at 13:51)

16. Fitbit, Track your steps, distance, and calories with Fitbit, http://www.fitbit.com/uk, (10 Dec 2013 at 10:13)

9