POLAR CIRCULAR Polar RCX3 redesign
APPENDIX
Faculty of Industrial Design Engineering, TU Delft ID4256 - project UXAD (spring 2016) Usability testing and User eXperience Assessment in Design June 21st, 2016
Tibbe Titulaer (4232100) Rick Bakker (4139267) Mengqi Yuan (4514033) Sylvia Machgeels (4205421) Coach: Arnold Vermeeren & David Keyson
CONTENT 01 1st Impression
12 User Inspection
02 Product Build-up
13 User Test 2.0 Set-up
03 Polar RCX3 UI Framework
14 User Test 2.0 Process
04 Similar User Interfaces
15 Testable Target
05 1st User Test Set-up
16 User Test 2.0 Results
06 1st User Test Results
17 New Watch UI Framework
07 Idea Generation
18 New Watch Icon
08 Idea Clustering
19 Application UI Framework
09 Concept Evaluation
20 Next Page Indication
10 Redesign Concept 11 Prototyping
Appendix 1
1st Impression A first exploration of the product, by means of own experience and user reviews, gives an impression about the interaction and experience. Furthermore, the build-up and functions are described in respect to the intended operation. We took three round tests, each round with one recorder and one observer. Participants were asked to think aloud and express their feelings. We recorded the whole process using both film and visualized notes.
1
1
Appendix 1
Reviews from other users To see other user’s impression and opinions about the product, we searched some reviews by customers (Amazon, 2014) (see original references on the right). Polar RCX3 RUN’s score from customers is currently 4.1 out of 5 stars. It shows that 64% of customers perfectly satisfied with the product, at the same time, 14% of customers not satisfied at all with the product. To figure out what’s the reasons of satisfaction and dissatisfaction of the product, we read several top reviews which were evaluated as the most helpful from other customers. One of the top reviews among positive ones who gave 5 stars to the watch mentioned that the watch is easy to use. Additionally, it is very comfortable and light. Also, he satisfied with a huge screen and the mode that users can pick from running, cycling and others. He is a person who enjoyed an intensive cycling and weight lifting and it is just wonderful for him to use. One of the top helpful reviews among positive ones was written by Mariano Torres on March 13, 2013. He gave 4 stars to the watch. He claimed that a strap of the watch did not look that durable. On the other hands, he said that it is a very good watch for specific functions like tracking user’s running and measuring the heart rate, the speed and the distance. However, he did not realize that it lacks timed intervals for circuit training. As a result, he decided to break the deal. Also, he thought that it is difficult to view during the running since the watch lacks backlight. There seemed some misunderstanding about the function of
2
the watch, for example, he thought that there is no alarm function. Overall, he thought that the watch would receive 5 stars if it included timed inter val function. In the point of interaction, he suggested that it could provide some features to helpusers keep track of what they do. On the other hand, another top review among critical ones who gave 2 stars claimed that the strap felt too thin and the monitor poked into the wrist after using for 10 minutes. Also he had difficulty to find the way to see cal, time and the heart rate in the screen. Additionally, the watch did not provide various options that users can customize for any activity. It seemed that this product was not expected one for him. Another top review was written by B.Allen on August 2,2012 with 3 stars rating. It is mentioned that the heart rate monitor and the stride sensor make the watch does a great job of gathering data and help people to train more precisely with fun. However, he claimed that the cost – he spend $300 for itis prohibitive compared with other models which have similar features. Some training programs which could be downloaded from the company’s website were not working for this model but only available for certain models. It seemed that this product was not expected one for him.
Appendix 1
3
Appendix 1
Reviews from other users
4
Appendix 1
5
Appendix 2
Product Build-up The figure shows the build-up of, as well the physical components as the system elements. On the right the elements are explained.
6
Appendix 2
Product Build-up 1. The Polar RCX3 training computer receives the data from the sensors and saves it. It is a system that enables the user to train, analyse, plan and share training data. 1.1. The main button is used to choose a program (running, cycling or other, to choose an option from a menu or to save a set data (lap). 1.2. The light button lights up the screen when pressed short or when pressed long it enables the user to lock buttons, set an alarm or change time. The light is only visible in the dark. 1.3. The back button is used to navigate backwards through menus or to pause/end a program. 1.4. The up button is used to navigate upwards through menus when pressed short or when pressed long it leads to a menu that contains data, settings, fitness test and connect. 1.5. The down button is used to navigate downwards through menus when pressed short or when pressed long it leads to a menu that contains data, settings, fitness test and connect. 1.6. The battery case can be closed or opened by turning it with the opening cap (4). The silicone ring makes sure the battery and other electronics are protected from water. The same battery case and battery can be found in the heart rate sensor connector 1.6.1 The battery life is 800 hours average but depends on the intensity of use.
automatically sync training data. 3. The Heart rate sensor strap has an electrode area which has to be moisturized with water. It goes around the user’s chest just below the muscles. The Heart rate sensor connector is connected to the strap. 4. The Heart rate sensor connector sends real time data to the training computer. Through electrical signals it measures the user’s heartbeat. It connects to the sensor strap and is place in the middle of the chest just below the chest muscles. 5. The S3 stride sensor is capable of measuring pace and distance with an 3% accuracy after calibration. The stride sensor does not make use of GPS 5.1. The battery case can be closed or opened by turning it with the opening cap (4). The silicone ring makes sure the battery and other electronics are protected from water. 5.1.1 The battery life is 200 hours average 6. The S3 stride sensor case 7. Polarpersonaltrainer.com is an online interactive training community and a personal training diary. A user creates an personal profile.
2. The DataLink data transfer unit enables the user to transfer all training data to a web service called polarpersonaltrainer.com. WebSync software has to be installed, it is possible to
7
Appendix 2
User Scenario On the following pages a user scenario of the intended operation (or desired operation) is shown. These are based on the Polar RCX3 user manual (Polar, 2015).
8
Appendix 2
9
Appendix 3
RCX3 UI Framework The UI (usability interface) framework explains the entire screen interface of the Polar watch and how it is connected to the physical interface (i.e. buttons). See figure on the right.
10
Appendix 3
11
Appendix 4
Similar User Interface On the next pages an analysis is shown of similar user interfaces. TomTom Spark, Apple watch, Nike (+ app), Mi band and Suunto are selected (Live Science, 2015; Wearable, 2016). A comparison is made on appearance & interaction, connection and the physical user interface of the product.
12
Appendix 4
13
Appendix 4
14
Appendix 4
15
Appendix 4
16
Appendix 4
17
18
Appendix 5
1st User Test- Set up The main focus of the user testing was to find the usability and experience issues in the Polar RCX3 as well as the qualities. Secondary goals were to find potential aesthetic improvements, instructional solutions, and product-service optimizations.
set up The user testing was conducted in studio comfort at the IO faculty of the technical university in Delft, which facilitates quietness and accessibility. Also, the participants were taken outside to take a 1 kilometre run. This was done behind the faculty, particularly close to studio comfort.
process To be able to gain maximal insights, the level of instructions was kept to a minimum. Instead, the users were to think out loud as they were discovering the product themselves. The box with the watch was given to the users, and they were asked to unbox it as if they had just ordered it online. In here, the participant’s first impressions of the watch was very important. Hereafter, the participants were asked to set-up the watch for a first run. When the participants considered themselves and the watch ready for the run, they headed outside. After running for a bit , the process was evaluated inside. This included to review the data of the running performance on the watch as well as on the computer. Before and after the procedure, there was a short interview and a closing survey. All results were well documented by two observers and two camera’s. Afterwards, the data was analysed immediately. This made sure no insights were lost.
goals > Identify weaknesses & strengths of the interaction and experience for first time & longterm users of the Polar RCX3 training computer > Learn about patterns of use to find out what is
19
PARTICIPANTS This figure shows the two groups, novice and expert users, which are divided into four types of users. As mentioned before the user test will target first time Polar watch users.
Actual first time users should have a representative first user experience, since this will set the imago of Polar (what do I think of such a product?).
20
Novice/experience first time users might very well become future long-term users as they already show interest in (some of) the functions (is this a product for me?).
The comparison between the users will give interesting insights to take into account for the redesign, since every type of user will experience different qualities and issues.
The experienced first time users will be an important focus, since they will use their first impressions as a comparison to their current or old sports watch (is this a better product for me?).
Long-term users will evaluate their satisfaction about the product and thus their opinion (do I still like this product?). These users are not included in the user test (p.)
Appendix 5
MEASUREMENT METHODS > patterns/order of actions > video recording (camera), observe > amount of actions > video recording (camera), observe > reasons for actions > think out-loud, observe > time to complete a task > video recording (camera), time recording (stop watch) > use of the controls/buttons > video of the product, observe > task completion > video of product & participant, observe > errors/problems > video of product & participant, observe > attitude towards product > interview/ questionnaire, facial expression > mental effort (what was hard?) > facial expression, questionnaire/interview > opinion on the product (qualities & issues) > questionnaire (rate experience and interaction on 1-5 scale for several aspects) > priority of problems & strengths > questionnaire (rate on 1-5 scale for problems/strengths) > influence of contextual factors > video, observe, questionnaire?
Instant data analysis
Notes taken by the data logger during the thinkaloud sessions( The test-subjects were asked to think-aloud during the evaluation sessions, explaining their interaction with the system and articulating their comprehension of the design.) writing down as much as insights possible during the tests. In this way, we have our first conclusion right after the user test.
Go through all transcript
In order to make sure we will not miss each problems, we gave each participants transcript different colour, and cut into pieces based on tasks. We went through all transcipt and clustered the problems based on design qualities, and we used it as the foundation of the design goal.
> difference static vs. dynamic > video, observe, interview (compare within participants) > difference right- vs. left handed > video, observe, interview (compare between participants)
21
Figure 8 data analysis
22
Appendix 6
1ST USER TEST RESULTS Task & Completion The participants were given four main tasks and recorded on whether it was completed or not, in order to see when the problems occur and how many experienced these problems. Furthermore, the difference in task completion between the types of participants is analyzed
The tasks follow the order of usability cycle from unboxing to data reviewing. In this way the main usability problems and experiences of the product can be found in relation to the specific part in the usability cycle.
Complete the task smoothly Struggling, need some hints Need all hints/ missing Participants N = Novice user A = App user H = HRM user W= Watch user Number = Age M/F = Gender
1 HM25 2 NM23 3 AM24 4 NF21
5 NM22 6 NF22 7 AM23 8 NM26
9 WM47 10 WF44 11 WF57
Task 1 Unboxing
The participants were asked to unpack the box, express their first impression about the product and figure out different components. purpose
To indicate their first opinion on the product and if they can recognize the function of each components. A nice first experience will be beneficial for further use of the product.
task completion 82% of the particpants wasn’t able to figure out what the different components were.
11 9
1 3 4 5 6 7 8 10 2
23
Appendix 6
Task 2 set up The participants were asked to set-up the watch for a run without extra hints (fig.6). The two original manuals were provided.
sensor > Check whether the heart rate sensor connected > Fill in personal information > Calibrate
purpose
task completion:
64% of participants wasn’t able to put on stride sensor and heart beat sensor in the correct way. Furthermore, 55% of participants did not check whether or not the stride sensor was connected or they didn’t know how to connect sensor with watch. 91% of the participants didn’t fill in their personal information. And 73% of participant didn’t calibrate the watch before the run.
To find intuitive operations about the navigation and physical use. Is it even possible to figure out which actions are possible or necessary and where and how to perform them?
required steps:
Put on watch > Put on stride sensor > Put on heartbeat
Complete the task smoothly Struggling, need some hints Need all hints/ missing 11 10
11
9
10
8
9
7
4
6
8
5
7
11 10
4
11
11
6
3
10
10
5
9
2
9
9
3
8
1
4
5
1
7
11
check whether heart rate sensor connected
check whether stride sensor connected
fill in personal information
2
take on watch
take on stride sensor
take on heart beat sensor
11 8
calibrate
1
1
1
1
2
3
4
2
2
3
5
6
3
3
4
6
7
4
4
5
7
8
5
5
6
8
3
6
6
7
2
2
7
9
8
10
9 10
24
1
Appendix 6
Task 3 Run Once the participants considered the watch to be fully set up, they were asked to go for a short run of one kilometer outside.
purpose
Learn about the experiences on the training computer when running outside and which problems occur according to which influential context factors
task completion Starting a lap and pausing a lap were considered to be the most basic operations. However, there was still 18% of the participants who had problems to start or pause a lap. Besides this, the data on the screen was perceived as too small by 72% of the participants
required steps:
Start a lap > Check data during run> Pause lap
10
11
9
10
8
9
7
8
5
7
4
6
3
5
2
4
1
1
start a lap
pauze a lap
6
2
11
3
25
Appendix 6
Task 4 Data Review After the run, the participants were asked to use the watch and the computer to review their running performance (fig. 7).
purpose
Found out if the provided data meet user expectation and can be found easily. And how those data being used.
task completion According to the task completion results, only 9% of the participants was not able to find the screen with the most recent data in the watch. However, 36% of participants ran into some problems when they tried to upload data to computer.
required steps:
Check Results in Watch > Synchronize through Websinc > Go to account on website> Find your run> Find the graphs with data> Compare data
11 10 9 8
11
7
10
6
9
5
5
4
4
2
2
1
1
finds screen with the most recent data 3
upload data to computer
3 6 7 8
26
Appendix 6
Overall Error Rate:
Even though all participants are using the Polar RCX3 for the first time, comparing the two groups shows an interesting difference. The task completion error rate with the novice, novice expert, and expert shows that, expert users have lower error rate than both novice and novice expert users.
Unboxing
Prepare
Run
Data review
Novice
100%
73.8%
21.5%
35.5%
50%
37.5%
12.5%
0%
Expert
27
Appendix 6
1ST USER TEST RESULTS Usability Problems From the user test a list of usability problems was created. Some of these were expected, e.g. not calibrating while other problems, e.g. data visibility came up during the testing. The problems are rated on severity and error rate and categorized in design elements (table The problem overview shows that some design elements are causing more problems than others. The Design elements of the product that cause the most usability problems among participants are: Transparency, logic and feedback. In the next paragraphs these problems and possible explanations will be elaborated on).
28
Appendix 6
29
Appendix 6
1st USER TEST- RESULTS Emotion Flow During the user testing, we observed that the emotions of the participants changed over time. The emotion flow is used to show the tendency of changing emotion and problems occurring during each phases. Problems are stated with red dots and qualities with green dots. The height of the problems dots has no relation to severity, but the number of dots shows the density of problems and qualities.
It is nice that the band can be adjusted and fit my wrist well.
According to the observation and quotes, we found that when participants were asked to prepare for a run, some of them were confused, frustrated and two participants said to feel stupid. However, during the run and after the run they were way more excited. As the emotion flow shows, the amount of problems has a direct relation to user emotion.
It is good to get some feedbacks when running
Looks professional
Looks cool It is light weight!
PREPARE FOR A RUN
RUNNING
EVALUATION
I have no idea what this component is..
How to attach the stride sensor on my shoes?
Why there are two menus?! What is sport zones? I quit..
Have to press “back” lots of times, I need a “home” button The watch looks outdate.. The heart rate strap makes me feel uncomfortable
How to start a lap? I think it is a LED..
Now I have to spend long time to read all this instructions, why this watch didn’t guide me!
30
The data on the screen are too small
I press the red button to start a lap, I think press again should be pause the lap, but it isn’t.
I am afraid the strap would slip down when running.
Some datas are too professional for me
Only tell me what I have done, but not how to improve my performance.
The synchonizing way is complicated and slow
I think the data would transfer to laptop automatically.
Appendix 6
Role of the Polar RCX3 Apart from the feelings (emotion flow) and overall opinion (questionnaire), we want to find out about the motivation for buying a Polar RCX3. This is addressed as the role or the relationship with the watch. This is visualized in figure 9 (also see definition for the axis in the figure). Six roles were defined, which differ on a scale from personal to practical and open (or free) to strict: trainer: handles with discipline, while training you to improve your performances every time coach: motivates by helping to achieve your goals (regarding performances) buddy: acts like a friend, running along to provide fun and involvement logger: provided you with a summary of the data you’ve generated, without any conclusions indicator: next to showing data, it gives feedback when needed (e.g. when running too hard) addition to gear: serves as another gadget to support you while running
From the matrix it can be derived that novice users have a higher need for a personal & open role compared to expert users. The desired role for novice users lies between buddy and coach (range between more or less open; and for some urging slightly to more practical) The expert users desire a role between indicator and trainer(where the range is between more or less practical). Only a few of the participants recognized their desired goal in the watch.. For most of the novice users the actual role was experienced as ‘logger’ while they’d desire a ‘buddy’ or coach’. The expectations for the expert users were generally met more often. When comparing some products with similar functions (for example TomTom, Nike + and iwatch), those similar products act more as a buddy with entertainment functions during running and rewards once achieved a goal (as a way to motivate people). This could explain why the expectations of the novice users often were not met. (see appendix p. for similar interfaces)
31
Appendix 6
Survey At the end of the user testing, a survey was conducted. This helped to get more tangible feedback on the perception of the Polar RCX3. Word
pairs, which are opposing one another, were used to help participants assess these (first) experiences.
Figure 10 shows the ratings that were given by the participants regarding the word pairs. The coloured horizontal bars show the range in which the participants have rated (lowest and highest rate visualized with a coloured circle). The vertical moving lines show the average rating per user type. The overlap in rating range is shown in a mixed colour.
32
Appendix 6
> novice & novice/experts in this case the two types in the novice group are compared: novice and novice/expert users. In general the watch is rated high in it’s professionality (for both novice and novice/experts). Differences in this rating are caused by the expectations these users had in advance. For the novice/experts it was as expected, while for the novice users the professionality was higher than expected (more difficult to set-up; more data & results afterwards (positively surprised)). This could be an explanation for the positive terms of the experience (involved,
encouraging, connected, convenient) being more agreed upon for the novice/expert users compared to the novice users. However, novice users have a higher rating in multi-sensorial and trustworthy. This could be because they don’t have any knowledge or experience on similar functions and products and are therefore overwhelmed with the amount of options ( multi-sensorial). Since they assume this product is meant for professionals (“this is really for professionals”), it makes sense to be trustworthy.
Figure 10 novice vs. novice/expert ratings
33
Appendix 6
> novice vs. expert In this case the two groups, novice and expert, are compared (fig. 11). The isolated dots in the figure are the outliers of the rating (i.e. one person rated very different). The overview shows that more negative experiences arise for the novice users (more red dots). Both novice and expert users rated the product to be more relatively vague instead of clear. These results are also seen in the observations
34
of the task completion (emotional flow, fig.8). Combining all these conclusions, it can be stated that first time users experience a lack of confidence in using the Polar RCX3. A first impression is really important to create affection with a product (Mcmillion, 2015; Yong Gu Ji & Sooshin Choi, 2014). As the results show the product is also rated to be demotivating which can be the result of feeling insecure about their actions. This aspect needs to be countered in a redesign.
Figure 10 novice vs. novice/expert ratings
Appendix 6
male & female On the right the results of the rating for the word pairs can be found. It can be seen that: > the data for the two female (both novice) is really divers (two totally different girls) and thus hard to conclude on this data > average data from the male participants is in between averages of novice and novice/experts (3 male novice/experts; 3 male novice) → not adding extra value in conclusions
of the watch. Things they said: > “it looks cheap” (due to the plastic material and thicknesses, due to package) > “it looks outdated” (due to pixel & black&white screen, due to website) > “it’s not nice compared to the apple watch” (due to design ‘competitor’ products) > “I don’t like the design” (due to material combination, [unknown factors])
from the interview and observation forms: It was noted by several participants (1/2 female; 2/6 male) that they didn’t like the design or appearance
It is important to note that the Polar RCX3 has a different version for female users (the same design, but with the colours white and pink instead of black and red).
35
Appendix 7
Idea Generation
36
Appendix 8
Idea Clustering
37
Appendix 9
Concept 1
38
Appendix 9
Concept 2
The watch is entirely operable with the thumb. Only the button for the ligt requires the index finger.
Uxad Group A2
39
Appendix 9
Concept 3
40
Appendix 10
Concept Evaluation
41
Appendix 11
42
Appendix 12
Usage Inspection Cognitive walkthrough • Go through a task step-by-step • With each step: try to think like a user
PROCEDURE Persona “typical” user of the product Age: 39 Occupation: Engineer John is a serious runner, he usually runs outside more than three times a week. Running for him is a good way to keep fit and also his interest. Last week, John bought his first sport watch in order to improve his performance.
Tasks Task 1 Find main menu and select “setting” steps: Turn the ring forward > turn the ring continuously to “settings” > press the ring to select Task 2 quick start a run steps: Turn the ring forward to the main menu > turn the ring continuously to “start” > press the ring to select “start” > get an alert (press button on the sensors) > press button on the sensors
> get sound feedback on the watch (show pairing process on the watch) > get feedback “sensors connected” > press the ring to start Task 3 Set a goal and start a run steps: Turn the ring forward > turn the ring continuously to “goal” > push the ring to select “goal” > push the ring to select “distance” > turn the ring to adjust distance (number goes up/down when turn left/right) > push the ring to set distance > watch will show two options: “other?” or “start”> turn the ring and select “start”
43
Appendix 12
Task 4 Running outside (change view while running, pause the run, stop the run) Check data on the screen > turn the ring to change view > press the “back” button to pause the run > push the ring to start again > double click the “back” button to stop the run Task 5 Review data on the watch and application double click the “back” button to stop a run > watch will show the results of the first type of data > turn the ring to go to the next screen (next type of data) > mobile phone receives a notification: “check more detailed result on the mobile phone” > swipe the notification to go to the application > scroll up and down to see the result
CONCLUSIONS FROM THE USER INSPECTION Problem 1 The relation of setting a “goal” and starting a “run” (where run would be ‘quick run’) on the main menu was confusing. The option “setting” in the main menu was vague as it could be confused with the options in start or “goal”. Potential Solutions: Replace the option “goal” into the menu “start”.
44
Conduct the actual walkthrough Walk through the action sequences for each task from the perspective of the “typical” users of the product. For each step in the sequence, imagine how the user would behave/experience the interaction regarding the following questions: Will the user try to achieve the right effect? Will the user notice that the correct action is available? Will the user associate the correct action with the effect that the user is trying to achieve? If the correct action is performed, will the user see that progress is being made toward the solution of the task?
Split the option “goal” into “my plan” (composed training plan) and “quick goal” (focus on specific data). Move the “plan setting” (making a plan) into the application to better facilitate the amount of options and information. Make the option “setting” more specific by making a division into settings about the user (“profile”) and settings about the watch (“watch”).
Appendix 12
Problem 2 The information visibility was low (overload of info) when there were more than four selections on one screen. Potential Solutions: Keep the amount of elements/functions on one screen to a maximum of four.
Problem 3 The interface didn’t feel like it can be extended when there are more pages (i.e. more than four elements/functions).
1
Potential Solutions: Using an arrow to indicate that there are more selections behind Using pages to indicates the number of pages (screens) and the current page.
A
A DISTANCE
1
TIME
HEART RATE
2
INTERVAL
4
2
CALORIES
number with pages
A
A DISTANCE
HEART RATE
TIME
CALORIES
INTERVAL
5
DISTANCE
TIME
pages
HEART GOALRATE
INTERVAL GOAL
CALORIES
progress bar
A
3
INTERVAL
HEART RATE 1
TIME
CALORIES
arrow with pages
2
DISTANCE
A
DISTANCE
HEART RATE
TIME
INTERVAL
CALORIES NEXT
text alert
6
DISTANCE
TIME
HEART RATE
INTERVAL
CALORIES
PREVIOUS
pages indicate direction
45
Appendix 12
46
Problem 4 The ring was sometimes miss turned (e.g. while running accidentally touching the ring) or buttons miss pressed, as the buttons were fairly close to the ‘grabbing surface’ of the ring.
Potential Solutions: Long press the ring to lock the watch, long press again to unlock the watch
Problem 5 The light button was too close to the back button, which could cause miss press.
Potential Solutions: Replace the light button to the top (easily pushed with index finger without having to change gesture); keep back button at original
Appendix 13
47
Appendix 13
Prototying
48
Appendix 14
USER TEST 2.0 - SET UP The first redesign concept and ideas are tested in the second user test to verify the results so far. The setup of this second user test is kept as close as possible to the first user test (same tasks, methods and location to provide an
Goal Identify weaknesses & strengths of the interaction and experience for first time users of the redesign of the Polar RCX3 training computer. As overall goal: improving the redesign according to the design goal Focus The second user test focusses on the interaction qualities that were described in the design brief: “We want people to feel confident while using the Polar RCX3 training computer, by providing a guiding, intuitive and trustworthy interface.� Qualities (User Interaction)
Effect (Interaction)
Trustworthy Intuitive w Guiding
Confident
optimal environment for evaluating the data. Some minor adjustments were made to create a better focus. The biggest change was the fact that we now worked with two scenario’s and a prototype.
Therefore the user test is divided into two parts, according to a physical (appearance and physical interaction like buttons) and a virtual model (UI framework, screen-flow). These models are tested separately to avoid confusion (see set-up) and gain the best results within the given time for building a prototype (an integrated model would be very difficult). physical vs. virtual model: The physical model is used during the unboxing (first impression on appearance and physical interaction) and during the run (interaction in dynamic actions). The virtual model is used during the setup and review. The app is used during the second part of the review. See the figures on the next pages.
49
Appendix 14
Next to the two scenarios, the user test is divided into two parts, according to a physical and a virtual model. These models are tested separately to avoid confusion.
The physical model in the new designed package. The original sensors were used in this setup. This model was used during the unboxing and during the run in the user test.
50
Interaction model for the virtual prototype. A physical ring is used by the participant to turn (next to the full physical model) while a LED ring on the virtual screen indicates the turning. This is to connect the two worlds and let
users experience the actionreaction interaction properly. This ring was not seen by the participant, but purely for our convenience. The screen are shown on a tablet (fig. 5).
51
Appendix 14
Participants:
The same target group is addressed, but different participants compared to first user test: - participants in the category novice & novice/expert (never used any kind of device for running OR use(d) similar functions or products) - participants in the category expert (use(d) another kind of sports watch) Two scenarios 1. New bought in the (online) store (4 participants): When starting up for the first time, the watch starts with asking for personal information as guidance for setting up your profile.
Not taken into account:
The manual is not included, since our goal is that the watch itself would be intuĂŻtive and guiding (thus not needing a manual.
52
2. Bought secondhand on Ebay (3 participants): The watch does not ask for personal information for first time, since it has already been used. They need to reset the watch or make a new profile.
Appendix 14
Process:
The process will be recorded. And this video will only be used internally, and will not be used for any other purposes. Is this ok for you? Today day will take about 90 minutes. The main goal is to analyze this redesign of the Polar training computer. As mentioned in the consent form we send you up front, we will do this with the following setup: Starting with asking you some background questions. This helps us to get an idea of what you already know. Hereafter, we will give you the product. You have the opportunity to examine it and we will ask you to do a number of things with it. Once you get to know the watch, we will go for a run. This is the best way to experience a runner’s watch. After the run there is a review of your run, a short interview & discussion, and an online questionnaire, to help us evaluate the results. Before we start, there is one more thing that we need to do. We have made a consent form (as send to you up front). Could you read and sign this? It’s important to know you cannot do anything wrong. Everything that you do will give us insights on the product to improve. Background Questions (10 min) How often do you usually run? Can you describe the steps you usually go through when going for a run? For example, think about the last time you ran? (e.g. preparation, place, time, warming up, cooling down, stretching, looking up roads or distances, setting goals) What is your goal of running?/ Why do you run?
Why do you run with a watch? What kind of watch are you using yourself? Why did you choose for this watch? How would you describe the communication between you and the watch? [first open question, then multiple choice]: e.g. buddy, coach, trainer, logging device, indicator, addition to gear) The goal of having this watch? What would you expect of a runner’s (Polar) watch (e.g. functions, actions, use..)? If the watch would be a person/character. What would you want the communication to be like? What would you find the most appropriate description for what you would want from such a watch? (e.g. buddy, coach, trainer, logging device, indicator, addition to gear) Now we would like to continue with the actual product/redesign, combined with the tasks. Unboxing (10 min) [use the physical model for unboxing] We want you to imagine that you have purchased the Polar watch online/ secondhand via Ebay and it now arrives on your doorstep. You are now about to open it. While you are doing this we would like to ask you to think out loud so we know what is going on. What was your first impression of the different components? [going through them one by one and asking what they think they are for] What do you think of the product appearance/ design? What is your feeling when this product is delivered on your doorstep?
53
Appendix 14
Set up for a run (15 min) [use the virtual model & real components] For this part we would like you to look on this tablet to go through the screen interface. You can control it by turning and pushing the ring on this box, which is connected to the screens. Normally you would of course have the screen on the watch itself, but for this test we separated the physical and virtual parts. Potential hints (when struggling for 1 minute (starts to be uncomfortable); start on top) Since you got it secondhand, the information of the old user is still there. Can you reset the watch? Do you have a goal for this run? > You can set your goal in the watch You chose ‘quick run’. Did you see the other options? > Can you also set a goal? Please connect the devices [write down which hints are used]
54
you imagine this to be a real watch, etc.? Please share your experiences with us. [after 2 laps] (How did it go? You feel well? Are you tired?) Please take a short rest… Any first impressions about the watch that you’d like to share with us? Let’s go back inside and have something to drink.. Review (10 min) [use the virtual model] When you’ve caught your breath again, we can review your results. We want you to do this in two ways. Let’s start with the watch itself. You can use the tablet again to see the screens.
Run (20 min) [use the physical model & real components] When the product is fully setup, I would like to take you outside for a run behind the IO faculty.
What kind of data do you expect from the watch? What kind of data can you actually derive from the training computer? Is the data you are getting enough? (What do you miss?) What can you do with this data? What do you think of how the data is shown/ visualized?
Please make yourself ready You can use the physical model to wear. You will not be able navigate in the screen. Therefore we would like you to focus at the ring and the buttons, the size of the screen, etc. 2 laps I would like you to do 2 laps. Please try to run like you usually do. When running, please wear this GoPro around your head and think out loud again. Does it feel convenient? Could
Potential hints How many calories have been used? What is the average speed and time spent for this lap? What other information would you like to know? Can you find it? What does it tell you?/ What can you do with this information? Was it how you expected it? How was it different? [write down which hints are used]
Appendix 14
Now we want you to upload the data to the online platform. We have pre-installed the software on this phone. You can use my account. [use the app] What can you do with this data? What would you want to do with the data? What data would you need/want? What do you think of how the data is shown/ visualized? Evaluation interview (5 min) How did the exercises go? Explain what went well/went wrong. Did the watch meet your expectations (before, while, after running)? Did the provided data meet your expectations? Explain. What do you like about the product? Explain What did you not like about the product? Explain If the watch would be a person/character. What would you want the communication to be like? What would you find the most appropriate description for what you would want from such a watch? (e.g. buddy, coach, trainer, logging device, indicator, addition to gear)
55
Appendix 15
TESTABLE TARGETS To conclude whether the design goal was met, criteria were formulated to serve as testable targets during the second user test (Table X).
These testable targets are tested with a questionnaire, interview questions (appendix Bx, p.x) and an observation form (appendix Bx, p.x).
Trustworthy Currently, the trustworthiness of Polar RCX3 is rated 5. After the second user test it should be rated with an average of at least a 6 (in the closing survey). Currently, the level of clarity of Polar RCX3 is rate 3. After the second user test it should be rated with an average of at least 5. The desired data can be read from the screen within 2 seconds.
It is desired for the users to understand and interpret the data as intended. Therefore they need to actively respond to messages (e.g. “please, connect the sensors�) provided by the training computer. The error rate in preparing for a run has to be reduced by 20% for expert and below 50% for novice participants (Currently the expert error rate is 37.5%; the novice error rate is 73.8%)
Users can derive and read the desired data while running within 5 seconds Intuitive Participants are able to figure out what the functions of each component are without using the manual
Guiding The user should not be directed into wrong menu’s (not make wrong menu choices) Users have the possibility to ignore the initiative of the watch/ messages from the watch should not irritate users.
56
Participants are able to figure out how to use each component without using the manual Participants complete all necessary set-up steps without hints or manual.
Appendix 16
USER TEST 2.0 RESULTS Questionnaire - Word Comparison > novice vs. experts Compared to the first user test, one word pair (one-sided vs. multi-sensorial) was taken out, since this was sometimes seen as confusing for participants and doesn’t add significant value for the second user test.
57
Appendix 16
> novice & novice/experts The difference in rating for the redesign between novice and novice/expert is smaller compared to the rating for the original Polar RCX3. This implies that the gap in usability/ experience is more on one level. The most remarkable differences are the rating for involved,, encouraging, professional and personal. Novice users rate higher for involved and encouraging, which fits their desired role of the watch (see fig.3 from the first user test).
58
Novice/expert users might be more used to the tone of these kind of sports products. It could be therefore that the professionality was rated higher by novice users compared to novice/experts. When we look at personal, the novice participant rate a lot lower than the novice/expert. This could have strong relations with their view of professionality.
Appendix 16
USER TEST 2.0 RESULTS Questionnaire - System Usability Scale (SUS)
The System Usability Scale (SUS by John Brooke, 1986) is a reliable tool to measure the usability of a product in a “quick & dirty� way. This questionnaire consists of 10 statements
that can be rated on a 5 point scale (from strongly agree to strongly disagree).
I think that I would like to use this product frequently Novice avg rate 4,8: all strongly agree Expert avg rate 4,0: more diverse rating Most participants agree (one expert rated neutral/3)
I thought there was too much inconsistency in the product Novice avg rate 1,8: most rated 2 Expert avg rate 2,0: all rated 2 All participants disagree
I found the product unnecessarily complex Novice avg rate 1,5: all strongly disagree Expert avg rate 2,0: more diverse rating All participants disagree (one expert rated neutral/3) I thought the product was easy to use Novice avg rate 4,3 Expert avg rate 4,3 All participants agree (most rated 4) I think that I would need the support of a technical person to be able to use this product Novice avg rate 1,0: all rated 1 Expert avg rate 1,0: all rated 1 All participants strongly disagree I found the various functions in this product were well integrated Novice avg rate 4,3: most rated 4 Expert avg rate 4,7: most rated 5 All participants agree
I would imagine that most people would learn to use this product very quickly Novice avg rate 4,8: most rated 5 Expert avg rate 4,3 All participants agree I found the product very cumbersome (troublesome) to use Novice avg rate 1,8 Expert avg rate 1,7 All participants disagree I felt very confident using the product Novice avg rate 4,5 Expert avg rate 4,3 All participants agree I needed to learn a lot of things before I could get going with the product Novice avg rate 2,3: more diverse rating Expert avg rate 1,3: most rated 1 All participants disagree
59
Appendix 16
The table below shows the difference in rating between the original product and the redesign. It can be seen that the redesign was rated more positively for the experience (more statements for qualities are agreed on, while statements for problems are disagreed on). Where users had trouble with the original product (rate of 3 or higher; e.g. they found it
60
unnecessary complex, needed to learn a lot of things before they could get going, didn’t feel very confident while using, found that there was much inconsistency), this was almost not experienced with the redesign (rate of 2 or lower).
Appendix 16
USER TEST 2.0 RESULTS Problem List
61
Appendix 16
62
Appendix 16
63
Appendix 16
64
Appendix 16
65
Appendix 16
USER TEST 2.0 RESULTS Task Completion
Complete the task smoothly Struggling, need some hints Need all hints/ missing
Participants N = Novice user W= Watch user M/F = Gender
1 NM 2 WM 3 WM 4 WF
5 NF 6 NM 7 NM
7
7
6
6
5
4
4
3
3
2
2
1 take on watch
TASK
1
5
Novice
20%
20%
Expert
0%
0%
Error Rate
66
check whet rate s conne
0%
33
Appendix 16
7
7
7
7
7
7
7
6
6
6
6
6
6
6
5
5
5
5
5
5
5
4
4
4
4
4
3
3
3
3
3
3
3
2
2
2
2
2
2
2
1
1
1
1
1
1
1
check whether stride sensor connected
fill in personal information
finds screen with the most recent data
upload data
k her heart sensor ected 4
start a lap
pauze a lap
4
%
0%
0%
0%
0%
0%
0%
3.3%
33.3%
0%
0%
0%
0%
0%
67
Appendix 16
USER TEST 2.0 RESULTS Emotion Curve
It is easy to switch among different data views
The uasge is intuitive Plan can help me making progress.
The button on the sensors are intuitive
The shoe lace on the sensor is a good hint
It’s interesting to have a smile at the beginning
PREPARE FOR A RUN
It is good to review data in a big screen. The achievements is motivated
The watch is easy to use while running The main menu is easy to find
RUNNING
It’s good to have a Polar runner community
EVALUATION
I hope the data review on the watch is a loop.
Do I need to confirm again to start? I am not sure about the orientation of the watch
I want to restart, but it stop the run.
How to set my personal screen?
I don’t know what the sport zone means
Do I need to press buttons on each sensors? The “me” on the application is confusing.
Not sure if I can set more than one quick goal I don’t know if I am doing well
68
The different profiles on the application is confusing
Appendix 17
New Watch UI Framework
69
Appendix 17
70
Appendix 17
71
Appendix 17
72
Appendix 17
73
Appendix 17
74
Appendix 17
75
Appendix 17
76
Appendix 18
New Watch ICONS POLAR CIRCULAR ICON MAIN MENU
profile
watch
result
start
START
quick start
my plan
quick goal
A heart rate
speed
time
calories
distance
press sensors
PROFILE
KG Age (number)
running (runner1)
gender
gender
(female)
(male)
weight (number)
sound
time
height (number)
fitness
interval
bicycle (cyclist1)
WATCH
screens
language
light
phone
REVIEW
77
Appendix 19
Application UI Framework
78
Appendix 19
79
Appendix 20
Next page designs
80
Appendix 20
81
82