GROUP B3
Mees Daalder (4228979) Linda Smit (4212126) Emanuele Gandini (4663667) Hanna Timmerman (4605985) Nasibeh Soltani (4622464)
ANALYSIS REPORT
UXAD | ARNOLD VERMEEREN & RENE VAN EGMOND
2
EXECUTIVE SUMMARY
00
EXECUTIVE SUMMARY In the last two months the E-bike system of the Sparta F8E has been analyzed. In the first period, the analysis phase took place, in which the strengths and weaknesses of the product were examined, which led the team to understand which were the aspects to be investigated when continuing the analysis. At this point it has been decided that, after a research about E-Bike users nowadays, the target group for this project will be commuters without any previous E-bike experience. The analysis of both the general aspects and the specific features of the bike led to a series of problems, which were important to investigate during user test. Six persons from the target group participated in the user test, in which is tried to identify Usability Problems of the E-bike system. First, an introductionary interview was held after which participants were asked to cycle through the park. After, task scenarios were played out, recorded Go-Pro footage was looked back upon and an experience questionnaire was filled in. To end the test, a final interview was held. This test was meant to answer five different research questions that were developed.
The first question was “How does the participant navigate to the functionalities during the first time use?”. It was important to understand how the participants interact with the functionality. More precisely, it has been noticed that the participants were looking for buttons in certain positions, or they assumed that a specific function was automatic. This question highlights that the language, the icon and the abbreviation in this system are not clear. The second one “Which usability problems is the participant facing during the first time use?” showed that the main problems are related to turn on and off the bike, to switch on and off the light, operate the buttons and understand the meaning of “MODE”.
the use of the product as complex and cumbersome, and they didn’t experienced the interface as consistent and user-friendly. The last question “Does the E-bike fit the expectations of the participant during first time use?” investigates the expectation of the participants. Before and after the test the participants were asked what their expectations of an E-bike compared to a normal bike are. It was highlighted that the user interaction with display is not their first priority, but intuitive interface is needed to operate the support easily.
The third research question was “What are the causes of the found usability problems?”. The most important cause of the occurred problems is that there are no clear usecues on the buttons to indicate which functionalities can be accessed by certain buttons.
All the results were combined into the design brief. The problems addressed in the text above are the starting point for this document, which resulted in the design goal “Commuters should be able to use and understand the E-bike system without receiving instruction during their first time use”. In addition to that, a design vision is included, in order to explain the feelings a future redesign should communicate.
The fourth question ,“How does the participant experience the E-bike system during first time use?”, was created in order to measure the feelings and the mood of the participants during the test. The results of it described
Finally a list is created with the specifications a redesign should have. Besides, it is described how it is possible to test how a future redesign meets the specifications and the design goal.
EXECUTIVE SUMMARY
3
00
TABLE OF CONTENTS
06 08 15 04 | USER TEST RESULTS
03 | USER TEST PLAN
02 | PRODUCT ANALYSIS
01 | INTRODUCTION
4 TABLE OF CONTENTS
19
36 40 42 46
A4 | USER TEST RESULTS
A3 | USER TEST PLAN
A2 | PRODUCT ANALYSIS
06 | REFERENCE LIST
05 | DESIGN BRIEF
TABLE OF CONTENTS
52
5
6
INTRODUCTION Hanna
01
INTRODUCTION This report contains the analysis that the Dynamo Design team executed on the SPARTA F8E. This report describes the E-bike system and usability assessment for the SPARTA F8E from start to design brief. The E-bike consists of two main parts: the phyiscal bike and the E-bike system. The first chapter provides a short introduction of the product and the project, such as context of use and target group. With the second chapter the focus lies on the user test plan, including the goals, test setup, and procedure. In chapter three, the team analyzed the collected raw data and categorized the results based on the five main research questions. Furthermore, with an overview of the whole process, the team classified the problems severity based on the team’s test goal. Finally, the team came up with a design brief that formulates a design goal. This design goal will be described in the final chapter. It also contains a vision for the interaction between user and system, and ends with design specifications for a redesign of the product that will made by the team in the next phase.
INTRODUCTION Mees Emanuele Linda Nasibeh
7
8
PRODUCT ANALYSIS
02
PRODUCT ANALYSIS This chapter gives an introduction in the product: the E-bike ‘Sparta F8E’ (Figure 2.1). It gives an overview of the product, how it can be used and what the functions are. A more detailed explanation about the different components of the product can be found in appendix 2.1. Besides, the context of the E-bike will described. One of these factors is the user of the E-bike. Five different user groups are defined and a target group is chosen to test the E-bike with.
Figure 2.1: Sparta F8E
PRODUCT ANALYSIS
02
01
Inserting the battery
Switch on the E-bike
03
Check status on the display
Figure 2.2: story board
STORYBOARD The storyboard (Figure 2.2) describes regular use of the E-bike based on the personal experiences of the team. While trying the bike, the team already discovered that there is no fixed order to use the product. It depends on the user and context which tasks and in which order they are used. In appendix 2.2 & 2.3, the relation between all components and a detailed task analysis can be found.
9
10
PRODUCT ANALYSIS
05
04
Using accelerator to build up speed
Cycling with pedal support
06
Change support
PRODUCT ANALYSIS
07
Step off bike
09
06
Gather data from trip
Take out battery & store E-bike
11
12
PRODUCT ANALYSIS
DESCRIPTION OF THE COMPONENTS During the project the team focussed on the E-bike system: the parts of the E-bike that enables the user to operate and get feedback on the different functionalities of the E-bike (the buttons and display, figure 2.3). Appendix A2.3 shows a task flow of how the user can operate the buttons.
Figure 2.3: description of the components
PRODUCT ANALYSIS
13
USER GROUPS OTHER PEOPLE
SYMBIOTIC PRODUCTS
Daily commuter: a long distance commuter, who travels every day by bike to work and home.
-CLOTHES -BACKPACK-BAGS -SUNGLASSES -GLASSES/LENSES
-TRAFFIC -COMPANION -CHILDREN -FRIENDS
-CHILD SEATS -GLOVES -BASKETS
Parents with children: parents who bring their young children to school on their bike can use the E-bike to get to the school quicker and easier.
-ELDERLY -LONG-DISTANCE COMMUTER -PEOPLE WITH PHYSICAL LIMITATION -PARENT WITH CHILDREN -OTHERS
-BIKE - SYSTEM
PRODUCT-SERVICE
COMBINATION
Elderly: people with more physical limitations can go out for a longer trip than with a regular bike. It could give them a new sense of independence and freedom.
USER
SUN/RAIN/WIND/COLD/HEAT/TRAFFIC/LIGHT /DARK/BAD ROAD/FLAT/HILL/NIGHT/DAY
People with physical handicaps: younger people with physical handicaps that prevent them from riding a regular bike, can use the E-bike as a good alternative.
ENVIRONMENT Figure 2.4: human interaction system
CONTEXT OF USE Figure 2.4 visualizes the context during the use of an E-bike. For each of the components in the context, there are multiple factors that could influence the context and are important to take into account during the user test. The factors are dependent on the user.
Other: the remaining group that have no particular reason to use an E-bike. They only use the E-bike to bike easier with the support. Figure 2.5: user groups
PRODUCT ANALYSIS
TARGET GROUP Due to the big variety of user groups, the team decided to focus on one of them: commuters without any previous E-bike experience. In recent years, this group is increasing in size (Hendriksen et al, 2008). Approximately 25% of the commuters in the Netherlands are using a bike as main transport to go to work (Top10 hell, n.d.). According to a research by Wiebe Engelmoer in 2012, E-bikes have the potential to cause a growth of 4% to 9% in the total amount of commuting trips in the Netherlands. In some areas this could even be up to 20%. Also other research shows that commuting with E-bikes is becoming more and more common (van der Zee, 2016). During redesigning,the reasons why commuters choose E-bike should be taken into account. Most E-bike commuters have a fairly long daily ride (10 – 20 kilometers) and when they arrive they cannot be huffing, puffing and sweating (Fietsersbond, n.d.). Less than 10 kilometers is fine by traditional bike and more than 20 kilometers is risky with battery life. In the Netherlands, companies, universities, local and regional authorities offer their employees bonuses when they come to work on an electric bike (van der Zee, 2016). Also, some employees have limited time to exercise, using an E-bike instead of their car or public transportation it is good opportunity to stay active.
The persona in figure 2.6, provides an overview of the average person in our target group. He lives about 20km from his work, and instead of using public transport (which he is doing now), Luuk is considering getting an E-bike. He has technological knowledge, but expects products to be easy to understand and use, because he has no time to read through extensive manuals.
Luuk Meijer Nationality: Dutch Age: 31 Luuk is an active person that likes to spend his free time on outdoor activities. He got married recently and has one 5 year old daughter.
Luuk works in a big office, where appearance is very important. He uses technology often, but has no patience. He doesn’t have time to read extensive manuals to learn how his devices work. Figure 2.6: persona of the target group
$
14
Luuk lives in Delft, but works in Den Haag.
He goes to his work by public transport everyday, but he would prefer a bike instead.
USER TEST PLAN
03
USER TEST PLAN During the beginning of this project, research was done on the first impression of the E-bike system, which was combined with findings from online reviews (appendix A3.1). Based on these findings, a research goal was set up to get an overall direction of the research. It was decided that the team will focus on the first time use of the E-bike system of inexperienced users. The first time is chosen because it has a big influence on the user’s opinion of the product and on further use of the product. In order to reach this research goal, the five research questions that are set up have to be answered. The user test is structured based on these questions: it will start with an usability test where the participant is asked to make a 800m ride. After the ride, the task scenarios that have to be completed provide more specific data about the use of the main functionalities of the E-bike system. The interview and ‘System Usability Scale’ questionnaire (Affairs, 2013) will provide insights on the user’s experience the E-bike system. See appendix A3.2 for a detailed explanation of the procedure.
Figure 3.1: user test
15
16
USER TEST PLAN
RESEARCH GOAL “The goal is to gain insights into how commuters use and experience the E-bike system during the first time use to discover usability problems, its causes and if the E-bike system meets the expectations of the participant.� RESEARCH QUESTIONS
USER TEST SETUP
1. How does the participant navigate to the functionalities during the first time use? 2. Which usability problems is the participant facing during the first time use? 3. What are the causes of the found usability problems? 4. How does the participant experience the E-bike during first time use? 5. Does the E-bike fit the expectations of the participant during first time use?
During the usability test, the E-bike will be tested in a simulated environment (insitro), where two actors will play the oncoming traffic (Figure 3.3). The participant will be recorded by a GoPro during the whole test. The other procedure steps will be performed at the end of the route. Actor 1 Actor 2 Route (800 m)
PARTICIPANTS Target group User group
Start/end
Product settings
6 Commuters
1 elderly
1 other A
Figure 3.2: participants
Figure 3.3: user test setup
USER TEST PLAN
02
PROCEDURE
Usability test: introduction
Explains thinking aloud
Facilitator
Participant
Recorded by GoPro
datalogger
01
Introduction & initial questions
Figure 3.4: user test procedure
02
Usability test: first time use
17
18
USER TEST PLAN
03
05
Task scenarios
SUS questionnaire & last questions
Asking questions
SUS questionnaire
04 Tasks 1. Change support 2. Change mode settings 3. Turn on light 4. Turn off E-bike system 5. Explain E-bike system
Usability test: looking back footage
USER TEST RESULTS
04
USER TEST RESULTS This chapter discusses the results and analysis from the user test. The team starts with raw data (Appendix A4.2) and ends up with the wisdom to create a proper design brief (Figure 4.2). The first step for each question was gathering all the raw data and structuring it. The team used that data to extract the useful information from it. This information was then transformed into knowledge by drawing conclusions from it. In de design brief it was decided what knowledge would be used in the redesign (wisdom).
Figure 4.1: instant data analysis
19
20
USER TEST RESULTS
During the user test, multiple ways of collecting data were used. The team observed the participants behaviour, recorded interviews and video taped all of their interactions. Furthermore, interviews were held about the user’s expectations and the participants were asked to fill in a questionnaire about their experience with the E-bike system. All of this data was structured in the ‘participant spreadsheets’ to creat a more structured data set. The results from observing the participants and hearing quotes made by the participants will answer the first three research questions. The questionnaire will be used for answering research question four. Lastly, research question five will be answered by the expectations and experiences mentioned by the participants. A total of eight people participated in the test. Six of them are from within the target group, and two of them are from different user groups. Figure 4.3, shows an overview of the eight participants. The two out of the target group (grayed out) are left out of the results in this chapter. The results of the six participants within the target group are compared later in this chapter to the two participants who didn’t belong to that group. They are used to compare and see to what extend the results are similar between the groups. This chapter will end with a discussion on the results. Due to different reasons, the results should be handled with care. The team should be aware of the possible flaws of the results when drawing conclusions.
Figure 4.2: data analysis process
USER TEST RESULTS
Overview of participants P1
Age: 61 Travel to work: Bike
P2
Age: 55 Travel to work: Bike
P3
Age: 55 Travel to work: Bike
P4
Age: 34 Travel to work: Train, walking
P5
Age: 24 Travel to work: Bike
P6
Age: 73 Travel to work: E-bike
P7
Age: 22 Travel to work: Bike
P8
Age: 50 Travel to work: Train, walking
Figure 4.3: participants overview
RQ1: NAVIGATION TO FUNCTIONS “How does the participant navigate to the functionalities during the first time use?� During the test, and afterwards looking back at the footage that was filmed, the team made observations on how the participants completed the given tasks. For each participant the completion sequence was noted, and each action was given a name. A visualisation for each of the tasks is made that shows the three first steps the participants took when trying to complete the task. During the observations, the participants were rationally trying to find the functionalities after they received the task. After they have taken about three steps, they stop with trying to find using logic but start to press buttons in a random order. The first three steps they take are indications where the participants expect to find certain functionalities. This is the most important information that the team wants to find out from the test. This information will be used as basis for the research questions 2 and 3. Each task could be completed with only pressing one button, in the image the right action is indicated in bold text. Each dot represents one participant, and the colors indicate in which order they performed the actions. On the bottom of the figure is a table with the statistics of the task completement. In appendix A4.1 more elaborate tables are found for each task with the statistics per participant.
21
22
USER TEST RESULTS
TASK 0: SWITCH ON E-BIKE
TASK 1: CHANGE SUPPORT
TASK 2: FIND MAXIMUM SPEED
At the start of the test, the participants were asked to bike a predefined round with the E-bike system. This task is called ‘Task 0’ because it does not refer to one of the ‘Task scenarios’ from the procedure. In order to perform this task, participants had to switch on the E-bike system.
The first task that was given to the participants is:
“Je bent nieuwsgierig naar het snelste dat je hebt gereden tijdens de twee rondjes die je zojuist hebt gefietst met de E-bike. Zoek op hoe snel dit was.”
It is interesting to see that 4 participants assumed that when they start riding the bike, that the system will turn on (Figure 4.4). Most of the participants click the MODEbutton as their second action, none did it as a first step. The second interesting occurance to see, is that the participants search the display for a button. This happens during several of the other tasks as well. In the end, the participants were able to switch on the E-bike relatively quick, although it did take more steps than needed.
“Er is harde tegenwind en je hebt gemerkt dat je moeilijk tegen de brug op kwam. Je nadert een heuvel. Stel de E-bike op zo’n manier in, dat je tegen de heuvel op kunt fietsen.” All in all, the participants understood the functionality fairly fast and knew how to operate this function (Figure 4.5). One participant used the accelerator, at that moment he had no idea about the Up/Down-buttons and the indication of support level on the display.
Here, people could have tried out the buttons while riding the bike and therefore already have a better idea on how to operate it. However, there are interesting events that happened (Figure 4.6). The first of them is that 2 participants searched on the display for a button. Furthermore, two participants thought that the MODEbutton was a way to activate a mode function, after which they could navigate through it with the Up/Down-buttons.
Furthermore, an interesting phenomenon happened with one of the participants. He used the MODE-button, because he assumed that there was are certain ‘mode’ for this sort of situation: a setting that was especially tailored for this task.
When going through the modes and information that is displayed, the participants easily recognized the abbreviations MAX and AVG. Both of the TRIP modes were harder to understand, and none of the participants understood was ODO meant. Furthermore, retrieving the maximum speed was not something the participants expected to be a functionality of the system. When asked the question, they quickly realised this was a functionality of the E-bike system. After that they could find the maximum speed rather fast.
Figure 4.4: navigation task 0 Figure 4.5: navigation task 1
Figure 4.6: navigation task 2
USER TEST RESULTS
TASK 3: TURN ON LIGHT
TASK 4: SWITCH OFF E-BIKE
“Je merkt dat het begint te schemeren en ziet de lantaarnpalen aan gaan. Zet het licht van de E-bike aan.”
“Je komt thuis na een ritje met de E-bike en ben niet meer van plan om vandaag nog op de E-bike te fietsen. Berg de E-bike op in de fietsenstalling.”
None of the participants were able to finish this task within 10 steps and needed one or two hints to complete the task. As can be seen in figure 4.7, the main area where the participants tried to find a button for the light was on the lamp itself. Then, most of the other buttons are tried in the hope of turning on the light. When they could not find it, some of the participants just assumed or hoped that it will turn on automatically when it would become dark. All the participants gave up on this task. Also, only 3 participants recognized the light-icon on the display. Even when having completed the task, the other 3 participants hadn’t recognized the icon.
The main issue with turning off the system is that the participants accidently get in the Service Menu of the system (Figure 4.8). This confused them and makes them rule out the possibility of turning it off by holding the Mode-button for a couple of seconds. Some of the participants already turned off the system with the task before so they remembered how to do it. However, they still managed to accidentally get into the Service Menu. Two participants thought and hoped that it would turn off automatically after a period of inactivity. Two participants click on the On/Off button on the battery. One tried to take of the display, however, at the end of the test other participants told that they would want to take the display with them to prevent theft. One participant searched for a button on the display. Only three participants managed to switch of the system within three steps.
CONCLUSION For each of the tasks the participant had to execute, only one step was needed to complete it. In four out of the five tasks, participants took more than that one step. They found it hard to find all the different functionalities of the system in the three buttons on the handlebar. There are no indications on or near the buttons to tell what functionalities they could be used for. That is why participants tried a lot of different actions in the hope to complete the task. The most interesting findings from this user test are: Participants search for buttons/touchscreen on the display Participants assume that switching the system on/off will go automatic Many of the functionalities are not where the participant would think he could find them Abbreviations, language, and icons are not really clear The Up/Down-buttons are clear
Figure 4.7: navigation task 3
Figure 4.8: navigation task 4
23
24
USER TEST RESULTS
RQ2: USABILITY PROBLEMS “Which usability problems is the participant facing during the first time use?” By observations the team made, and the task completion sequences from question one, first insights were gained into which problems the user has while operating the E-bike system. By using the instant data analysis (Kjeldskov et al., 2004), all the problems from the test results were clustered. After clustering these different problems, they were divided into 8 main usability problems. Each of the problems are rated on their severity. The severity is based on frequency in which the problem occurred, the impact of the problem, and the persistence of the problem. Table 4.1 shows the problems, this table shows for each main problems the subproblems, the way it observed and heard, and which participants encountered the problem. A more elaborate table can be found in appendix A4.3. It was decided that the focus would lie on the main usability problems with a high severity(severity 3 or 4). These problems are severe enough to prevent the user to use the E-bike system without effort. If these problems are not improved, the user would not be able to fully understand the system and how to operate it. The problems 1-5 are subject to further analysis.
# 1
Severity
Problem interpratation
4
It is not clear how to switch on the E-bike system.
1.1
3
1.2
3
1.3
2
1.4
4
2
4
2.1
3
2.2
3
3
3
3.1
3
3.2
3
3.3
3
3.4
3
3.5
2
3.6
2
4
3
4.1
3
4.2
3
5
3
5.1
3
5.2
3
It is not clear how to switch off the E-bike system.
It is not clear how to turn on the lights.
Hard to operate one button at a time.
It’s not clear what the different MODES mean.
Table 4.1: overview of usability problems
Sub problem interpratation
Observerd / heard
Participants
Takes a lot more steps than necessary They expect physical buttons around the display Can’t find the button, this is where they expect it to find it. Can’t find button, assume it switches on automatically Assumes it switches on automatically, they expect this from an E-bike
Searches for physical buttons on display
P3, P4 & P5
Searches for physical buttons on display
P2, P3, P4 & P5
Says they assume it will switch on when starting bike.
P1, P2 & P5
Says they assume it will switch on when starting bike.
P1, P2 & P4
Doesn’t switch off the E-bike system It’s not clear how long to press the MODE-button to switch it off. Assumes it switches off automatically, they expect this from an E-bike
Going into the service menu
P8, P1, P3 & P5
Said that it turns off automatcally
P8 & P4
Needs 2 hints They expect physical buttons around the display Can’t find the button, this is where they expect it to find it. They expect to find the button on the light itself The icon is not clearly recognized as ‘light’ Can’t find button, assume it switches on automatically Assumes it switches on automatically, they expect this from an E-bike
Searches for physical buttons on display
P8
Searches for physical buttons on display
P8, P2 & P5
Checks dynamo to turn it on
P1, P2, P3, P4, P5 & P8
Doesn’t recognize light icon
P4, P5, P6 & P8
Says they assume it will switch on when starting bike. Says they assume it will switch on when starting bike.
P1, P2 & P8 P1, P2, P3, P4, P5 & P8
Needs 2 hints The buttons are too close to each other. They get another interpretation of the buttons as supposed.
Presses two buttons at once accidentally
P2 & P3
Didn’t notice that they pressed two buttons at once.
P1 & P5
Takes some time to understand the functions. They don’t recognize max as the maximum speed. It is not clearwhich button has to be pressed to find the max speed.
Skips over the right mode.
P1
Presses random button.
P4 & P8
USER TEST RESULTS
Line color:
Turn on the E-Bike
Adjust support +1 (0 to 5)
3
Change light status (on, auto, off)
Change mode values (trip (km),
trip (h), max. speed, avg. speed, odo)
Reset mode values
1
Indicates which button is pressed
4 5
Adjust support -1 (0 to 5)
Turn off the E-bike
2
Line thickness:
Indicates button press time = short press (< 2s) = long press (> 2s, < 4s)
Change speed status (current, avg., max speed)
Set the time
Service menu
TASKFLOW A task flow (Figure 4.9) is made of the E-bike system that gives an overview of the different functionalities of the E-bike. Each of the found main usability problems (UPr) is placed in this task flow. The grayed out boxes in the task flow are functions that were not tested, so there were no problems found for them.
Change service menu status (cells,
= very long press (> 4s)
battery, voltage, remaining cap.)
Overview main UPrâ&#x20AC;&#x2122;s
Leave menu
Time +1 (hour or minute)
Figure 4.9: task flow with usability problems
Switch between hour & minutes
Time -1 (hour or minute)
1. It is not clear how to switch on the E-bike system. 2. It is not clear how to switch off the E-bike system. 3. It is not clear how to turn on the lights. 4. It is hard to operate one button at a time. 5. It is not clear what the MODES mean.
25
26
USER TEST RESULTS
P1
PROBLEM DESCRIPTIONS For the main UPrâ&#x20AC;&#x2122;s examples are given of observations that the team made, or quotes from the participants are added to give richer insights into the problems. A more detailed description of the problems and their causes is given in the explanation of research question 3.
USER TEST RESULTS
27
28
USER TEST RESULTS
CONCLUSION By listing and clustering all the occurred problems that are observed during the tests five main usability problems are found. 1. It is not clear how to switch on the bike 2. It is not clear how to switch off the bike 3. It is not clear how to turn on the lights 4. It is hard to operate the buttons 5. It is not clear what the different MODES mean
USER TEST RESULTS
Figure 4.10: Causes UPr 1
RQ3: CAUSES “What are the causes of the found usability problems?” For each of the found usability problems it is important to know why these problems occur. The information from table 4.1 and the figures in ‘RQ2: Usability problems’ already give insights in what the causes of the usability problems are. For each of the main UPr’s is visualized what the causes are and what parts of the bike they refer to (Figure 4.10 4.14).
Figure 4.11: causes UPr 2
29
30
USER TEST RESULTS
Figure 4.12: causes UPr 3
Figure 4.13: causes UPr 4
Figure 4.14: causes UPr 5
USER TEST RESULTS
PROBLEM DESCRIPTIONS
Figure 4.15: Relations of the causes
The first three usability problems are facing with comparable causes. Figure 4.15 illustrates how these causes could be related to each other. This doesn’t mean that the causes will always affect each other. One important cause is that, there are no use-cues on the buttons for switching on/off or turning on the light. This can influence if and where people will search for a certain button on another place. Besides, the multiple functions of the buttons make it difficult for the participants to understand the meaning of the buttons and it will be more complex to provide the user the right use-cues. Another recurrent cause is that people assume some functionalities will happen automatically, because the participants cannot find the right functionality or expect that from an E-bike. Also, the usability problem(4) with the physical buttons has an influence on how participants interpreted the operations of the buttons. It can change their previous interpretation and can influence their next action. The causes for UPr-5 are slightly divergent and not directly related with the other causes. The problems seem not that much important for the user, but has a notable role on the user interface and buttons. Although, the information of the UI is too little to directly understand the functionality.
RQ4: EXPERIENCE E-BIKE “How does the participant experience the E-bike during first time use?” In order to get an answer on the how the user experiences the E-bike, the participants were asked to fill in a SUSquestionnaire (Affairs, 2013). Some of the statements are reformulated so that all the statements were asked in a positive manner. This makes it for the team easier to get an overview of the experience of the participants. The complete SUS-questionnaire can be found in appendix A4.4. Four of the ten statements are left out of the analysis because the results were not relevant to the research question or could not be tested. Figure 4.16 shows how the participants rated the remaining statements. The colors indicate how negative(red) or positive(green) the participants rated the different questions. In this way, it is easy to identify which statements and thus the overall experience can be improved.
31
32
USER TEST RESULTS
SUS QUESTIONNAIRE RESULTS 2. I donâ&#x20AC;&#x2122;t find the E-bike system unnecessarily complex.
6 I thought the E-bike system is created with a good consistency
1) 3 2) 0 3) 1 4) 1 5) 1
1) 3 2) 2 3) 0 4) 1 5) 0
3 I thought the E-bike system was easy to use.
8 I found the E-bike system very wieldy to use.
1) 2 2) 2 3) 1 4) 0 5) 1
1) 4 2) 2 3) 0 4) 0 5) 0
5 I found the various functions in this E-bike system were well integrated
10. I donâ&#x20AC;&#x2122;t need to study the manual before I could get going with this E-bike system
1) 1 2) 3 3) 0 4) 2 5) 0
1) 3 2) 1 3) 1 4) 1 5) 0
Completely disagree
Figure 4.16: SUS questionnaire results
Completely agree
CONCLUSION The results from the SUS-questionnaire gives an impression on how the participants experienced the product. The results of this questionnaire are coherent with the previous identified problems. The results of statement 2 and 8 underpin that the participants experience the E-bike system as complex and cumbersome. The results of statement 6 and 10 are also negative relative to the E-bike system. More than half of the participants gave the lowest score to these statements. This means that the system interface will not be experienced as consistent and user-friendly. Statements 3 and 5 also show negative results, however, not as negative as the others. These interactions with the system are not the biggest problems perceived by the participants. The assumption was made is that people may be less severe on judgements related to their personal performance.
USER TEST RESULTS
P8
RQ5: EXPECTATIONS OF THE E-BIKE “Does the E-bike fit the expectations of the participant during first time use?” During the user test, the participants were asked to explain what they expected of the E-bike, and especially how it differs from their regular bike. At the end of the user test the participants were asked how the E-bike met their expectations. Figure 4.17 shows and compares which participants had which expectations before the test and what their experience of the bike was after the test. As can be seen in the figure, most of the participants expect the E-bike will bike easier and faster, especially for long distances or going uphill. Besides, less maintenance and a long battery life are the other expectations that were frequently mentioned by the participants. Because the user test is based on first time use, the participants didn’t provide any feedback about the maintenance and battery. At the end of the user test, the participants experienced the E-bike bikes faster and more comfortable than they initially expected. This makes them imagine that they can travel longer distances with the E-bike.
P5
P6
P4
P5
P3
P4
P2
P3
P8
P1
P2
P4
Faster with less energy
P4
No sweat
P2
Good battery life
P4
P2
P4
P2
P1
P2
Easy to bike on height differences
P2
Travel longer Less distance maintenance
P2
P3
Comfortable to use
= Expectation E-bike (before test) = Experience E-bike (after test)
Figure 4.17: expectations & experience of the user
CONCLUSION In conclusion, for most of the participants, the speed and comfort of the bike exceeded their expectations. Unfortunately, they didn’t talk about the display and what they expected from it. This could mean that the user interaction with the display is not their first priority, however, an intuitive interface is needed to operate the system easily.
33
34
USER TEST RESULTS
COMPARISON RESULTS TARGET GROUP & USER GROUP
When executing the user tests, it was decided that next to the 6 target-group participants, two people from outside the target group would participate as well. Their results are not taken into account in analysing and drawing conclusions, but are used to compare and see to what extend the results are similar between the groups. In appendix A4.5, an overview of the complete UPr table is given, including P6 & P7. P6, a TU Delft student (‘other’), scored low on ‘severitypoints’: 5 out of an average of 15.75 (where each experienced problem is multiplied by its severity: appendix A4.5). She had a very inquisitive attitude and a high technological knowledge which might have resulted in the low number of found UPr’s. P7 (Figure 4.18), an elderly woman (‘elderly’), ended up with just above average ‘severity-points’: 18. Here, there are no clear conclusions that can be drawn. Part of her problems are explained by the fact that she had prior experience with E-bikes and expected the Sparte F8E to behave in the same way, which caused confusion. Also, her limited technological knowledge (which she indicated herself), might have influenced the results. Because so few user-group participants were tested, only assumptions can be made. From this research, it is assumed that there are no clear differences between the target- and tested user-group. The UPr’s are divided equally over the different main UPr’s, not seeing clear patterns (not for example elderly had more problems with controlling the buttons). Our ‘other’ participant did have less ‘severity-points’ than average, but this could have easily been that she does not properly represent this usergroup, so no hard conclusions can be drawn from it.
Figure 4.18: elderly participant
USER TEST RESULTS
DISCUSSION In research it is important to critically look back upon your results, and look at the limitations of your research. One of the limitations of our research is limitation is that some people accidentally discovered some features during the the earlier steps of the user test. In doing so, these participants already knew how to complete these tasks later on in the test. However, it’s possible to trust the data collected because the team was able to detect problems related to the spoiled tasks by looking at the actions of the other participants. Some limitations were related to research questions. For instance, the fourth one is about subjective experience of the bike. Testing such a subjective aspect is very difficult, because results are related to the users personality, and different people may provide different results. However, there is a clear trend in the way participants filled in the questionnaire. For this reason it’s still possible to consider the data about e-bike as valid.
In this questionnaire, people also were asked to evaluate their experience with a scale from 1 to 5. In doing so the data collected were not fully explicative of the experience. However, with this method is possible to understand in which aspects of the E- bike system are lacking, and it provides a method for a future comparison. Also, in the translation from Dutch to English, some of the information got lost in translation. As a matter of fact, the whole thinking aloud part was in Dutch, and some expressions were difficult to translate. On the other hand, a lot of time was spent in searching for the correct expression, and all of the most important parts were carefully analyzed after this phase. The three Dutch group members watched to keep the data lost in translation at a minimum. Another limitation is related to research question five, where the SUS questionnaire was used. Some of it’s questions are about the user’s experience over a longer period of time, while the test only took place 45 minutes. Therefore it is hard to say something about a long term experience. However, good insights were given
while asking the participants of their expectations. And they were able to realistically imagine how they would experience it over a longer period of time. A final considerable problem is about the task where the participant should turn on the light. During the test, the participants were asked to turn on the light during daytime, while the standard light setting is on ‘automatic’. This is a scenario that would most likely not happen in real life. The problems encountered during this task would therefore not happen in real life, so they could be discarded as serious problems. However, there are cases where the user would like to make use of this function (e.g. mist and rain). Also, this does affect the intuitivity of the system and underpins the previous found problems in the usability of the system interface. This is why the team decided to regard it as a serious problem.
35
36
DESIGN BRIEF
05
DESIGN BRIEF The design brief describes the focus for the redesign of the E-bike system â&#x20AC;&#x153;Sparta F8Eâ&#x20AC;?. The design goal is based on the results of a previous user research, which investigated how intuitive the E-bike system was during the first time use of inexperienced users. During the user test, six different participants were tested within the chosen target group of commuters. The user test pointed out five main usability problems. These problems are described into this design brief and why they should be redesigned. From these problems, a design goal is formulated in which is explained how the user should experience the redesign. The interaction vision gives inspiration in how the design team wants that the interaction with the E-bike system should feel. To test if the redesign meets the state design goal a list of requirements and wishes has been drawn. These design specifications are also based on the usability problems and will be used to compare the redesign with the current product.
Figure 5.1: Design brief
DESIGN BRIEF
MAIN USABILITY PROBLEMS
37
38
DESIGN BRIEF
Figure 5.2: Interaction vision
DESIGN BRIEF
DESIGN GOAL “Commuters should be able to use and understand the E-bike system without receiving instruction during their first time use.” INTERACTION VISION “Users should experience this E-bike as coming home after a holiday.” During most holidays, you literally go away from your daily routine and home. When coming home after the holiday (Figure 5.2), you have to get used to your own house. At first, everything will feel new. But you quickly get used to your familiar and secure surroundings and adapt fast into your normal way of life again. You know what you can expect from your home and where you can find things with respect with your holiday residence.
QUALITIES Familiar
You have a memory and experience with most products in your home. You know what you can expect from them, because you have a daily experience with your home. The user should also have this familiar feeling with the E-bike, when using it for the first time. The user should know what he can expect from the bike, before he has experienced it yet.
Secure
Because of your everyday experience of your home, you can find things with your eyes closed. After a holiday you didn’t have that experience and therefore, you will explicitly realize this when coming home. It takes some time to find that familiar place in your home. It feels new for a short time. This novelty the user also will have during the first time use of the E-bike, but we want to give the user the security that he can easily find the functionalities and also realize this.
DESIGN SPECIFICATIONS The redesign should eliminate all of the current critical problems. After analysis of the second user test we should not find any of the critical UPr’s we detected before. The team should not discover any new critical problems during the second user test. After analysis of the second user test we should not find any new critical UPr’s. The participants should interact with the E-bike system more intuitively. The participants should complete the task scenarios in less steps than during the previous user test. People should recognize more of the functions of the E-bike system. We will ask the participants to guide us through the functions of the E-bike system and compare the number of functions with the previous results. The experience of the E-bike system use should be improved through the redesign. The answers to the SUS for the current- and re-design form will be compared by adding all of the given scores per question. When the total of the re-design is higher the experience is improved. Participants expectations should still meet their actual experience. We found that participants got what they expected from the E-bike: this should still be the case with the redesign.
39
40
REFERENCE LIST
Affairs, A. S. (2013). System Usability Scale (SUS). Retrieved April 10, 2017, from https://www.usability.gov/how-toand-tools/methods/system-usability-scale.html Engelmoer, W.(2012).The E-bike: opportunities for Commuter Traffic,Retrieved April 10,2017, from http:// www.fietsberaad.nl/library/repository/bestanden/Masterthesis_ebike-for-commuter-traffic.pdf Fietsersbond. (n.d.). Is een e-bike slecht voor mijn conditie? Retrieved April 10, 2017, from https://www. fietsersbond.nl/de-fiets/fietssoorten/elektrische-fietsen/ voor-wie/ Hendriksen, I., Engbers, L., Schrijver, J., Gijlswijk, van, R., Weltevreden, J., Wilting, J. (2008). Elektrisch fietsen. Marktonderzoek en verkenning toekomstmogelijkheden. Retrieved April 10, 2017, from http://www.fietsberaad. nl/library/repository/bestanden/Rapport%20KvLB&G-2008-067%20compleet.pdf Kjeldskov, J., Skov, M. B., & Stage, J. (2004). Instant data analysis. Proceedings of the third Nordic conference on Human-computer interaction - NordiCHI â&#x20AC;&#x2DC;04. doi:10.1145/1028014.1028050
06
REFERENCE LIST
Top 10 Hell. (n.d.) Top 10 Countries with Most Bicycles per Capita. Retrieved April 10, 2017, from http://top10hell. com/top-10-countries-with-most-bicycles-per-capita/ Zee, van der, R. (2016). Can e-bikes revolutionise longdistance commuting? Retrieved April 10, 2017, from https://www.theguardian.com/cities/2016/sep/14/ebikes-long-distance-commuting-speed-pedelec-electriccycles
42
APPENDIX
A2.1 ANALYSIS PRODUCT The product is a bike with pedal support, the E-bike. The user can get extra support from the bike to be able to go faster while requiring less energy, power and effort from the user. The main contact points between the user and the system providing the support are the display, the buttons, the accelerator, the peddles, and the battery. The total system consists out of these four attributes and a motor which executes the support.
DISPLAY
The display is placed on the middle of the handlebar so the user can see it clearly while biking. The controls are placed at the left handle, with these controls the user can change the settings, modes, and support intensity. The display displays the information on the statuses of the settings, modes, and the intensity level.
ACCELERATOR
A2
Figure A2.1: overview of display functions
The accelerator (Figure A2.3) is placed on the left side of the handlebar. Is separated from the handle so it can be turned around the axis by the user. The accelerator can be used to give the bike speed without pedalling. It can be speed up to 3 km/h. This function can be used for example to â&#x20AC;&#x2DC;startâ&#x20AC;&#x2122; the bike and getting the bike in motion before starting to pedal. Furthermore, it can be used while walking next to the bike, to give extra support when walking uphill, since the bike is quite heavy. And finally, it can be used while biking to give some extra impulse to the pedal support, for instance when biking uphill or overtaking someone.
PRODUCT ANALYSIS Figure A2.2: overview of display
APPENDIX
PEDALS
The pedals (Figure A2.5) are used to move the bike forward. The pedal support is the main function of the E-bike. When the user increases or decreases the pedal support with the button on the left handle, he will feel the changes in the pedals. The user receives feedback on his actions through the pedals. Figure A2.5: pedals
BATTERY
The battery (Figure A2.6) provides the power for the pedal support, lights, and display. The battery should be charged regularly by regularly use. On the left side of the battery are five lights that indicate how full the battery is. Each light represents 20% of the battery power. An indication of the battery is shown on the display as well.
Figure A2.3: accelerator
BUTTONS
Figure A2.6: battery
MOTOR
The motor (Figure A2.7) is located in the front part of the bike, between the wheel and the handlebar. All the inputs from the buttons and accelerator converge in it. This part has the function to supply with energy the front wheel, depending on the support grade the user choose and also depending on the twisting of the accelerator.
Figure A2.4: overview of buttons functions
Figure A2.7: motor
43
44
APPENDIX
A2.2 RELATION BETWEEN COMPONENTS The E-bike consists of two main parts: the bike and the E-bike system. For this project the focus lies on the system, and there will not be taken a look at the physical design of the bike itself. An overview (Figure A2.8) is given on the different parts of the systems and how they relate to each other and the user. 1. With the left hand, the user can operate the buttons and the hand accelerator. 2. The user can see all relevant information on the display. 3. The pedals are used to move the bike and these are points where the user feels the bike support. 4. The user can remove the battery in order to charge it. The battery is able to show how fully charged it is. 5. While clicking the buttons the user gets feedback through the display. 6. The buttons can be used to increase and decrease the support level and switch between modes with information about the trip.When using the accelerator, the motor starts accelerating the bike. 7. The battery supplies the display with power, the display shows how much power the battery has. 8. The battery supplies the motor with energy. 9. The motor provides the pedals with the right amount of power.
2
1
6
5
8 4 7
3 9
Figure A2.8: relations between user & E-bike
APPENDIX
A2.3 TASKFLOW This task flow (Figure A2.9) was made to provide an overview of the different functionalities of the E-bike, and visualize the way to enable these functionalities (the button that is required to press and the corresponding time to press it)9
Adjust support +1 (0 to 5)
Change light status (on, auto, off)
Change speed status (current, avg., max speed)
Turn on the E-Bike
Change mode values (trip (km),
trip (h), max. speed, avg. speed, odo)
Reset mode values
Set the time
Adjust support -1 (0 to 5)
Line color:
Indicates which button is pressed
Turn off the E-bike
Service menu
Change service menu status (cells, battery, voltage, remaining cap.)
Line thickness:
Indicates button press time = short press (< 2s) Leave menu Figure A2.9: taskflow
Time +1 (hour or minute)
Switch between hour & minutes
Time -1 (hour or minute)
= long press (> 2s, < 4s) = very long press (> 4s)
45
46
APPENDIX
A3.1 IMPRESSION OF USABILITY In order to get an impression of the usability and experience of the Sparta F5E each member of the team used the product, online reviews were looked up, bike shops were visited and another Skype meeting with the company stakeholder was held.
IMPRESSION OF USABILITY & EXPERIENCE In order to get an impression of the usability and experience of the Sparta F8e each member of the team used the product, online reviews were looked up, bike shops were visited and another Skype meeting with the company stakeholder was held.
PERSONAL USE OF THE PRODUCT
Each of the Dynamo team members used the product for about 15 minutes. Afterwards, a summary of the use was written down, highlighting the strengths and weaknesses of the usability and experience with the product.
A3
USER TEST PLAN
CONSUMER AND PROFESSIONAL REVIEWS
Also, online research was done on the Sparta F8e. Reviews were gathered and recurring problems and strong points were written down. Then, a bike shop was visited to ask
about their opinion on the Sparta F8e. It was mentioned that most people return to her when they have technical issues, not with usability problems. Because of this she could not help us defining the usability and experience. Input from company stakeholder Finally, another Skype meeting was planned with the company stakeholder. In this meeting more in depth questions were asked (summary of the meeting will follow in the next deliverable) about the usability of the product, confirming some of the previously found strong and weak points.
OVERVIEW FINDINGS
The findings of the previously mentioned research techniques are visualized (figure A3.1 - A3.3). Figure A3.1 and A3.2 give the strengths and weaknesses related to the E-motion E-bike system. Figure A3.3 shows the +/points related to the other parts of the product that we have to keep in mind when redesigning the product. The colors underneath each of the boxes show where this information was found. Some similar ones are grouped by the type of problem, showing themes that occur more often. The bars are the side indicate the severity of the problem (cluster).
APPENDIX
STRENGTHS
Figure A3.1: Strenghts operating & feedback of the E-bike system
47
48
APPENDIX
WEAKNESSES
Figure A3.2: Weaknesses operating & feedback of the E-bike system
APPENDIX
Figure A3.3: Strenghts & weaknesses other parts E-bike
49
50
APPENDIX
A3.2 PROCEDURE “The goal is to gain insights into how commuters use and experience the E-bike system during the first time use to discover usability problems, its causes and if the E-bike system meets the expectations of the participant.” There are five main research questions that will be answered by executing an user test. For each of these question is described what the learning goals are. 1. How does the participant navigate to the functionalities during the first time use? 2. Which usability problems is the participant facing during the first time use? 3. What are the causes of the found usability problems? 4. How does the participant experience the E-bike during first time use? 5. Does the E-bike fit the expectations of the participant during first time use?
LOCATION
The set of the test took place at three different locations in Delft, due to the availability of the participants. The participants had to bike a round in a quiet place without traffic. The test has been done in an in sitro environment, because this is safer than testing in real traffic, and the team could control the situation. Furthermore, within this location, the team was able to simulate a certain realistic scenario, which is stopping with the bike. Two actor simulated a scenario, where the participant had to stop the bike. The participants were not allowed to use gloves or sunglasses, so the they wouldn’t influence the usage of the bike.
PRODUCT SETTINGS
In order to have consistent testing, the settings of the bike were the same for each participant. Figure 3.3 portrays the display with the settings.
PARTICIPANTS
Six commuters with ranging ages, and commuting time participated in the test. Two people outside this target group participated in the test as well and were used to compare the results.
OBSERVATION TECHNIQUES
During the usability test, the participant will be recording the handlebar and his hands with a GoPro camera on his head. After the test, the team can look back at the footage and see what happened during the ride. During the user test, the facilitator of the test and the participant will look back at the footage and will be asked questions about what and why he did certain things.
PROCEDURE
Table A3.1 shows a quick overview of the test procedure. Below, a more elaborate description of the different steps of the procedure can be found. The test will start with a short interview about the expectations of the user followed by an usability test, task scenarios and an interview. The usability test will give insight in how people use and experience the product in natural settings without giving an explanation about the E-bike. The assumption is made that after the usability test, the participants hasn’t used all the functionalities due to the short distance or because there is no need to use it. These functionalities should still be researched in order to find out if they are easy-to-use in during the first time use. Therefore, the participants receives some task scenarios to execute, and they will be asked questions about their experience with the E-bike. The user test will end with some concluding questions to discover their underlying thoughts about the E-bike. The instructions of the user test are in Dutch, because is easier for the participants to speak and explain their feelings in their mother tongue. Talking in another
language could form a barrier for the participant to freely express their thoughts.
1. Interview
The test facilitator will brief the participant about the planning of the user test and asks the participant some questions about their expectation of the bike. It is important that the participant will not come in contact with the E-bike before these questions. Waarmee reis je dagelijks naar je werk/studie? Als je dit vervoermiddel zou vervangen voor een E-bike, wat zou de E-bike dan volgens jou allemaal moeten kunnen? (Ask why) Wat verwacht je dat het verschil tussen je normale fiets en een E-bike?
2. Usability test
The facilitator helps the participant with putting on the GoPro and explains to ‘think aloud’ in order to be able understand their thinking and choices. Task 0: Using/Experiencing the E-bike The facilitator gives the participant a key, designating the E-bike and asking: “Kun je met deze E-bike voor ons 2 rondjes rijden?” The participant will be shown a map on how he should drive. There should be no further explanation about how to use the E-bike system, because the test focuses on the first use and experience with the E-bike.
3. Task scenarios
The facilitator will ask the participant to do tasks standing in front the E-bike, which makes it easier to observe the tasks. This can have an influence on the results, because they have a fewer amount of tasks to do simultaneously (steering and watch out for traffic). The facilitator will encourage the participant during these tasks to think
APPENDIX
aloud. The most important features are chosen to test, based on the task flow and the first impressions of the product. Task 1: Change support “Er is harde tegenwind en je hebt gemerkt dat je moeilijk tegen de brug op kwam. Je nadert een heuvel. Stel de E-bike op zo’n manier in, dat je tegen de heuvel op kunt fietsen.” Hint: “Kan u dit doen zonder de versnelling te veranderen?” Kunt u mij uitleggen wat je daarvoor precies hebt gedaan? Task 2: Change mode settings “Je bent nieuwsgierig naar het snelste dat je hebt gereden tijdens de twee rondjes die je zojuist hebt gefietst met de E-bike. Zoek op hoe snel dit was.” Kun je vertellen hoe je erachter bent gekomen? Task 3: Turn light on “Je merkt dat het begint te schemeren en ziet de lantaarnpalen aan gaan. Zet het licht van de E-bike aan.” Waar verwacht u deze functionaliteit te vinden? Hint: “Je zou bij de knoppen op het stuur kunnen kijken.” Wat denk je dat het verschil is?
4. Interview
The facilitator and the participant will look back together the movie of the trip. The facilitator will ask questions about the actions of the participant with questions as “Kan u uitleggen waarom je ‘...’ deed?”, “Wat gebeurde er op dit moment?”.
5. SUS questionnaire
Watching the footage back will help the participant to evaluate the system on its experience. After looking back the video, the participant will fill in the System Usability Scale questionnaire(appendix 4.3). Questions In welk opzicht voldoet deze E-bike aan uw verwachtingen? Waar was u verder tevreden over? Kan je uitleggen waar je ontevreden over was? Hoe zou je dit willen verbeteren? Waarom zou je dit product wel/niet in de toekomst aanschaffen?
Task 4: Turn off E-bike system “Je komt thuis na een ritje met de E-bike en ben niet meer van plan om vandaag nog op de E-bike te fietsen. Berg de E-bike op in de fietsenstalling.” Kun je mij vertellen waarom je “...” doet? Task 5: Understand E-bike system “Zou je me nog één keer mee kunnen nemen door alle mogelijkheden van de E-bike?” Kun je mij vertellen wat je onlogisch vindt? Hoe zou je dit willen verbeteren? Wat zijn hiervan de functies die je vaker zou gebruiken?
Table A3.1: overview of the procedure
51
52
APPENDIX
A4.1 NAVIGATION OF FUNCTIONS
A4
USER TEST RESULTS Table 4.1: steps to complete a task
APPENDIX
A4.2 USER TEST RAW DATA In order to structure all of the data that was collected during the testing, a predefined Google questionnaire was used (link: https://goo.gl/1Fbv0F). This ordering of data made it easier to later make the step towards the Usability Problem table using the Quick Data Analysis method. A PDF of all of the data that was filled in through the questionnaire can be downloaded on WeTransfer (https:// goo.gl/EZ8hLN). Besides this questionnaire, a spreadsheet was made for each of the task scenarios, structuring the different steps that were taken by the participants (https:// goo.gl/U2G6WF).
#
Severity
Problem interpretation
1
4
1.1
3
It is not clear how to switch on the E-bike.
1.2
3
1.3
2
1.4
2
2
4
2.1
3
2.2
3
3
3
3.1
3
3.2
3
3.3
3
3.4
3
3.5
2
3.6
2
4
3
4.1
A4.3 USABILITY PROBLEMS OVERVIEW The table 4.2 shows for each problem the manner in which it was presented. Was it an observation or was it something the participants said. Furthermore, it shows which part of the bike the problem belongs to, which participants experienced the problem, during which task the problem occurred and finally, the severity of the problem.
4.2
3
5
3
5.1
1
5.2
2
6
2
7
2
8
1
It is not clear how the E-bike switches off.
It is not clear how to turn on the light
It is hard to operate one button at a time.
It's not directly clear to understand what the modes mean.
It is not clear that the display has no touchscreen. It's not clear that the display cannot be taken off the Ebike. It is not clear how to use the accelerator
Sub problem interpretation
Observed/heard
Participants (target group)
Participants (user group)
Takes a lot more steps than necessary They expect physical buttons around the display They cannot find the right button and this is a place where they expect to find it. They cannot find the right button to turn on the E-bike so assume that it happens automatically They expect the E-bike will switch on automatically, because they assume this from an electrical product.
What?
Switch on bike
Buttons
Searches for physical buttons on display
P3, P4, P5
Switch on bike
Display
Searches for physical buttons on display
P2, P3, P4,P5
Switch on bike
Display
Thinks it turns on when starting with biking
P1,P2,P5
P7
Switch on bike
Buttons
Thinks it turns on when starting with biking
P1,P2,P4
P7
Switch on bike
Buttons
Switch off bike
Buttons
Switch off bike
Buttons
Switch off bike
Other
Turn on light
Buttons/UI
Turn on light
Display
Turn on light
Display
P7
Turn on light
Other
P7
Turn on light
UI
Turn on light
Buttons/UI
P6
Turn on light
Buttons/UI
P6
-
Buttons
-
Buttons
Mode function
Buttons/UI Display/UI
Don't turn off the bike
It's not clear how long to press the Going into the service menu mode-button to turn off the E-bike. They expect the E-bike will turn off Said that it turns off automatically automatically, because they assume this from an electrical product. Needs 2 hints(and then still presses random buttons) They expect physical buttons Searches for physical buttons on display around the display Searches for physical buttons on display They cannot find the right button and this is a place where they expect to find it. They expect to find the button on Checks dynamo/light to turn it on/off the light itself The icon is not clearly recognized Doesn't recognize light icon as 'light' They expect the light will turn on Assumes that light turns on automatically automatically, because they cannot find the correct button. Assumes that light turns on automatically They expect the light will turn on automatically, because they assume this from an electrical product.
P8, P1, P3 ,P5
The buttons are too close to each other They get another interpretation of the buttons then supposed.
Presses two buttons at once accidentally
P2, P3
Didn't notice that they presses two buttons at once Takes some time to understand the Mode function
P1, P5
They doesn't recognize max as Skips over the right mode max speed It is not clear which button they Presses random buttons have to use to find the max speed Touched display
Table 4.2: overview of all usability problems
Task
P8, P4
P8 P8,P2,P5 P1,P2,P4,P5,P8, P3 P6,P8,P4, P5
P6 & P7
P7
P1, P2, P8 P1,P2,P4,P5,P8,
P4, P1
P7
P1,P8, P3
P7
P3,P4,P2, P8,P1
P7,P6
Find max speed Find max speed -
Wants to take off the display
P2,P3
P6
Switch off bike
Display
Use it randomly(wrong)/not used at all
Everyone
P6 & P7
-
Other
Buttons Display
53
54
APPENDIX
A4.4 SUS QUESTIONNAIRE The System Usability Scale questionnaire Helemaal mee eens 1. Ik denk dat ik het prettig zou vinden het E-bike systeem vaker te gebruiken 2. Ik vond het E-bike systeem onnodig ingewikkeld 3. Ik vond het E-bike systeem makkelijk in gebruik 4. Ik denk dat ik hulp nodig heb van een technisch persoon om het E-bike systeem te kunnen gebruiken
Helemaal mee oneens
A4.5 COMPARISON TARGET GROUP & USERGROUP The table 4.3 shows for each p roblem the manner in which it was presented. Was it an observation or was it something the participants said. Furthermore, it shows which part of the bike the problem belongs to, which participants experienced the problem, during which task the problem occurred and finally, the severity of the problem.
5. Ik vond de verschillende functies in het E-bike systeem goed geĂŻntegreerd 6. Ik vond dat verschillende functies binnen dit E-bike systeem onsamenhangend waren 7. Ik kan mij voorstellen dat de meeste mensen makkelijk leren om dit E-bike systeem te gebruiken 8. Ik vond het E-bike systeem moeilijk te gebruiken 9. Ik voelde me heel zelfverzekerd terwijl ik het E-bike systeem gebruikte 10. Het duurde lang voor ik het E-bike systeem goed door had om er echt mee aan de slag te kunnen
Table 4.3: severity rating between participants
APPENDIX
55