EXECUTIVE SUMMARY
GROUP B3
Mees Daalder (4228979) Linda Smit (4212126) Emanuele Gandini (4663667) Hanna Timmerman (4605985) Nasibeh Soltani (4622464)
FINAL REPORT UXAD | ARNOLD VERMEEREN & RENE VAN EGMOND
1
2
EXECUTIVE SUMMARY
00
EXECUTIVE SUMMARY This report presents an usability research of the Sparta F8E, specifically on the E-bike system: the parts of the E-bike that enable the user to operate and get feedback on the different functionalities: the buttons, display and User Interface (UI). User tests are conducted to uncover usability issues with the current design. This is done with participants from the chosen target group: commuters without prior E-bike experience. Participants are presented with task scenarios in which different functionalities of the system have to be used. Also, the test investigates the usability experience of the system. The analysis of the gathered data leads to different usability problems in using the current E-bike: 1. It is not clear how to switch on the bike 2. It is not clear how to switch off the bike 3. It is not clear how to turn on the lights 4. It is hard to operate the buttons 5. It is not clear what the different MODES mean For each of these problems the causes are discussed. One of the main causes for different problems is the lack of clear use-cues to inform
the user what functionalities can be accessed through the different buttons. Furthermore, it is found that there are too many functions for each button, confusing the participants.
the different data of the trip in a second screen, and navigate through them easily. The light is taken out of the system and a button is placed on the front light.
A questionnaire was used to measure the participants usability experience of the system. This shows that they found it complex and cumbersome in use, and the system was not experienced as user friendly.
The second user test is kept similar to the first, in order for the results to be comparable. Through comparing the results from both tests, it is found that the previously found problems are solved, and the usability experience improved. However, four new problems are found, of which two are still a minor problem after taking the limitations into account:
All results are then combined into the design brief, which is used as the basis for the redesign. The design goal for the redesign is: “Commuters should be able to use and understand the E-bike system without receiving instruction during their first time use.” In addition, a list of design specifications is set up, which the redesign should meet. The redesign remains close to the current product. The main changes are: a separate ‘power’ button on the display, the composition of the buttons on the handlebar, the interface, and the light. The interface is now split up in a “home screen” portraying the current speed, battery, support level, and current time. The user can find
1. It is not clear how to scroll through the trip info 2. It is not clear when a trip starts and ends Finally, recommendations for this two problems and the suggestions mentioned in the user test are described.
3
00
TABLE OF CONTENTS
06 08 15 04 | REDESIGN PROPOSAL
03 | FIRST USER TEST
02 | PRODUCT ANALYSIS
01 | INTRODUCTION
4 TABLE OF CONTENTS
28
37 55
07 | DREFERENCES
06 | RECOMMENDATIONS
05 | SECOND USER TEST
TABLE OF CONTENTS
58
5
6
INTRODUCTION
Hanna
01 INTRODUCTION This report summarizes the information collected in five months of work, in which team Dynamo studied the E-bike’s system. In the first part of this document, the most important Sparta F8e’s parts are analysed, and a description of the chosen target group is presented to the reader. Studying of the product and its context has been fundamental to Team Dynamo to set up the user test described in chapter three, whose aim was to highlight usability problems investigating participants interaction with the E-bike’s system. Indeed, the results of the driven user test spotted five main usability problems, which were used as a starting point to generate the redesign solution in chapter four. The team’s proposal is the result of a synthesis phase, whose aim is to
find the best solutions for the usability problems, always keeping in mind the chosen target group’s needs. In order to verify if the design changes solved the main usability problems, another user test has been set up. In this case, participants had to perform similar tasks to the first user test, interacting with a prototype of the redesign proposal. The outcome of this phase is described in chapter five, in which procedure, analysis and results of it are presented to the reader. Eventually, In the final part of this report, conclusion and recommendation for the company are presented, to show achievements of the design process and where further research has to be done.
EXECUTIVE SUMMARY Mees
Emanuele Linda Nasibeh
7
8
PRODUCT ANALYSIS Figure 2.1: Sparta F8E
02
PRODUCT ANALYSIS This chapter gives an introduction in the product: the E-bike ‘Sparta F8E’ (figure 2.1). The use of this E-bike is dependent of the context and the users. Therefore, the context and user groups are analysed and a target group is chosen to give a more focused advice. This target group is analysed and a storyboard is provided to show the interactions of this users with the product. At the end, an overview of the product and functionalities is given.
PRODUCT ANALYSIS
USER GROUPS SYMBIOTIC PRODUCTS
OTHER PEOPLE
-CLOTHES -BACKPACK-BAGS -SUNGLASSES
-TRAFFIC -COMPANION -CHILDREN -FRIENDS
-GLASSES/LENSES -CHILD SEATS -GLOVES -BASKETS
Daily commuter: a long distance commuter, who travels every day by bike to work and home.
Parents with children: parents who bring their young children to school on their bike can use the E-bike to get to the school quicker and easier.
-ELDERLY -LONG-DISTANCE COMMUTER -PEOPLE WITH PHYSICAL LIMITATION -PARENT WITH CHILDREN -OTHERS
-BIKE - SYSTEM
PRODUCT-SERVICE
COMBINATION
USER
SUN/RAIN/WIND/COLD/HEAT/TRAFFIC/LIGHT /DARK/BAD ROAD/FLAT/HILL/NIGHT/DAY ENVIRONMENT
Figure 2.2: human interaction system
CONTEXT OF USE Figure 2.2 visualizes the context during the use of an E-bike. For each of the components in the context, there are multiple factors that could influence the context and are important to take into account during the user test.
Elderly: people with more physical limitations can go out for a longer trip than with a regular bike. It could give them a new sense of independence and freedom. People with physical handicaps: younger people with physical handicaps that prevent them from riding a regular bike, can use the E-bike as a good alternative. Other: the remaining group that have no particular reason to use an E-bike. They only use the E-bike to bike easier with the support.
Figure 2.3: user groups
9
PRODUCT ANALYSIS
Luuk Meijer, 31 Nationality: Dutch
TARGET GROUP Due to the big variety of user groups, the team decided to focus on one of them: commuters without any previous E-bike experience. In recent years, this group is increasing in size (Hendriksen et al, 2008). Approximately 25% of the commuters in the Netherlands are using a bike as main transport to go to work (Top10 Hell, n.d.). Also, Dutch companies, universities, local and regional authorities offer their employees bonuses when they come to work on an electric bike. Thus, com muting with E-bikes is becoming more common (van der Zee, 2016).
Luuk is an active person that likes to spend his free time on outdoor activities.
The persona in figure 2.5, provides an overview of Luuk: an average person from our target group. He lives about 12km from his work, and instead of using public transport, Luuk is considering buyings an E-bike. $
10
He got married recently and he need to save some money to buy their own house.
Luuk works in a big office, where appearance is important. He doesn’t want to be sweaty when he arrives at work. He uses technology often, but he doesn’t have time to read extensive manuals to learn how his devices work.
Luuk lives in Delft, but works in Den Haag. He uses the public transport to commute every day.
Figure 2.4: persona of the target group
PRODUCT ANALYSIS
02
01
Inserting the battery
Switch on the E-bike
03
Check battery status on the display
Figure 2.5: Storyboard
STORYBOARD The storyboard (figure 2.5) describes a possible use scenario of the E-bike by a commuter who is making a trip to his work. The order of the actions could be different, because there is no fixed order to use the product. The task flow in Appendix A1 provides more detailed insight in the different possibilities of the system.
11
12
PRODUCT ANALYSIS
05
04
Using accelerator to build up speed
Cycling with pedal support
06
Change support
PRODUCT ANALYSIS
07
Step off bike
13
14
PRODUCT ANALYSIS
DESCRIPTION COMPONENTS During the project the team focussed on the E-bike system: the parts of the E-bike that enables the user to operate and get feedback on the different functionalities of the E-bike (the buttons and display, figure 2.6).
Figure 2.6: description of the components
FIRST USER TEST
03
FIRST USER TEST In order to find out how people actually use the system, a user test is conducted. This chapter describes the research plan, results and conclusions of the first user test. The user test’s aim was to gain insights in the first time use of the E-bike system of inexperienced users, and thus test the intuitiviness of it. In order to do so, research questions were set up and a procedure for the test was created (Appendix A2). The user test was done with 6 commuters (target group) and two participants within the user group and took place in three different research environments. The data (Raw data in Appendix A3) are analysed by means of these five research questions and are summarized in this chapter (more elaborate in Appendix A4). The team ended up with five main problems and a conclusion of how the participants experience the usability of the E-bike system.
15
16
FIRST USER TEST
USER TEST PLAN RESEARCH GOAL
“The goal is to gain insights into how commuters use and experience the E-bike system during the first time use; to discover usability problems, its causes and if the E-bike system meets the expectations of the participant.�
USER TEST SETUP
During the usability test, the E-bike was tested in a simulated environment, where two actors played the oncoming traffic (figure 3.2). The participants are recorded by a GoPro during the whole test. The other procedure steps are performed at the end of the route.
Actor 2
RESEARCH QUESTIONS
Actor 1
1. How does the participant navigate to functionalities during the first time use? 2. Which usability problems is the participant facing during the first time use? 3. What are the causes of the found usability problems? 4. How does the participant experience the E-bike during first time use? 5. Does the E-bike fit the expectations of the participant during first time use?
Route (800 m)
Start/end
PARTICIPANTS Target group User group
Product settings
A
Figure 3.2: user test setup 6 Commuters
Figure 3.1: participants
1 elderly
1 other
FIRST USER TEST
Facilitator
Participant
02
Usability test: introduction
Recorded by GoPro datalogger
01
Introduction & initial questions
Questions: 1. Name & age? 2. Transport to work? 3. Expectations of an E-bike?
02
Usability test: first time use
Figure 3.3: user test procedure
17
18
FIRST USER TEST
03
Asking why participants acted the way they did.
Task scenarios
05
SUS questionnaire & last questions
SUS questionnaire
04 Tasks 1. Change support 2. Change mode settings 3. Turn on light 4. Turn off E-bike system 5. Explain E-bike system
Usability test: looking back footage
Questions: 1. Fulfill expectations? 2. What you liked most? 3. Anything you disliked? 4. Ways to improve this?
FIRST USER TEST
MAIN USABILITY PROBLEMS The user test is executed with 8 participants of which 6 participants of the target group are used to analyze and draw conclusions in this paragraph. Figure 3.4 shows the data processing of the user test. The findings are described in the five research questions to reach the research goal of this study (Appendix A4). This resulted in five main usability problems and these will be discussed in this chapter. For each problem is explained what the problem is, how it is found and what causes this problems.
Figure 3.4: data analysis process
19
20
FIRST USER TEST
ON
Usability Problem 1.1: It is not clear how to switch on the E-bike. Switching on the a crucial youthe needE-bike. to succeed in this in order to get access to the Usability Problem 1: ItE-bike is notisclear howfunctionality: to switch on
?
E-bike’s Also, it has big impact on the user’s firstto time experience Switching on theother E-bikefunctionalities. is a crucial functionality: youaneed to succeed in this in order get access to thebecause E-bike’s it is the first functionality they encounter. We found that participants feel like switching on the bike should itbe very other functionalities (see figure 4.09 - Task flow). Also, it has a big impact on the user’s first time experience because is the first functionality they encounter. found get thatitparticipants feelfact like that switching on the bike shouldwere be very obvious, and feel dumb if theyWe cannot to work. The multiple participants not able to obvious, and feel dumb if they get it toinwork. that multiple participants not able to it on, switch it on, even aftercannot succeeding this The oncefact before, indicates that this were functionality is switch not intuitive. even after succeeding in this once before, indicates that this functionality is not intuitive.
The participants were asked to bike a round with use
Usability Problem 2: It is not clear how the E-bike switches of the off. E-bike system. Four participants started riding
OFF
?
?
When the user does not switch off the system, it will remain on until next use or until the battery empty. on, Also,expecting it was it the bike without turning theis system found that many users end up in the service menu when switching offwould the E-bike: this is a pointon. of When big frustration. Also,it didn’t go automatically they noticed people rule out the MODE button after getting into the service menu, turn causing confusion on which buttonsearching to use. for an on automatically they started ‘power’ button. Three participants started searching for a button on the display and there were also three participants which thought that the screen was a touch screen. Eventually, all participants turned on the E-bike with the ‘MODE’ button. The ‘MODE’ button has Usability Problem 3: It is not clear how to turn on the lights . to indicate that this functionality is also for no use-cue switching on the which could be the reason Although the E-bike default light setting is on automatic, there are situations where thesystem, user would want control over why people had found difficulty turn on the bike. the light settings (e.g. when it is misty and the sensors do not detect darkness). It was thattothe ‘light-button’ placement does not correspond with people’s expectations. What causes this problem could be related to the difference in foreknowledge of the participants about E-bikes they have seen earlier on other technological products. This could have influenced why people search for a certain button or touched the display. Usability Problem 4: It is hard to operate one button atHowever, a time.it is expected that the end users also will in foreknowledge. Therefore, the two findings about Having to pay attention to which buttons you press distracts you fromdiffer traffic. Also, when accidently pressing this problem andofits causes canbecause be trusted buttons at the same time can cause confusion: people rule out certain functionalities the buttons the and used to improve in the final redesign. feedback they get does not match with the action they think they performed.
ON
is the first functionality they encounter. We found that participants feel like switching on the bike should be very obvious, and feel dumb if they cannot get it to work. The fact that multiple participants were not able to switch on, TEST FIRSTitUSER even after succeeding in this once before, indicates that this functionality is not intuitive.
?
Usability Problem 1.2: It is not clear how the E-bike switches off. WhenProblem the user does not switch the system, it willswitches remain on off. until next use or until the battery is empty. Usability 2: It is not clearoffhow the E-bike
OFF
Also, it was found that many users end up in the service menu when switching off the E-bike: this is a point
When the user does not switch off the system, it will remain on until next use or until the battery is empty. Also, it was bigmany frustration. Also, rule menu out the ‘MODE’ button getting service menu, causing foundofthat users end up inpeople the service when switching off after the E-bike: thisinto is a the point of big frustration. Also, confusion on which button to use. people rule out the MODE button after getting into the service menu, causing confusion on which button to use.
?
The problem with turning off the E-bike was also
Usability Problem 3: It is not clear how to turn on the lights due. to the assumption that it went automatically. Although the E-bike default light setting is on automatic, there are situations where the user would want control over the Three of them remembered that they turned the light settings (e.g. when it is misty and the sensors do not detect darkness). waswith found the ‘light-button’ systemIton thethat ‘MODE’ button and tried to placement does not correspond with people’s expectations. use that button again to turn it off. One of the recurring issues is that they accidentally get into the service menu. This makes them confused and rule out the ‘MODE’ button for turning off the E-bike. One participant got into the service menu two times in a row, while he knew how to turn it Usability Problem 4: It is hard to operate one button at a time. off. The ‘MODE’ buttons has too many functions, Having to pay attention to which buttons you press distracts you from traffic. Also, when which could be accidently the reasonpressing that thetwo participants buttons at the same time can cause confusion: people rule out certain functionalities of the buttons because didn’t expect it should also be usedthe for turning it feedback they get does not match with the action they think they performed. off. This multifunctionality of the ‘MODE’ button makes it also difficult to have a proper use-cue that fits all this functionalities.
?
Although the participants encountered problems
Usability Problem 5: It’s not clear what the MODES mean.with this functionality, two of the participants
?
It was found that the different MODE data was hardly used during the firstalready time use. The MODE-button big and by figured out switchingisofftoo accidentally obvious compared to the importance of its functionality. Also, the MODE-button tooThis many functionalities all onhow anotherhas task. could have influenced one button, which can cause confusion for the user. If the role of this button becomes clearer and causes less they operated the functionality, but confuthe results sion, this could also result in less usability problems for other functionalities. v show no difference between the results of these participants and the other participants.
21
22
OFF
FIRST USER TEST
found that many users end up in the service menu when switching off the E-bike: this is a point of big frustration. Also, people rule out the MODE button after getting into the service menu, causing confusion on which button to use.
?
Usability Problem 1.3: It is not clear how to turn on the lights . Although the E-bike default light setting is on automatic, there are situations where the user would want
Usability Problem 3: It is not clear how to turn on the lights . control over the light settings (e.g. when it is misty and the sensors do not detect darkness). It was found Although the ‘light-button’ E-bike default light setting isdoes on automatic, there are situations where the user would want control over that the placement not correspond with the participant’s expectations. the light settings (e.g. when it is misty and the sensors do not detect darkness). It was found that the ‘light-button’ placement does not correspond with people’s expectations.
?
None of the participants were able to finish this
Usability Problem 4: It is hard to operate one button at atask time. within 10 steps or without hints. Instead
?
Having to pay attention to which buttons you press distracts you from traffic. Also, for when of looking theaccidently light in thepressing system,two four buttons at the same time can cause confusion: people rule out certain functionalities of the buttons because the participants started looking on the physical feedback they get does not match with the action they think they performed. light itself expecting to find a button. When they couldn’t find it the right button, two hoped and assumed it turned on automatically. Only half of the participants recognised the light icon on the display. The icon on the display is not Usability Problem 5: It’s not clear what the MODES mean. outstanding enough to provide the user feedback It was found that the different MODE data was hardly used during the first MODE-buttonon. is too and for thattime theuse. lightThe is automatically Thebig reason obvious compared to the importance of its functionality. Also, the MODE-button has too many functionalities all on usethis problem could also be that there is no one button, which can cause confusion for the user. If the role of this button becomes andthat causes less confucue, which tellsclearer the user the ‘UP’ button can sion, this could also result in less usability problems for other functionalities. v be used for turning on the light.
The default setting of the light is on automatic. Therefore, it could be considered if users would turn on the light manually during daytime. There are cases where the user would like to turn on the light (e.g. mist, fog and rain). Besides, it was found that this task caused a lot of frustration and influences the first time use experience as can be seen in usability experience questionnaire (p. 25).
the light settings (e.g. when it is misty and the sensors do not detect darkness). It was found that the ‘light-button’ placement does not correspond with people’s expectations. FIRST USER TEST
?
Usability Problem 1.4: It is hard to operate one button at a time. Having to pay attention to which buttons you press distracts you from traffic. Also, when accidentally Usability Problem 4: It is hard to operate one button at a time.
pressing two buttons at the same time can cause confusion: people rule out certain functionalities of the
Having to pay attention to which buttons you press distracts you from traffic. Also, when accidently pressing two buttons because the feedback they get does not match with the action they think they performed. buttons at the same time can cause confusion: people rule out certain functionalities of the buttons because the feedback they get does not match with the action they think they performed.
Usability Problem 5: It’s not clear what the MODES mean. Two participants accidentally press two buttons
?
It was found that the different MODE data was hardly used during theatfirst use. TheThe MODE-button tooclose big and thetime same time. buttons areistoo to obvious compared to the importance of its functionality. Also, the MODE-button has too many functionalities all on each other and it is difficult to distinguish the one button, which can cause confusion for the user. If the role of this three buttonbuttons becomes and causes less confuonclearer the handlebar without looking. sion, this could also result in less usability problems for other functionalities. v One participant explicitly said that she didn’t
like the buttons because of the material and had trouble operating them. Although only two participants encountered problems with pressing one button at once, it still has a big influence on the intuitiveness of the system. The feedback will not match with the action and it also could distract from the traffic to pay attention to the buttons. Therefore, it is important to investigate how this problem can be improved.
23
24
Having to pay attention to which buttons you press distracts you from traffic. Also, when accidently pressing two buttons at the same time can cause confusion: people rule out certain functionalities of the buttons because the feedback they get does not match with the action they think they performed.
FIRST USER TEST
Usability Problem 1.5: It’s not clear what the MODES mean. Usability Problem 5: It’s not clear what the MODES mean.
It was found that the different MODE data was hardly used during the first time use. The ‘MODE’ button is
?
It was thatobvious the different MODE data wasimportance hardly used during the first time use. The MODE-button is toohas bigtoo andmany toofound big and compared to the of its functionality. Also, the ‘MODE’ button obvious compared to the importance of its functionality. Also, the MODE-button has too many functionalities all on functionalities all on one button, which can cause confusion for the user. If the role of this button becomes one button, which can cause confusion for the user. If the role of this button becomes clearer and causes less confuandalso causes confusion, couldfor also result in less usability problems for other functionalities. sion,clearer this could resultless in less usability this problems other functionalities. v
During the test, the participants had to look up the maximum speed they drove during biking around. Four participants used the ‘MODE’ button, but to scroll through the different modes the participants encountered more problems. Two participants searched for buttons on the display and also two clicked the ‘UP’ and ‘DOWN’ button. Besides, two of the participants didn’t recogniseMAX as maximum speed at first sight. When they scrolled through all modes, they concluded that MAX would stand for maximum speed. When asked to explain all the other modes, two participants had difficulties to understand the different abbreviations. The information of the modes on the display provide too little explanation for the user to understand what all the modes mean. Another reason for this problem could be the related to the knowledge participants have from other products, which use ‘mode’ differently. However, this could also be the case with the end users. Besides, multiple participants mentioned that they will not use this functionality. Although this makes this problem less severe, the company should take into account that this user research is only done with commuters and the sample of the other groups was too small to draw a conclusion.
FIRST USER TEST
USABILITY EXPERIENCE E-BIKE SYSTEM At the end of the user test, a method of John Brooke (1986) is used to discover how the participants experience the usability of the E-bike system. This method consists of a System Usability Scale questionnaire with 10 statements to rate on a scale from 1-5, from completely disagree to completely agree. Only 6 statements are used in the results to measure the usability experience of the E-bike. The four that are left out could only be tested over a longer period of time. Since the test is about first experience and takes 45 minutes, the results of these statements don’t give insights on first time use.
The results in figure 3.5 show an overview of the results (appendix A4.5). Here, higher bars mean a better usability experience. It shows that the usability of the system is experienced as complex (M = 2.5) and cumbersome (M = 1.3) and that the interface of the E-bike is not perceived as intuitive (M = 2.0) and consistent (M = 1.8). This could be caused by the difficulties the participants encountered with the light, and the turning on and off of the E-bike system. Most buttons has multiple functionalities, which also caused these problems. This made the participants feel that these functionalities weren’t well integrated
Figure 3.5: bar charts of the usability experience questionnaire
(M = 2.5). Encountering these problems might also be the reason why it was stated that the system was not easy to use (M = 2.4). The results of this experience usability questionnaire could be influenced by subjective aspects as user’s experience and background. However, the results show such clear patterns and the found usability problems are connected to this findings that these results are considered valid.
25
26
FIRST USER TEST
DISCUSSION The findings in this chapter are already discussed related to the limitations of the test. This gave insight in the reliability and importance of the results. Based on these findings and the research goal, the difference in severity between the usability problems can be determined to define the focus for the redesign of the E-bike. The first task of the user test resulted into the most severe problem. Switching on the E-bike should be very obvious and is necessary to make use of the other functionalities, but none of the participants understand this in one try. This will influence the user’s first experience and has a big impact on the intuitiveness due to the frustration the participants had. This frustration could also be identified when the participants switched
off the E-bike. Although this functionality will not influence other functions, it will leave a negative experience and therefore a severe usability problem to solve. The cause of both problems is the multifunctionality of the ‘MODE’ button. This also had an effect on the problem participants had with understanding the mode function. Although most participants of the target group mentioned that this functionality was not important for them, this usability problem is still important to improve, but related to the other problems it is the least severe. The problem with turning on the light could also be seen as less severe due to the limitations of the test, but the moment of interaction with this functionality has a big impact on the intuitiveness, which makes this problem more
severe than understanding the mode functions. The last problem ‘operating multiple buttons at a time’ could have influenced the earlier mentioned problems, but this is taken into account when analysing the results. Pressing multiple buttons instead of one could cause confusion in the feedback the user received from the product. Since only two participants encountered this problem, the problem will be seen as less severe than the three first problems. The order of severity between these usability problems is already used to structure the results and it is also used to create a design brief for the redesign (appendix A5).
FIRST USER TEST
Task completion
P1
Usability problems
P2
P4
P8
P1
P3
P5
task 0: Switch on system
P1
P5
P2
P3
P4
P8
task 1: Change support
P1
P3
P8
P2
P4
P5
task 2: Find max speed
1.5
P1
P2
P3
P4
P5
P8
task 3: Turn on light
1.3
P1
P3
P9
P5
P2
P4
task 4: Switch off system
1.2
P3
P4
P5
P6
P1
P3
Pressing multiple buttons
1.4
Completed tasks in:
UPr
1.1
UPr
UPr
UPr
UPr
Severity of problems
First try
Serious problem
Second try
Critical problem
Third or more Figure 3.6: Overview of task completion related to the usability problems
27
28
REDESIGN PROPOSAL
04 REDESIGN PROPOSAL This chapter gives an overview of the redesign of the Sparta F8E that was developed according to the results of the user test. A design brief was created to formulate the design goal and specifications for the redesign based on the user test results (appendix A5). The different parts are explained and a relation with the user test results is included.
REDESIGN PROPOSAL RESEARCH GOAL
“The goal is to gain insights into how commuters use and experience the E-bike system during the first time use; to discover usability problems, its causes and if the E-bike system meets the expectations of the participant.”
User Interface
Buttons
Display casing
GENERAL OVERVIEW
Light switch
Figure 4.1: Overview redesigned parts
Figure 4.1 shows an overview of the different parts of the product that were designed, and the placements of those parts. The placement of those parts are exactly the same as in the current product, because the user test didn’t show evidence to change this. Therefore, the redesign of the display and button casing won’t bring radical changes. This will be explained in more detail in the following sections.
29
30
REDESIGN PROPOSAL
BUTTONS DESIGN OVERVIEW + / - button These buttons change the current support level.
TRIP INFO button This button displays the different recorded data by switching from homescreen through this information.
Figure 4.2: Buttons redesign
Figure 4.3: Buttons in Sparta F8E current design
REDESIGN PROPOSAL
BUTTONS DESIGN DECISIONS ‘+’ and ‘-’ button The user test showed that the participants used the arrow buttons (figure 4.4) to operate the recorded. Therefore, the symbols are changed in ‘+’ and ‘-’ to relate it better to the support. Also, matching the button composition with the division of the user interface makes it easier to make this connection. The spacing between the two buttons is increased to avoid people accidentally pressing the wrong button. Besides, the position of the support is also determined to make them easy to reach during biking, because the support buttons are used most frequently during biking.
+ _
‘TRIP INFO’ button The participants indicated that the meaning of the different MODES were not clear during the user test. This was partly caused by the name of the button (‘MODE’), which is now called ‘TRIP INFO’. This button has been separated from the support buttons to avoid confusion that they need to be operated together. To increase this sense of separation, the button has a different shape. It also reduces the chance of accidently pressing two buttons at the same time.
Trip Info
Figure 4.4: Placement of buttons and elements on UI
31
32
REDESIGN PROPOSAL
DISPLAY DESIGN OVERVIEW
DISPLAY DESIGN DECISIONS
Casing The casing of the display is unchanged, except for the ‘power’ button.
Casing The usability problems that were found during the testing were not necessarily related to the display. We believe that the UPr’s can be solved with other, smaller changes in the current design and thus the display remains unchanged (figure 4.5) ‘power’ button A separate button to turn the E-bike system is added to the casing, because of the usability problems with turning on and off the system. The button is placed in the middle of the display as a result of a paper prototype (appendix A6). During the user testing it was found that many participants forgot to turn off the E-bike. To avoid draining the battery when this happens, the E-bike system will turn on automatically after 5 minutes of inactivity. Inactivity is measured when the pedals are not rotating or no buttons are used.
Figure 4.5: Display redesign
‘power’ button This button will either turn on or off the system. Also, after 5 minutes of inactivity the system will automatically turn off.
REDESIGN PROPOSAL
LIGHT DESIGN OVERVIEW Light switch The lights are ‘removed from the E-bike system’. This functionality uses a slide that is on top of the front light.
LIGHT DESIGN DECISIONS The user test showed the problems participants encountered with switching on the light. The participants started searching for a physical button on the light itself. Therefore, a physical switch is placed on the front light (figure 4.6).
Figure 4.6: Light slider redesign
Moreover, the light icon from the UI is removed, because the light cannot be operated from the handlebar anymore. Besides, the switch on the light explains if the light is on, off or auto.
33
34
REDESIGN PROPOSAL
INTERFACE DESIGN OVERVIEW Homescreen The same type and sizes of the display are used as in the current product. While the arrangement and sizes of the different elements on the screen have changed (figure 4.7).
Time
Battery Gives the user a percentage of the current battery level. Starts to slowly blink when the battery is almost empty.
Figure 4.7: Interface redesign Support The support levels are indicated with empty boxes, which indicate the current support level, while being able to see the maximum level.
Speed Shows the user what their current speed is.
Figure 4.8: UI in Sparta F8E current design
REDESIGN PROPOSAL
Trip info screen The trip information consists of five different data recorded by the E-bike. Four of those are about the data that is gathered about the trip itself (trip data), and one about the total amount of distance that is traveled with the E-bike (total distance) (figure 4.9).
Recorded data - Trip time - Trip distance - Average speed - Max speed - Total distance
Figure 4.9: Interface redesigns
Indication dots Below the recorded data, indication dots show the user where he is in this menu and how many menu items there are.
35
36
REDESIGN PROPOSAL
INTERFACE DESIGN DECISIONS Support The support level has a prominent place on the display because it is a frequently used feature. The maximum possible support level can be seen by the user, which is indicated by the empty outlined boxes. This feature is added, since the participants remained clicking up when they already reached level 5. Speed On the home-screen, the speed has a central position and is displayed the biggest. This is because it was found that the participants found this feature the most important and used it most often. Time The time has been portrayed in the top center of the screen, because time is important for this target group, mainly when they should be on time for their work.
Battery The battery icon on the current product is very prominent although is it not used very often as a functionality while biking (figure 4.8). It is a feature that is only checked at the beginning and end of a trip. Therefore, the battery will be smaller and placed on the top right of the screen. The level of the battery is indicated with a percentage, and when it is below 20%, it will slowly blink, telling the user it needs to be recharged in the near future. Although the team is aware that this change is not possible with the current techniques of battery, the team believes that this change would provide a better user experience.
Trip info The user test shows that participants had problems with understanding the different modes. It was found that commuters don’t see these as an important functionality of the E-bike, since they drive the same route every day, which make them less interested in this data. Therefore, this information is placed on a separate screen, which can be operated by pressing the ‘TRIP INFO’ button. In this way the home screen is not cluttered with information that the user doesn’t use. Navigating through the recorded data should be done by pressing the ‘TRIP INFO’ button multiple times. When the user doesn’t press that button in 10 seconds, the screen will automatically jump back to the home screen, because the user can then focus back on the road when he has seen the information he wants and doesn’t have to look to get the current speed back on the screen again. Moreover, it was found that the MODE abbreviations were not clear to all participants. Now that the recorded data is on its separate screen, this data can be displayed bigger and with longer descriptions (Trip time, Trip distance, Trip avg speed, Trip max speed, Total distance) and corresponding units are added (e.g. km/h).
SECOND USER TEST
SECOND USER TEST
To see if the redesign solved the main usability problems, a second user test is executed and the results of both tests are compared to each other. The procedure of the second test is similar with the first one and a comparison between the results of both tests is made(appendix A7.1). The tasks results of the redesigned system are used to answer the questions to what extent the previously found usability problems are solved through the redesign. Furthermore, the results on the usability experience questionnaire are compared to investigate if this is improved (the raw data and results of this user test can be found in appendices A8 and A9). Moreover, new usability problems have been found with the prototype. All the results and findings are verified with the limitations of the research, and an overall conclusion is drawn to see whether the redesign solved the main problems.
37
38
SECOND USER TEST
USER TEST PLAN
PARTICIPANTS Target group
RESEARCH GOAL
“The goal is to gain insights into how commuters use and experiences the redesign of the E-bike system during the first time use, to discover whether the previously found usability problems are solved and if the redesign causes no new severe usability problems.” RESEARCH QUESTIONS 1. How does the participant navigate to the functionalities during the first time use? 2. Which usability problems is the participant facing during the first time use? 3. What are the causes of the found usability problems? 4. How does the participant experience the redesigned E-bike system during first time use? 5. To what extend did the redesign solve the current UPr’s?
User group
7 Commuters
2 elderly
Figure 5.1: participants
USER TEST SETUP The second user test was kept as close to the original procedure as possible. This way, the results are more reliable to be compared. In the second user test, the participant is not able to drive on the bike. Because of this, the exact location is of less importance and we were able to test at different locations without it influencing the results. The task scenarios that were presented to the participants remained the same, just as the usability experience questionnaire. Because it was decided not to use the expectations that were asked for during the first test, these were left out of this test as well. On the next page, a visual overview of the different procedure steps are given.
01
Introduction & initial questions
Facilitator
Participant
Figure 5.2: Redesign user test procedure
SECOND USER TEST
Tasks 1. Change support 2. Change mode settings 3. Turn on light 4. Turn off E-bike system 5. Explain E-bike system
03
SUS questionnaire
SUS questionnaire
02
Task scenarios
04
Reflection
Questions: 1. Fulfill expectations? 2. What you liked most? 3. Anything you disliked? 4. Ways to improve this?
39
40
EXECUTIVE SUMMARY
Figure 5.3: Prototype overview
SECOND USER TEST
PROTOTYPE Figure 5.3 shows an overview of all of the different parts of the prototype that was used for the second user test. The aesthetics of the prototype do not correspond to what the product would look like in real life, however this doesn’t influence the usability. The screen has the same size of what the actual screen size would be, and a ‘power’ button is placed on casing of the display. This is no functional button and its effect will be simulated by the ‘Wizard of Oz’ that makes the information on the screen appear(ON) and disappear(OFF). The buttons on the handlebar are functional and connected to an arduino. These buttons react on the actions of the participants and the effects on the display will be shown. A complete overview of the techniques used in this prototype can be found in appendix A7.2.
Limitations The prototype has some limitations that influence the user test. These are taken into account when analyzing the results. A big limitation of the prototype is that the participants cannot ride the bike, since it is completely wired with cables. Therefore they can’t experience the support and the reactions of it to them pressing the ‘+’ or ‘-’ button. Furthermore, the speed and distance cannot be measured and fake data was used. This makes it harder for the participants to imagine that they actually drove the bike and relate that to the supposed distance they biked. The light on the bike has two settings: On and Off. The real product would have On/Off/ Automatic. Moreover, the light does not turn on while standing still(but only while biking), but this would be the case with the real product when it is turned on.
41
42
IMG HERE
COMPARATIVE STUDY The results from this test are analysed and compared with results from the first test. The task completions show that the participants could more easily and faster finish the tasks compared to the current design. Each task is analysed and visualised separately. Then can be seen how much they have been improved. Even though UPr 1.4 ‘press two buttons at once’ was not part of a task scenario, it is interesting to see how many participants encountered problems with the buttons of the prototype and compare this to how many had troubles with the current product.
SECOND USER TEST Task 0: Switch on system With the current product, all participants had trouble switching on the system: none of them managed to do so in one step. In the redesign the ‘power’ button is introduced. Now, only one participant was not able to switch the system on in one step.
Figure 5.4: Comparison switch on E-bike
Task 1: Change support In the first test, four out of six participants were able to switch the support correctly on their first try. Using the redesign, all participants were able to change the support right away. However, P4 was confused by the meaning of the ‘+’ and ‘-’. She related it to her regular gear, assuming that ‘-’ would result in less resistance. Because of this, she pressed the ‘-’ button when asked what she would do biking up a bridge.
Figure 5.5: Comparison change support
43
44
SECOND USER TEST
Task 2: Find max. speed During the first test, three out of six participants had troubles finding the maximum speed. Using the redesign, all participants understood the ‘TRIP INFO’ button had to be pressed first. Then, one participant tried to scroll through the options using the ‘+’ and ‘-’ buttons. The remaining participants did use the ‘TRIP INFO’ button correctly to do this.
Figure 5.6: Comparison max. speed
Task 3: Turn on light In the first test, none of the participants were able to turn on the light within 3 steps. Using the redesign, six out of seven found the light switch right away.
Figure 5.7: Comparison switch on lights
SECOND USER TEST Task 4: Switch off system Using the current design, only two participants were able to switch off the system in one step. Now, five out of seven were able to do so. The remaining participants pressed the ‘TRIP INFO’ button or forgot to switch it off. However, the system will turn off automatically after 5 minutes, so forgetting to switch it off is not a big problem.
Figure 5.8: Comparison switch off E-bike Pressing physical buttons During the first test it was observed that two participants accidentally pressed two buttons simultaneously. Using the redesign, this happened to none of the participants.
Figure 5.9: Overview number of participants accidentally pressing two buttons
45
46
SECOND USER TEST
Figure 5.10: Comparison task completion first and second user test Discussion Comparing these task scenarios gave insight in the improvement of the redesign relative to the current product. In the first user test, the participants encountered problems with different tasks (figure 5.10). The findings show that the tasks ‘Switch on system’, ‘Switch off system’ and ‘Turn on the lights’ have been improved considerably. Although the task ‘Finding the maximum speed’ went better during the second test, there was still confusion on how to navigate through the different recorded data. According
to this, it is recommended to investigate this problem further. Another interesting insight is the task ‘changing the support’ which didn’t cause any problems during the first test and caused some confusion in second user test. The participants didn’t directly understand which of the two buttons resulted in less or more resistance while paddling. However, due to the limitations of the prototype, the participants could not experience and feel the feedback of the support in the
paddles when pressing the ‘+’ and ‘-’ buttons. It is expected that this wouldn’t occur when participants feel the feedback while biking as can be seen in the first user test. Therefore, no further research is recommended for this problem. Although the limitations could have influenced the results, the findings can be trusted and a conclusion can be drawn. The previously found usability problems are solved, however new usability are problems discovered (p. 49).
SECOND USER TEST
USABILITY EXPERIENCE E-BIKE SYSTEM As in the previous user test, the participants were asked to indicate their experience of the usability of the E-bike system using the System Usability Scale questionnaire. To be able to compare the results of both tests and easily see if the redesign improved the usability experience, all the statements are reformulated to a positive sentence. Figure 5.11 illustrates the experiences of both the current and redesigned system. It clearly shows that the usability of the redesign is experienced more positive compared to the current design. Some statements of the usability experience questionnaire are left out of the comparison because they could not be tested and are therefore irrelevant. Appendix A9.4 shows the more elaborate results of the usability experience
questionnaires. The results of the statements “I do not find the E-bike system unnecessarily complex” and “I found The E-bike system very wieldy to use” portray positive responses by users to the redesign proposal. The statements “I thought the E-bike system is created with a good consistency” and “I do not need to study the manual before I could get going with this E-bike system” were used to investigate the interface. The results reveal that the new interface is experienced as user-friendly and consistent. In the statement “I thought the E-bike system was easy to use”, six out of seven participants assigned the maximum score.
The statement that received the lowest score was about how the functions of the system were well integrated in the system. The team assumes that this is rated lower because the light. Three participants thought it would be nice to operate the light with the system and would like to see the light integrated. This suggestion will be taken into account when making recommendations to improve the redesign. Although this statement has a lower score, this score increased with 1,9 points. Besides, the average of the first user test (M = 2,01) compared with the second user test (M = 4,6) is 2,6 points higher and all statements score higher than in the previous user test. This makes the subjectivity of this questionnaire more reliable due to the clear pattern. Therefore, the usability of the redesign is experienced more positive than the current design on the tested aspects.
47
48
SECOND USER TEST
(I completely agree)
SECOND USER TEST
5 4
Average ratings from participants
3 2
(I completely disagree)
1
“I don’t find the E-bike system unnecessarily complex.”
“I thought the E-bike system was easy to use.”
“I found the various funtions in this E-bike system were well intergrated.”
“I thought the “I found the E-bike E-bike system is system very wieldy created with a good to use.” consistency.”
“I don’t need to study the manual before I could get going with this E-bike system.”
Corresponding SUS question = 1st user test (N=7) = 2nd user test (N=8)
Figure 5.11: Comparison average ratings usability experience questionnaire between first and second user test
49
NEW USABILITY PROBLEMS The comparative study gave insight in the differences between the user tests. This paragraph describes the new found usability problems during the second user test. For each is explained what the problem is and what it causes. The limitations of the test are taken into account, to decide which of these new problems are severe and realistic or not and could be improved further.
50
SECOND USER TEST
+ _
Usability Problem 2.1: It is not clear how to scroll through the trip info. Trip Info
?
It was unclear how to navigate to the different trip info options. After pressing the ‘TRIP INFO’ button, these people tried to navigate through the different options using the ‘+’ and ‘-’ button. Also, some people were confused by the fact that the screen would return to the current speed after 10 seconds.
Two of the seven participants didn’t understand that the ‘TRIP INFO’ button and the ‘+’ and ‘-’ buttons are separate functions. Instead of using the ‘TRIP INFO’ button to cycle through the options, they tried to use the ‘+’ and ‘-’ button. A possible reason for this was that the use-cue provided by the shape of the ‘TRIP INFO’ button was not clear to them. Another aspect that could have caused this problem is that the interface of the recorded data switched back too quickly to the home screen, which makes the participants confused about the operation of the trip info function. This is caused by the fact that participants were asked to explain the trip information out loud for the user test, which is not a situation that is likely to happen in real life use. Therefore, the team suggests to keep the automatic switch back to the homescreen, but to improve the usecue of the ‘TRIP INFO’ button to solve this problem.
SECOND USER TEST
Usability Problem 2.2: It is not clear when a trip starts and ends. In order to use and understand the trip info data, users needs to know when the values of the different trip options reset (e.g. the trip time). In the redesign this was not understood by the participants.
Two of the participants did not understand when the trip would start and end. Because of this, they could not make sense of the presented trip data. This could be caused by the fact that too little information is provided on the UI about the recorded data, which isn’t changed related to the current product. Therefore, the limitations of the prototype could have influenced this problem. The participants didn’t see real time changes, which also argues why the participants during the first user test didn’t encounter any problems with this. Although this limitation could have influenced the results and the fact that only two participants encountered this problem, it could be concluded that it is not intuitive. Therefore, it is suggested to improve the recorded data. However, the target group indicated that this functionality is not important for them, which makes this problem a minor problem.
51
52
SECOND USER TEST
Usability Problem 2.3: It is not clear that the light switch will turn both the front and back light. When asked to turn on the light, users found the switch on the front light instantly and switched it to on. However, some users expected a similar switch on the backlight to turn that on.
? All participants found the switch on the light instantly. Two participants tried to find a similar switch on the back light. This is caused by the fact that the front light doesn’t indicate a connection with the back light. One participant indicated that he is used to have a separate ‘back light’ button due to the button on his own bike. Due to the limitations of the prototype, only the placement of the button could be tested. Besides, these limitations also caused that participants didn’t get feedback if the light will be on, when turning it on. This could have influenced that the participants cannot indicate if the backlight also will emit light. Both participants mentioned, when they couldn’t find a button for the backlight, they will ride the bike and check if it will emit light. Therefore, this problem will be discarded.
SECOND USER TEST
?
Usability Problem 2.4: It is not clear that there are multiple trip info options. One participants did not understand the indication dots underneath the trip info. They assumed that there was only one screen of trip information and tried to find the max speed on the trip time screen.
To one of the participants it was not clear that there were multiple trip info options. She assumed that the max speed would be on the first recorded data screen, while this was the trip time. She did not realise that there was an option to continue to other data as well. This could be caused by the fact that she did not understand the indication dots. None of the other participants, including the ones from out of the target group, encountered this problem when they were asked to explain it. The E-bike screens use icons and symbols that are part of today’s technology language. Therefore, it is expected that this participant is not a frequent user of such devices, which will cause that this type of users probably not understand the meaning of these symbols in the interface. The team doesn’t expect many similar users in this target group with the upcoming technology. This makes this problem not a severe problem anymore.
53
54
SECOND USER TEST
DISCUSSION All findings in this chapter are related to the limitations to draw reliable conclusions on which recommendations could be based. Next to comparing the findings and analyzing the limitations, the redesign could be checked against the design goal and design specifications to identify eventual new recommendations. DESIGN GOAL “Commuters should be able to use and understand the E-bike system without receiving instruction during first time use.” The comparison of the tasks showed that the usability problems of the current product didn’t appear in the second user test (figure 5.12). The only task that caused similar problems was task 2 ‘find maximum speed’. These problems didn’t appear in the first user test, although the redesign wasn’t that different. A reason for this difference could be the limitations of the prototype and the more detailed questions that were asked about this functionality in the second user test. Although these limitations, the team expects that the two problems with ‘scrolling through trip info’ and ‘understanding the start and end of trip data’ could appear when using this redesign. Therefore, these problems are recommend to solve. The other two problems described in this chapter could be discarded due to the limitations these problems are not seen as a problem anymore.
Although the redesign ensures recommendations, the design specifications have been realized and the design goal has been achieved. However, the company would be suggested to take into account that this E-bike is tested with one target group ‘commuters’, when using the results of this user test. Both the current design and the redesign are also tested with two
participants out of the target group, which are used to draw also conclusions for the other user groups (appendix A4.7 & A9.5). Although these participants didn’t encounter other problems than the target group, nothing significant can be concluded due to the fact that these findings are only based on two participants out of two different user groups.
Figure 5.12: Comparison usability problems first and second user test
RECOMMENDATIONS
06
RECOMMENDATIONS The second user test showed a positive difference due to the redesign. However, this test showed also two new problems and the participants mentioned some suggestions at the end of this test. These problems require further research, because it is expected that the user could encounter this problems as well. Therefore, recommendations for this problems and suggestions are made related to the results of both test. These recommendations are not proved during this test and need further research.
55
56
RECOMMENDATIONS
Scrolling through recorded data (UPr 2.1) The team saw that some of the participants tried to use the ‘+’ and ‘-‘ buttons to scroll through the recorder data. During the test the participants looked at the ‘TRIP INFO’ button when using it to enter the recorder data interface. Therefore, the team expects that an arrow on this button would make its functionality more clear. This could be tested with a paper prototype to determine if this is a good solution.
Light integrated (usability experience test) Some participants expected a separate button for the back light as well, when they found the button on the front light. This wouldn’t have been a problem with a fully functioning prototype, since they could see that both lights would turn on. Furthermore, a couple participants suggested that they would like to see the light as part of the system with a button they could safely operate while biking. Therefore, a ‘light’ button on the display is recommended. This will also avoid confusion whether this button belongs to only the front light or should be used for both lights. To integrate the light better in the system, an icon could be placed on the screen interface.
AUTO
Material of the buttons None of the participant accidentally pressed two buttons at once. The prototype consisted of 3D printed material and not made out of the realistic final material. The team suggest that the company researches which material is best for using on the buttons on the handlebar to avoid this problem.
5
12:35
97%
RECOMMENDATIONS
Returning to main menu Some participants experienced the automatic return from the recorded data to the home screen too fast. However, during the test the participants were asked to think aloud, and this takes more time compared to only using it. Therefore, the company should do some research on what would be the best timing to avoid confusion for the user. Start & end of a trip (UPr 2.2) During the test, some participants said that they didn’t understand what the recorder data exactly records. They wondered when it starts measuring their distance/time etc, if this is daily, weekly, or monthly based. A suggestion is to reset the trip data each night so that the user indicates when it is starts. In addition, an explanation in the manual with further information could explain it how the user can reset the data manually.
Gear icon To avoid confusion between the functionalities of the support and the gear, it is recommended that the gears should use up and down arrows, since the support is using ‘+’ and ‘-‘ icons. Considering that + means more support, and therefore less resistance and easier paddling, and a + on the gear would mean a higher gear, more resistance and more difficult paddling. This would be conflicting information for the user, so it is better to avoid this by using different icons for support and gear.
57
58
EXECUTIVE SUMMARY
Affairs, A. S. (2013). System Usability Scale (SUS). Retrieved April 10, 2017, from https://www. usability.gov/how-to-and-tools/methods/systemusability-scale.html Hendriksen, I., Engbers, L., Schrijver, J., Gijlswijk, van, R., Weltevreden, J., Wilting, J. (2008). Elektrisch fietsen. Marktonderzoek en verkenning toekomstmogelijkheden. Retrieved April 10, 2017, from http://www.fietsberaad.nl/ library/repository/bestanden/Rapport%20KvLB&G-2008-067%20compleet.pdf Kjeldskov, J., Skov, M. B., & Stage, J. (2004). Instant data analysis. Proceedings of the third Nordic conference on Human-computer interaction NordiCHI ‘04. doi:10.1145/1028014.1028050 Top 10 Hell. (n.d.) Top 10 Countries with Most Bicycles per Capita. Retrieved April 10, 2017, from http://top10hell.com/top-10-countries-withmost-bicycles-per-capita/
07
REFERENCE LIST
Zee, van der, R. (2016). Can e-bikes revolutionise long-distance commuting? Retrieved April 10, 2017, from https://www.theguardian.com/ cities/2016/sep/14/e-bikes-long-distancecommuting-speed-pedelec-electric-cycles