Usability Evaluation Report Conor Bergin, Ronan Healy October 10th, 2017
Usability Evaluation Report Conor Bergin, Ronan Healy October 10th, 2017
contents Introduction Overview Executive Summary
02 03
Methodology Task analysis Defining Scenarios Location Observation Pre-task Questions Quantitative Measurements Debrief Questions Unpacking Information User Journey Mapping Heuristic Evaluation
06 11 11 12 13 14 14 15 15 15
Results Observations Quantitative Data Journey Maps Pain Points Heuristics Recommendations Conclusions
18 18 19 23 27 29 30
Appendices A: Hierarchical Textual Description 1 B: Hierarchical Textual Description 2 C: Observations at Smithfield Luas Station D: User Testing Questionnaire E: Individual User Journey Mapping Scenario A F: Individual User Journey Mapping Scenario B
32 34 36 38 40 42
introduction Overview Dublin’s light rail transit system (Luas) is operated by Transdev and carries on average 90,000 passengers per day.1 The Luas network comprises two major lines; the Red line, which runs from Tallaght to The Point and from Saggart to Connolly, and the Green line, which connects Brides Glen to Stephen’s Green through Sandyford. Ticket vending machines (TVMs) located at each stop provide a point of sale for standard and long term Luas tickets as well as facilitating the top-up of Leap cards through NFC readers. Introduced in December 2011, Leap cards are prepaid transport cards which can be used on all the primary transport systems in Dublin (Dublin Bus, DART, Bus Éireann, Iarnród Éireann, Swords Express and Luas). From the 3rd - 10th October 2017 a testing team (NCAD MA Interaction Design students Conor Bergin and Ronan Healy) undertook an investigation into the viability of the Luas TVMs through a period of usability testing and analysis.
1 https://www.luas.ie/faqs.html#faq02
Luas Zonal Map showing Red and Green lines. https://www.luas.ie
2
Executive Summary Usability testing carried out by the testing team began by proposing two typical tasks to be examined. These tasks were subsequently broken down step by step through task analysis. This process of task analysis revealed two scenarios worthy of further investigation due to being either excessively convoluted or confusing in nature. The first stage of data collection involved a 15 minute period of observation conducted at Smithfield Luas station during rush hour. Insights gathered at this stage assisted the formulation of a questionnaire for user testing. During user testing, and in concurrence with the testing scenarios, the testing team used this questionnaire in order to establish the patterns and pre-existing knowledge of each participant. After the scenarios were tested users were debriefed in a semi-structured interview format to gather further information on their experience with the TVM. After collecting the necessary data from user testing the testing team evaluated the TVM with reference to existing usability principles or heuristics. Although the usability testing did not uncover any critical issues it has identified some major problems including: •
• • • • •
The zonal ticket map for selecting travel destination is confusing and requires geographical knowledge of Dublin to use efficiently. The process for adding multiple traveller tickets to a purchase often causes confusion among users. The TVMs time out rapidly and without any warning whilst topping up leap cards. The display is excessively cluttered especially at the payment options screen. The leap card readers are not addressed on-screen and thus are difficult for novice users to locate. The touch screen can often be unresponsive and accidental selections are common.
This document contains the methodology and results of all conducted usability testing as well as recommendations for the improvement of the Luas TVMs. All relevant testing material is appended to this report. Luas TVM on Red line
3
4
methodology Task analysis The testing team began their preliminary investigation into the usability of the Luas TVMs through a task analysis of two distinct processes. The two tasks were chosen to each be unique yet typical tasks and were analysed by charting every step needed to achieve the goals. The tasks examined were: Task 1: Using the machine to top up a Leap card. Task 2: Buying a physical ticket without a Leap card.
Task 1
Task 2
Task analysis research
6
Task Analysis 1: Leap Card Payment
Leap card ‘Cash and Go’ screen
7
Topping up leap card
As a result of the first task analysis it was realised that that the method to top up a Leap card by any amount smaller than â‚Ź10 might not be immediately obvious to novice TVM users.
8
Task Analysis 2: Non-Leap Payment
Standard ticket payment screen
9
Adding more passengers
After the second task analysis it was discovered that there were a large number of steps required to purchase tickets for multiple travellers.
10
Defining Scenarios Drawing on the information gathered by the task analyses, two testing scenarios were established to be brought forward to user testing. These scenarios were designed to interrogate the abililty of the Luas TVMs to carry out the original tasks. Specific scenarios with defined goals were created instead of overly structured tasks which might have led the user by mentioning the UI structure or process and could have potentially influenced the testing. Scenario A You are a family of 2 adults and 2 children looking to travel oneway to Tallaght without leap cards. (Stop at payment) Scenario B You have a â‚Ź5 note, your leap card is out of credit and you want to get on the next Luas. (Stop at payment)
The success criteria established for the testing were to achieve the goal of each scenario without prompting or encountering errors and to complete the task within a reasonable amount of time.
Location In order to test a diverse sample of users that was representative of the Luas user base, the testing was conducted at the Red line stations Smithfield and The Four Courts on the Red line and the Green line stations Stephen’s Green and Harcourt. These stops were selected as they exemplified a range of both busy and quiet stations.
Smithfield Luas Station
11
Observation A 15 minute period of observation was undertaken at Smithfield station during rush hour. This allowed for insights to be gained at a time when people may not have been willing to participate in the testing. Observation was also carried out during the user testing as an additional method of collecting TVM user information.
Pre-task Questions Prior to presenting the scenarios, the testing team asked participants a series of questions to assess their level of knowledge of the ticket machines. These questions were kept brief and were intended to ease the participant into the testing process. Due to the relatively short amount of time between trams, it had to be made clear that the testing would not take very long. It was also important to focus on the fact that there could be no wrong answers and that the participants were free to give any response they desired. (See Appendix D) • What is your main form of transport? • How often do you use the LUAS? • Do you use a leap card? If YES • How do you usually top-up your leap card? Each participant was asked to then perform both scenarios A and B. The scenarios were explained to each participant and were also presented on a page in text for reference. Before the scenarios took place they were invited to think aloud as the tasks were performed. Each participant was also asked if they would give permission for the ticket machine screen and their hands to be recorded on video. During the scenarios users were reminded to think aloud and if struggling, encouraged to identify their problem.
13
User Testing Materials
Quantitative Measurements The following quantitative measurements were recorded throughout the course of the testing: • • • • • •
Whether the participant could complete the scenario. Time needed to complete the scenario. Whether the participant used a Leap card or bought a ticket in scenario B. Number of screens encountered during scenario. Number of touches required to complete the scenario. Number of errors made.
Debrief Questions After the scenarios were completed, each participant was debriefed in a semi-structured interview with a focus on the user’s self analysis: • • • • •
Did you have any issues or frustrations with using the ticket machine? How would you rank the experience of this machine out of 10? What would make it a 10/10 for you? Are there any features or improvements that you would like to see with this machine? Is there anything which influences what machine you use when purchasing a ticket / adding leap card credit?
14
Unpacking Information Positive and negative findings were gathered using the questionnaire results alongside video recordings of the user testing. Completion rates and other quantitative data was also collected in this manner.
User Journey Mapping A journey map was created for both scenarios; each divided into sections according to subtasks and then broken down into individual screens. Each user’s experience throughout both scenarios was charted along the maps on a scale from very negative experience to very positive experience. The journey maps were emphasized by direct user quotes taken throughout the process as well as other significant events such as errors encountered along the way. The testing team plotted pain points underneath the maps and subsequently listed opportunities to reduce these pain points.
Heuristic Evaluation Following the initial user testing a further session of usability testing was carried out. The testing team examined the TVM interface independently with reference to established usability principles or heuristics. These heuristic principles are as follows: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
Visibility of system status. Match between system and the real world. User control and freedom. Consistency and standards. Error prevention. Recognition rather than recall. Flexibility and efficiency of use. Aesthetic and minimalist design. Help users recognize, diagnose, and recover from errors. Help and documentation. User Journey Mapping Research
15
results Observations During the initial period of observation during rush hour at Smithfield station it was noted that no Luas travellers used the TVMs to top up their leap cards. This led to the inclusion of questions in the user testing which investigated usual top up methods. It was also observed that some users would pass over a machine in favour of others. This even meant sometimes crossing the tracks to use a TVM on the opposite platform to their travelling direction. In order to explore the possible reasons for this a question was included to assess potential influences on choosing a TVM. (See Appendix C)
Quantitative Data Scenario A Complete
Scenario B Complete
Time A | B (s)
Leap/ Ticket
Screens (no.) A|B
Touches (no.) A|B
Errors (no.) A|B
19s | 35s
Leap
6 | 4
8 | 3
0 | 1
35s | 32s
Ticket
6 | 4
7 | 3
0 | 0
35s | 28s
Ticket
8 | 7
9 | 4
1 | 1
21s | 15s
Ticket
7 | 6
7 | 6
0 | 2
55s | 17s
Ticket
8 | 4
9 | 3
1 | 1
29s | 45s
Leap
6 | 3
8 | 1
0 | 1
52s | Na
N/a
19 | N/a
4 | N/a
48s | 24s
Leap
26 | 3
1 | 1
User 1. Young Mother
User 2. Two Male Students
User 3. Two Young Men
User 4. Northern Man
User 5. Businessman
User 6. Young Woman
User 7.
Unpacking User Testing Data
Elderly Woman
11 | N/a
User 8. Two Young Students
6 | 6
18
Journey Map Scenario A Interviews & Usability Testing 1. Proficient after map screen.
7. Task ended due t & frustration. 7. Refused to pick Tallaght. “You can’t because this is the Green Line.”
2. “How do we double up?”
Positive Experience 1. “Where’s Tallaght?”
3. “Can you book a family ticket?”
7. Confused by map screen. 3. Accidental button press.
Neutral Experience
Negative Experience
Main Menu
Ticket Selection
Pain Points Main Menu Unclear where to buy multiple tickets.
Ticket Selection Choosing a location on the Dublin map slows people down. The Dublin map is an unexpected option. Users hesitate at the ticket type screen and look for a “family” ticket. Users unaware they could purchase Red Line tickets on the Green Line. Main menu button is very close to Tallaght on the map. Info about the walk to the opposite Luas line is not initially presented.
Opportunities Main Menu
Ticket Selection
Multi-ticket option.
A destination screen which doesn’t require knowledge of Dublin’s layout.
Ticket buying progress bar.
Increase clarity about Red/Green Line cross-travel. De-clutter selection screens. Provide a clear hierarchy to the ticket options. “Remember” user’s regular ticket choice.
19
to confusion
5. “I normally don’t use it, I’m just reading through it.” 8. Unresponsive button required 19 attempts.
User 1 : Mother Age 30 - 40 User 6 : Woman Age 25 - 35 User 2 : Male Students Age 20 - 30 User 3 : Two Men Age 25 - 35 User 5 : Businessman Age 30 - 40 User 8 : Students Age 18 - 20
User 4 : Northern Man Age 25 - 35
User 7 : Woman Age 60 - 70
Adding More Passengers & Information Screen
Adding More Passengers Buttons can be very unresponsive. Long waiting times when adding more tickets. It takes a long time to get to the final payment screen. Difficulty finding the “Add more tickets” option.
Adding More Passengers Family ticket option which skips to the “Add more passengers” screen. Reduce waiting times between screens. Options for expert users to increase speed.
20
Journey Map Scenario B Interviews & Usability Testing 3. Removed leap card - “Then I guess I just have to get the ticket.”
3. “You can’t top up by a fiver anymore can you?” 6. Machine wouldn’t read the leap card.
Positive Experience 1. Attempted to put the leap card into the bank card reader.
User 6 : Woman Ag
8. System timed out.
User 8 : Students A
Neutral Experience
Negative Experience
User 1 : Mother Age
Main Menu
Leap Top-Up
Pain Points Main Menu Leap card options are unclear.
Leap Top-Up Some people have difficulty with finding the leap card slot. Cash and go is an unclear option. Cluttered payment option screen. Long waiting period when removing the leap card (cancel transaction). Screen can time out while the user reads the payment options. The leap card reader regularly doesn’t work.
Opportunities Main Menu Provide on-screen leap card info.
Leap Top-Up Direct the user to the leap card slot with on-screen prompts. Payment screen overhaul to declutter and change “cash and go”. Offer help to the user rather than time-out during the pay screen.
21
ge 25 - 35
2. “Single adult... yeah it takes 5 euro notes” 4. Continued to click “Add more tickets” after finishing, possibly due to confusion of the task. 5. Decided not to use the leap card.
User 3 : Two Men Age 25 - 35 User 2 : Male Students Age 20 - 30
User 5 : Businessman Age 30 - 40
Age 18 - 20
User 4 : Northern Man Age 25 - 35
e 30 - 40
Ticket Selection
Ticket Selection People were unaware that they could top-up their Leap card with 5 euro. UI is oriented to standard ticket sales. Some users were unaware of what cash the machine accepts.
Ticket Selection Seamless integration of Leap card and standard ticket interface options. Increase awareness of what the machine can and can’t do.
22
Observations: Pain Points Major Severity Card payments fail regularly: • The user has to restart the entire process if the card payment fails as the machine cancels the transaction. Minor Severity Long ticket and change dispense time: • During observations a user missed the Luas due to how long it took the machine to dispense the ticket and change. • A user payed for an “Adult Single” ticket with a €20 note and had to wait as the change was dispensed entirely in coins. Cash notes are rejected regularly: • The machines can reject cash notes multiple times in a row. • One user moved to a different machine after their note was returned to her twice. The other machine worked normally.
Scenario A: Pain Points Major Severity The map of Dublin requires geographical knowledge of Dublin and causes confusion: • Most users would hesitate while looking at the map and become unsure of themselves. • User 8 selected the incorrect stop due to confusion as they had already spent a significant amount of time on the map screen. Adding multiple tickets confused and slowed down users: • Most users would initially look for a family ticket option before selecting the “Adult Single” ticket and then adding extra tickets. • User 5 returned to the ticket selection screen after selecting “Adult Single” as he believed he had made a mistake. • User 7 believed that she had to buy each ticket individually.
23
Information about crossing red/green line isn’t clearly indicated whilst purchasing tickets: • Additional information considering the walk between the Luas lines is only indicated if the user selects back from the pay screen and then re-enters it. • User 7 believed it was not possible to purchase a ticket for the Red Line on a Green Line machine and refused to do so. • The machines do not currently have any information regarding the impending cross city Luas connection. Minor Severity The process is lengthy in a situation where the user is rushing: • There are 6 screens that the user must progress through to complete Scenario A. • Users took between 19 seconds and 55 seconds to get from the main menu screen to the payment screen (times do not include payment). Positives • • • •
Destination Map Selection Screen Causes Confusion
The map screen supports a user’s mental model of Dublin if they are aware of the city layout. The ticket price updates in real time so the user is always aware of the additional costs. It’s easy to go back or return to the main menu if needed. Language and audio options are available and clearly accessible.
Long delays between certain screens
24
Scenario B : Pain Points Major Severity Machine can time out while choosing payment amount. • The machine timed out and cancelled the transaction before user 8 could find the correct payment option. • New users reading the information on the screen may have to restart the process. Payment option screen is cluttered and lacks hierarchy. • There are a lot of options and text throughout the payment screen which can be overbearing. • Buttons such as the “Fill Up” option are very close to the standard top-up options which could result in a large payment if the user is unaware. • No users were aware of the “Cash & Go” option before the scenario. Leap card readers are difficult to find and occasionally don’t work. • There is no on-screen information on the main menu relating to the leap card. • User 1 was confused about the use of the Leap card and attempted to insert it into the bank card slot. • User 2 wasn’t sure where to insert the Leap card and opted not to use it for the scenario. • The Leap card reader didn’t work for user 6 during the scenario so another machine had to be used. Minor Severity “Cash and go” is ambiguous and easily missed. • User 3 mentioned that he used to be able to top up by €5, he was unable to find the option after inserting the Leap card and decided to buy a standard ticket. • The machine timed out while user 8 was looking for the option to top up by €5. Positives • • •
25
The user can check their current Leap card balance and their travel history. Cash added is updated in real time. The payment screen only shows the necessary information.
Overall System: Pain Points Major Severity Buttons can be unresponsive and accidental presses are common. • The “Main Menu” and “Back” buttons are very close to the Zone 4 map options opening the user up to accidental presses. • User 3 accidentally returned to the main menu from the map screen. • User 8 needed 19 presses to add extra tickets to the order due to the unresponsive interface. Minor Severity Very long waiting times between some screens can hinder users who are in a rush. • The machine takes a long time to load the screen for adding more tickets, it also makes an audible mechanical noise while preparing for the transaction. • Cancelling a transaction takes a significant amount of time.
Transaction times out too rapidly
‘Cash and go” is an ambiguous label
26
Visibility of System Status
Match Between the System and the Real World
1. 2.
Consistency and Standards
User Control and Freedom
4. Error Prevention
3.
5. Recognition Rather Than Recall
Flexibility and Efficiency of Use
6. 7. Aesthetic and Minimalist Design
Help and Documentation
Help Users Recognise, Diagnose and Recover From Errors
8. 9. 10.
The name “Flexi Ticket” is 2. unclear.
Language and audio options + are available.
The main menu is different
option.
The current stop flashes + while on the map screen.
The map supports a user’s
mental model of Dublin if + they know Dublin’s layout.
The physical map above the
unfamiliar with Dublin’s 3. layout.
The map confuses those
The proximity of “Back” and
from the DART machine and screen and the on-screen + map 2. does not offer a Leap card are consistent.
The more common “Standard
Selecting ticket type on main menu adds an extra step.
Ticket” option is larger and “Main Menu” buttons is too + more 2. close visible. to the map options.
1.
User is not warned of the 15 minute walk between Red
to the pay screen.
and Green lines unless they 3. press “Back” and then return
Ticket menu is unclear where
you can purchase multiple 3. tickets.
Ticket information explained
graphically in the map.
textually (Central 1 to RED 1) 2. when previously explained
The ticket menu has a lot of options and looks 2. cluttered.
Users can time-out while 3. thinking, without a prompt.
Searching for the “Add more tickets” option 2. slows down the user.
Menu is very cluttered and 2. text heavy.
+
Long waiting times when 2. adding more tickets.
Waiting times here are
updates in real time + asPricetickets are added.
The only screen which allows multiple options to be 1. chosen.
Five travellers of a single type is the maximum 2. amount
“Main menu” is very close to
Unclear that coins are the only form of change.
Easy to return to the main + menu if needed.
1.
more tickets” and could 2. be“Addpressed accidentally.
Failed bank card payments
Overall System
No navigation map available 2. throughout the process.
The system supports a “Back” + button. Only linear navigation is 2. supported. 3. Buttons are unresponsive.
+ Menu buttons are consistent.
DART and Luas machines 2. are inconsistent.
Unresponsive screen could
lead to mistakes or being 2. delayed.
No navigation map available 2. throughout the process.
Pages are cluttered and 2. unclear. is clunky and difficult 2. toDesign read.
Transaction cancellations return the user to the main 3. menu + are clearly stated. without a prompt.
3. Major Severity
4. Critical Severity
answered.
Luas website is not user User is not warned of the 1. friendly. 15 minute walk between Red and Green lines unless they 3. press User claimed the emergency “Back” and then return 0. call button does not get to the pay screen.
1. Cosmetic Severity 2. Minor Severity
with the rest 2. ofinconsistent the process.
Positive
27
Heuristics
The tables on this page show the results of the heuristic evaluation conducted by the testing team. Each scenario was assessed on a screen by screen basis and as an overall interface with reference to the ten established heuristic principles. Any issues flagged were ranked on a scale of 0-4
A large amount of global issues pertaining to the entire interface were uncovered at this stage. These include the unresponsive buttons and overall clunky visual aesthetic.
28
6.
Help and Documentation
+
Help Users Recognise, Diagnose and Recover From Errors
10.
9.
8.
Aesthetic and Minimalist Design
Flexibility and Efficiency of Use
Recognition Rather Than Recall
5.
7.
Error Prevention
4.
User Control and Freedom
3. Consistency and Standards
Match Between the System and the Real World
2.
1. Visibility of System Status
can time-out while 3. Users thinking, without a prompt.
is packed with 2. Screen too many options.
“Cash & Go” allows users to
“Fill Up” option is very close to the regular Leap card top-up amounts.
by any amount above + top-up five euro.
3.
+
The user can check current Leap car balance and their payment history.
“Cash & Go” is an unfamiliar
and could cause the 3. phrase user to avoid it.
4. Critical Severity
answered.
website is not user 1. Luas friendly. claimed the emergency 0. User call button does not get
cancellations + Transaction are clearly stated.
are cluttered and 2. Pages unclear. is clunky and difficult 2. Design to read.
navigation map available 2. No throughout the process.
return the user to the 3. could main menu.
Accidental button presses
to mistakes or being 2. lead delayed.
Unresponsive screen could
and Luas machines 2. DART are inconsistent.
+ Menu buttons are consistent.
system supports a “Back” + The button. linear navigation is 2. Only supported. 3. Buttons are unresponsive.
card can be removed 2. Leap after top-up.
Unclear when exactly the
navigation map available 2. No throughout the process.
3. Major Severity
is clear and only shows + Screen necesary information.
top-up has no info on 3. Leap stops or routes.
shows current balance, + System new balance and cash inserted.
option is very close 3. “Back” to the “Top-Up” option.
inserted is updated on + Cash the screen in real time.
1. Cosmetic Severity 2. Minor Severity
Leap car pocket could be unclear.
The main menu is different from the DART machine and does not offer a Leap card option.
Positive
1.
2.
+
Language and audio options are available.
on-screen leap card 2. No information.
menu change when + Quick Leap card is inserted.
Overall System
Recommendations Recommendation
A
Justification
Priority
Create a quick ticket option for multiple travellers/families.
Adding multiple tickets is currently confusing and slowed down users hugely in testing.
High
Replace map based selection or provide an alternative method for selecting desired destination.
Unlike on Dart TVMs which use an alphabetical search system it is difficult to quickly find a stop on the Luas's zonal map system without prior knowledge of its geographical location.
High
Overall
Reduce waiting times between screens.
Many screens have lengthy waiting times to load which slows down ticket purchasing dramatically
High
A
Provide a clear hierarchy to the variety of ticket options to reduce chance of accidental presses.
Ticket options as well as Leap card buttons ‘Fill Up’ and ‘ Cash and Go’ are too close together and too similar in size causing confusion and accidental selections.
Mid
Include an indication of current progress in relation to total ticket purchase.
Would prove useful in a situation where the user is rushing and needs to know how much longer it will take. This could also be used as a navigation system to ‘jump’ backwards within the interface.
Mid
Increase clarity of Luas cross-city travel through a prompt when selecting.
In preparation for the impending Luas cross-city connection more emphasis should be placed on ability to travel from line to line.
Low
Provide frequent fare ticket options to act as shortcut for expert users.
Dart TVMs use a system of frequent fares which are presented early on to users and vary from station to station. This could also be implemented on the Luas to speed up users.
Low
Change name of 'Cash and Go' to something more indicative of its function.
‘Cash and Go’ is an ambiguous name which does not tell the user much about its function as the location for topping up leap cards by amounts less than 10 euro.
High
Provide leap card option on the main menu which directs user to the leap card reader.
Multiple users found it difficult to locate the leap card reader despite it being placed beside the screen. On-screen directions should be provided.
Mid
Offer assistance to users rather than time-out (at least a timer or warning)
Time outs happen very rapidly and without warning forcing the user back to the main menu.
Mid
Provide better feedback about leap card status
It is unclear to users when exactly they can remove their leao cards after topping up.
Mid
Revise visual layout of interface to remove excessive 'clutter'.
Poor visual layout often leads to mistakes from users and important information is easy to miss due to lack of visual order on screens.
High
Implement a more responsive touch screen system.
Current touch screen is very ineffective at times and frustrates users
High
Provide contactless/NFC Payment options.
Contactless payment would further speed up transaction times as well as combatting the issue of not registering bank cards.
Mid
Print receipt on request only
Receipts are often left to pile up in the machine as users rarely take their receipt with them.
Low
Screen 1
A Screen 2
A
Screen 3
A Overall
A Screen 2
A Screen 2
B Screen 2
B Screen 1
B Screen 3
B Payment
Overall
Overall
Payment
Payment
Empty Luas Station at Harcourt
29
Conclusion Analysis and user testing of the Luas TVMs has provided the research team with the data to conclude that there is clear scope for improvement for the machines. While the machines do not have any problems of critical severity, they contain a multitude of major problems which hinder the usability and experience of the user. Consistent areas of confusion common to the majority of users such as the map screen exist throughout the platform. Unclear visuals and lengthy, needlessly complicated processes can frustrate the user in an area where speed and ease of use are crucial. Implementing the recommendations above and continuing to gather and analyse valuable user research will lead to a more user-centered experience, reduce queues and increase overall customer satisfaction.
Appendix A Hierarchical Textual Description 1
32
Appendix B Hierarchical Textual Description 2
34
Appendix C Observations at Smithfield Luas Station
Observations at Smithfield Luas Station 4th October 2017 : 5.30-5.45pm P1 : Towards Connolly / The Point P2 : Towards Tallaght / Saggart Use Study of P1 5.30-5.35pm: Ɣ No machine use on P1 Ɣ Luas to Connolly arrives P1 at 5.35pm. Approx. 10 people boarded. 5.35-5.40pm: Ɣ 1st user P1: Attempted to pay for ticket with €20 note. Machine rejected note. Duration 25s Moved to second machine and successfully purchased ticket with same €20. Duration 11s (Additional 10s for change) Ɣ 2nd user P1: Paid for ticket with card. Did not collect receipt. Duration 35s Ɣ 3rd user P1: Paid for ticket with cash. Collected change. Duration 15s Ɣ 4th user P1: Paid for ticket with exact cash. No change collected. Duration 8s Ɣ Luas to The Point arrives at 5.40pm. Approx. 18 people boarded. 5.40-5.45pm: Ɣ 1st user P1: Paid for ticket with exact cash. No change collected. Duration 12s Ɣ Luas to Connolly arrives at 5.43pm. Approx. 10 people boarded. User Observations: Ɣ Ɣ
Ɣ
Ɣ
Ɣ
Ɣ Ɣ Ɣ
Young male paid for leap card top-up on the machine with cash, inserted leap card and one screen press. Male student arrived at P2 at the same time as the Luas. He pressed the screen 3 times for a single ticket and inserted cash. The ticket took too long to print and the Luas left before he received his ticket and change. A middle aged woman attempted to pay for a single ticket with cash and realised she didn’t have enough change in hand. She had to search for more change in her bag. She pressed the screen 4 times. A young woman was buying a ticket with cash, she inserted a €20 note which was rejected by the machine. She moved onto the other machine which was accepted. The change was dispensed completely in coins which took roughly 10 seconds to dispense. Young man placed his coffee on the seat and bought a ticket with his card on P1 without taking the receipt. He walked across to P2 to wait for the Luas. There were no queues at the ticket machines on the side that he was waiting for the Luas. A man paying for a ticket with change didn’t have enough change on hand and had to search through his bag for more. He apologised to the person waiting behind him. Most leap card users did not use the ticket machines and tagged on at the scanners. [On Red Line] It is common for a group of people to crowd the machines and seats without getting on a Luas.
36
Appendix D User Testing Questionnaire
Notes
Participant:
What is your main form of transport?
Do you use a leap card?
How do you usually top-up your leap card?
Tallaght Scenario
Leap Top-Up Scenario
Did you have any issues or frustrations with using the ticket machine?
How would you rank the experience of this machine out of 10?
What would make it a 10/10 for you?
Are there any features or improvements that you would like to see with this machine?
Is there anything which influences what machine you use when purchasing a ticket / adding leap card credit?
38
Appendix E Individual User Journey Mapping Scenario A
Individual Journey Maps Scenario A: You are a family of 2 adults and 2 children looking to travel one-way to Tallaght without leap cards.
User 1 : Mother Age 30 - 40
User 2 : Male Students Age 20 - 30
1. Knew exactly how to add more tickets.
2. Hesitation on the map screen with brief hesitation when adding more tickets. Overall proficient users.
1. Expert user who knew the machine well.
1. No problem with ticket screen. 2. “So 2 and 2 and continue.”
Positive Experience
Positive Experience 2. “Tallaght? Where’s Tallaght?”
2. “How do we double up?”
1. “Where’s Tallaght?” 2. “One way... So we’re going one way... So two adults.”
Neutral Experience
Neutral Experience
Negative Experience
Negative Experience
Main Menu
Ticket Selection
Adding More Passengers & Information Screen
Main Menu
User 3 : Two Men Age 25 - 35
Adding More Passengers & Information Screen
User 4 : Northern Man Age 25 - 35
3. No problem adding more tickets.
3. “2 Adults, 2 children, and that’d be it.” 3. Initial confusion due to the accidental button press but but managed to add the extra tickets quickly.
Positive Experience 3. “Okay so, standard tickets.”
Ticket Selection
3. “Do you book them individually or is there a way that you can book a family?”
3. Accidentally pressed the main menu button.
3. Hesitation
Positive Experience 4. No problem choosing standard tickets from main menu.
3. “To Tallaght...”
4. Hesitation when adding tickets.
Neutral Experience
Neutral Experience
4. The user seemed slightly confused throughout the scenario although this could be due to being put on the spot.
4. Hesitation at map screen.
Negative Experience
Negative Experience
Main Menu
Ticket Selection
Adding More Passengers & Information Screen
Main Menu
User 5 : Businessman Age 30 - 40
Ticket Selection
Adding More Passengers & Information Screen
User 6 : Woman Age 25 - 35
6. Easily added more tickets.
6. Expert user with no major problems.
5. No problems adding more tickets.
Positive Experience
Positive Experience 5. Became proficient after recovering from the ticket screen confusion.
5. Hesitation at map screen.
6. Quick ticket selection.
5. “I normally don’t use it, I’m just reading through it.”
6. Slight hesitation at map screen. 5. Quickly saw the “add more tickets” option the second time.
Neutral Experience
Neutral Experience 5. Wasn’t sure if he made the right choice so went back to the previous screen.
Negative Experience
Negative Experience
Main Menu
Ticket Selection
Adding More Passengers & Information Screen
Main Menu
User 7 : Woman Age 60 - 70
Ticket Selection
Adding More Passengers & Information Screen
User 8 : Students Age 18 - 20
8. Generally proficient at using the machine, the critical error may have been due to the pressure of being put on the spot.
7. Confused by the map screen.
Positive Experience
Positive Experience 7. “Do I have to buy the tickets individually?...”
7. Navigated between the map screen and the main menu multiple times.
8. “So you just have to keep on doing it then?” - Talking about adding more passengers.
8. “I actually don’t know where Tallaght is (laughs)”
7. “You want to go to Tallaght?...” “You can’t because this is the Green Line.”
8. “I’m probably doing what they don’t want me to do...”
8. Button was unresponsive and took 19 attempts to work.
7. Critical error - user selected the incorrect stop without realising.
Neutral Experience
Neutral Experience 7. Critical error - user selected the incorrect stop.
7. Scenario ended by interviewers due to user frustration and cunfusion.
Negative Experience
Negative Experience
Main Menu
Ticket Selection
Adding More Passengers & Information Screen
Main Menu
Ticket Selection
Adding More Passengers & Information Screen
40
Appendix F Individual User Journey Mapping Scenario B
Individual Journey Maps Scenario A: You are a family of 2 adults and 2 children looking to travel one-way to Tallaght without leap cards.
User 1 : Mother Age 30 - 40
User 2 : Male Students Age 20 - 30
2. “Single adult... yeah it takes 5 euro notes” 2. They didn’t consider using the leap card at all, and questioned if the machine accepted 5 euro notes or not.
Positive Experience
Positive Experience 1. She was unsure about how a leap card works.
2. “You want to select top-up or something?” 1. Attempted to put the leap card into the bank card reader.
Neutral Experience
2. Gestured to cash slot - “Yeah here, so standard ticket.”
Neutral Experience 2. “Just put the 5 euro note in the...”
1. Found cash and go after prompting 1. Not knowing where to put the leap card frustrated her and threw her off for the rest of the scenario.
1. Found the slot after prompting.
Negative Experience
Negative Experience
Main Menu
Leap Top-Up
Main Menu
Ticket Selection
User 3 : Two Men Age 25 - 35
Leap Top-Up
Ticket Selection
User 4 : Northern Man Age 25 - 35
3. Proficient at standard ticket process.
Positive Experience
3. He used to always top-up with a fiver but believed the option was gone due to the “cash and go” option.
3. “You can’t top up by a fiver anymore can you?”
Positive Experience
4. Decided not to use the leap card. 3. Hesitation at the main menu.
4. Continued to click “Add more tickets” after finishing, possibly due to confusion of the task.
4. Hesitation before selecting “Standard Tickets”. 3. Selected standard ticket.
Neutral Experience
Neutral Experience 3. Inserted leap card - “Yeah just a tenner.”
3. Removed leap card - “Then I guess I just have to get the ticket.”
Negative Experience
Main Menu
Leap Top-Up
Negative Experience
Main Menu
Ticket Selection
User 5 : Businessman Age 30 - 40
Leap Top-Up
Ticket Selection
User 6 : Woman Age 25 - 35
6. She was profecient with the process but the broken machine extended the process.
Positive Experience
Positive Experience 5. No hesitation selecting “Standard Tickets”.
5. Decided not to use the leap card.
5. No problems throughout, didn’t use the leap card as he doesn’t usually top it up himself (Tax Saver).
6. “Oh...? Oh cash and go, right.”
6. Machine wouldn’t read the leap card.
Neutral Experience
Neutral Experience
Negative Experience
Negative Experience
Main Menu
Leap Top-Up
Ticket Selection
Main Menu
User 7 : Woman Age 60 - 70 (Did Not Attempt)
Leap Top-Up
Ticket Selection
User 8 : Students Age 18 - 20
8. “That’s weird isn’t it?” - Unable to find how to use a fiver.
Positive Experience
Positive Experience
Neutral Experience
Neutral Experience
Negative Experience
Negative Experience
8. The ambiguity of how to top the leap card up by a fiver caused the user to think and the system eventually timed out.
8. System timed out. 8. “For feck sake.”
Main Menu
Leap Top-Up
Ticket Selection
Main Menu
Leap Top-Up
Ticket Selection
42