Project Report | Project Usability and User Experience Assessment in Design (Part 2) by Yu Huang

RaceClocker.

A project about the usability and user experience of RaceClocker. Coaches Arnold Vermeeren Marina Wellink Stakeholder RaceClocker Cees van Dok Project team A1 Anna Smulders Maureen Sanchez de Boevé Shunqi Tang Shuyue Jin Yu Huang Team A1 | RaceClocker

June 22th 2021 | Design for Interaction | IDE faculty - Delft University of Technology

UXAD 2021 | Phase two

Intro. Subtitle 1 2

RaceClocker is an online timing tool for manual timing of sport races. Its goal is to provide race managers and timekeepers with accurate racer timing for small-scale races. In this project, our team - consisting of five Design for Interaction students from the IDE faculty at TU Delft - has researched and redesigned this race timing tool in order to improve the usability. The project is part of the course Usability and User eXperience Assessment in Design (UXAD) and is divided into two ten-week phases. In the first ten weeks we evaluated the current timing tool and identified design opportunities. This process is described and illustrated in our previous report of phase one.

In this report phase two is described, in which we focussed on conceptualizing and detailing the redesign concept. This report shows our prototyping process, final redesign, user testing and recommendations.

Figure 1 RaceClocker ‘s interface of a timer Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

Table of contents. 4

53 4. User testing 7

8 12

54 56 57 58 60

1. Recap of phase one Design Brief. Concept direction

15 2. Iteration 16 18

Iteration process Reasoning behind design decisions

33 3. Final redesign 33 34 36 40 42 43 50 Team A1 | RaceClocker

Metaphor Storyboard Use flow Information architecture Design Guidelines User interfaces 5 design solutions Team A1 | RaceClocker

Purpose of the test Participant & schedule Test content Set-up Methods for data collection

65 5. Data analysis 66 67 78

Data collection Data results Data interpretation

87 6. Recommendations 88 92

Quick fixes and interfaces Future recommendations

95 Refrences 96 Appendices

UXAD 2021 | Phase two

1. Recap of phase one.

After user testing and evaluating the current timing service of RaceClocker during phase one of the project, we shaped a design brief. The design brief presents the research findings and evaluation results, in order to improve the timing tool, including problem statements, scope, design goal, and design criteria. The complete Design Brief can be found in report one. We ended phase one with a concept direction based on this design brief and individual shaped concepts, which is summarized in this chapter. The report of phase one contains a detailed overview of this concept direction.

Team A1 | RaceClocker

UXAD 2021 | Phase two

Design Brief Problem statement

To summarize the main research findings and evaluation results, we divided our findings into five important categories.

1. 2. 3. 4. 5.

The RaceClocker app’s current hierarchy of pages causes confusion among users. Users can easily get lost in the many pages due to a complicated useflow. The user receives no clear feedback after clicking a button or icon. Some buttons’ or icons’ meanings cause misunderstanding or confusion. While using the product, eye-hand coordination and operation within the timer is difficult to manage.

Scope

The current timing tool RaceClocker consists of three use phases (Figure 2). Preparing the race as a race manager (1), timing the racers as a timekeeper (2) and after the race, sharing and viewing the race results for both race managers, time keepers, audiences and many others (3). After evaluating the usability of RaceClocker, we decided to focus on the second phase, keeping track of time during a race. Since we discovered this timing phase has an important impact on a sports race.

Figure 2 Scope > Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

Design goal and vision

UI Design criteria

We want to make frequent users feel in control when using RaceClocker. Time recording during a race should be intuitive, efficient and fast. Besides that, users should feel confident when using and navigating through the functions RaceClocker offers.

Based on our problem statements, the following criteria are formulated. These form a guideline within the redesign process for improving the user experience and interaction with RaceClocker.

As a vision, we want to use RaceClocker to bridge the gap between the physical and digital world for timekeepers.

UXAD 2021 | Phase two

Hierarchy

Use flow

Feedback

Information of buttons

Operations

•

The interface should be simple and clear. Users should clearly know in which status of the process flow they are.

• • •

Team A1 | RaceClocker

The layout of the interface should comply with conventional using habits, which means it should be logical and intuitive for users. The interface system should be flexible and adjusted to the preference of the user. The number of functions for the timekeepers should be reduced compared to the current product. The right information needs to be given at the right time.

Users should automatically get instant and correct feedback after each action.

• •

Users can finish their tasks with RaceClocker without wrong attempts. The meaning of the buttons is understandable for users without resorting to external sources. The interface of the website should be readable for people with visual impairment.

Team A1 | RaceClocker

•

The product enables users to time the racers without having handeye coordination problems. Users are able to undo the wrong action easily.

UXAD 2021 | Phase two

Concept direction end of phase one Based on our criteria and the individual concepts designed, we set our design directions to lead our design process. Some main design solutions are summarized in this section, the complete design direction can be found in the report of phase one, chapter four.

Figure 3 Welcome page 12

Smooth user flow We intended to use the onboarding pages to simplify the use flow and make users experience the whole process confidently. For example with guiding the user to the timer with a welcome page (figure 3), the user receives more guidance and confidence will therefore increase. Physical interaction To provide the most accurate time to timekeepers, we make full use of the physical interaction in the phone including the volume button to record the time, as an example in figure 5 shows. Also the vibration feedback after timing in figure 4, provides a haptic feedback to the timing of the user. Combining them with the digital screen, the bridge between the digital world and the physical world can be built.

Figure 4 Sound or vibration

Figure 5 Volume button timing Volume button

Welcome screen Sound or vibriation feedback

Digital information Due to the many changes in the interfaces for the user (e.g. racers who have assigned a time or races that did not pass the line yet), color codes will be used to illustrate the different statues. Landscape operation The landscape will be the main mode for users to operate considering the multiple elements in a race. Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

2. Iteration

This chapter describes what we did throughout the iteration process, and the reasoning behind the main iteration cycles that led to the final design (before user tests). We started from visualising the initial ideas we generated in the first phase via inVision. After peer testing, we converged our ideas into a coherent solution (iteration 1) and made a hi-fidelity prototype in Figma. Later we integrated the feedback from pre-pilot tests and transferred to Protopie for crafting our final prototype to fully realize and implement all of our concepts (iteration 2&3).

Team A1 | RaceClocker

UXAD 2021 | Phase two

Iteration Process INDIVIDUAL CONCEPTS

INITIAL IDEATION lo-fi protoyping

USER TEST

PEER TESTING with students

We used inVision to quickly visualize our initial ideas.

Peer testing setup

Actions - We used inVision to visualise our individual concepts and make lo-fi prototypes

Actions - We printed all interfaces we drew via inVision, clustered them in different piles as the testing setup.

Purpose: visualisation In this phase, we didn’t limit ourselves that much in terms of ideas. We just listed all of our initial ideas and quickly visualized them for further tests, based on the results of which we intended to select our ideas.

Purpose We were aimed to test different parts of our design separately and compare different solutions by asking for feedback.

https://uxada1.invisionapp.com/freehand/UXAD-1tJ2InmBa

Team A1 | RaceClocker

ITERATION 1 hi-fi protoyping

PRE-PILOT TEST with coaches

Research questions and corresponding interfaces.

Actions - We decided to use Figma to make the hi-fidelity prototypes. - We ruled out some of our concepts according to the feedback and kept some of them (please refer to the next part for more details). - We structured the information architecture according to the user journey and some extreme scenarios to decide the interfaces we were going to include in the hi-fi protoypes. Purpose: structure and converge In this phase, we started to rule out some of our concepts and focus more on validating those with more potential.

ITERATION 2 hi-fi protoyping

We set up UI guidelines in Fgma and realize complicated effects via Protopie.

Actions - We set up the pre-pilot test remotely and arranged the tasks carefully to make sure the test & feedback session wouldn’t last over 1h.

Actions - We decided to transfer to Protopie to realize more intended effects in our concepts. - We decided our UI guideline which includes the sizes of buttons, the font, the color code etc. - We fine tuned several details to improve our redesign (please refer to the next part for more details).

Purpose We expected to test part of our prototypes and got some practical suggestions on testing process.

Purpose: realization By using Protopie, we were able to realize our concepts more thoroughly (eg. the realization of haptic feedback, and the control of hard keys etc.).

Team A1 | RaceClocker

USABILITY INSPECTION

ITERATION 3 hi-fi protoyping

We tested the setup for the final user tests ourselves.

Actions - We tested our final design within our group before the first user test. Purpose We intended to prevent a large part of the user test being ‘wasted’ on problems that we could have discovered before conducting the actual test

Actions - We fine tuned several details to improve our redesign (please refer to the next part for more details). Purpose: fine-tuning We fine-tuned several details to get prepared for the actual user tests.

UXAD 2021 | Phase two

Reasoning behind design decisions

( follow the last page)

Initial Concepts REASONING

INITIAL CONCEPTS

initial concepts

lo-fi protoypes

Flexible Layout

REASONING

INITIAL CONCEPTS

initial concepts

lo-fi protoypes

Gesture Design

To make the interfaces suit different preferences of different timekeepers, we made it easy to customize the interface. For example, users can easily change the size of buttons by pinching with 2 fingers. The digital timing button can be easily moved to different places on the screen as users wish.

We used gestures like ‘long press‘ to trigger the expansion of the drop list of more actions. In this way, we make the main interface simple and only contain the necessary elements related to timekeeping.

Users can long press the button for more operations.

Color Code 1.0

We used different colors to indicate different states of the participants: orange indicating the participant has already passed the last split point and is approaching, green indicates the participant hasn’t passed the last split point, and grey indicates those timed participants.

Users can swipe left or right for more operations.

Sorting Ways

The feature of sorting by the approaching states is aimed to cluster those approaching participants to help timekeepers to reduce their cognitive load (screen 1). This sorting way can be quickly switched into sorting by the order of numbers (screen 2), which helps timekeepers to find a number quickly. The ‘hide/show the timed participants’ helps them to focus more on the unregistered participants (screen 3). ( continue on the next page)

2 Directions

Because we found that some timekeepers like to use RaceClocker in a landscape direction, we decided to design for both directions, including the landscape and portrait direction.

screen 1

screen 2

Team A1 | RaceClocker

screen 3 Team A1 | RaceClocker

UXAD 2021 | Phase two

Iteration 1 PEER TESTING with students

Peer testing

REASONING

Fixed Layout

Together with another UXAD student group we tested and discussed the interfaces presented above.

INTERIM DESIGN

iteration 1

hi-fi protoype

Revised

Based on the feedback we got from the peer test, we decided to abandon the flexibility in layout to avoid inconsistency which might cause confusion. We also separated the one-click mode with two-step mode (first record the time then assign the time to racers) and decided to mainly focus on the two-step mode since it was regarded as the most logical way of timekeeping. We remained the 2 directions and designed both list and grid layout for two-step mode.

UI Guidelines

New

We generated a set of UI guidelines to make sure the visual language in the product is clear and coherent.

Color Code 2.0

Revised

We changed the color code into a more intuitive one. We abandoned the color of green and used dark blue for selected racers, grey for timed ones, orange for soon approaching ones and white for deselected ones.

Team A1 | RaceClocker

( continue on the next page )

Team A1 | RaceClocker

UXAD 2021 | Phase two

( follow the last page ) racer

REASONING

INTERIM DESIGN

iteration 1

Remove the Digital Timing button

Add Indicators of the Number of Assigned to a Certain Time

with coaches

Pre-pilot test

Revised

A pre pilot test was executed in order to find final aspects to be revised before executing our final tests.

We removed the digital timing button. A timekeeper can assign a time by pressing the volume button only, to avoid confusion.

PRE-PILOT TEST

hi-fi protoype

New

We added a number in the timing button to indicate the number of racers that have been assigned to this time to provide more feedback.

Confirmation of Assigning

New

We also added a confirmation step for assigning. After finishing assigning, the user can click the time button again to confirm and the time will disappear.

Team A1 | RaceClocker

UXAD 2021 | Phase two

( follow the last page )

Iteration 2 REASONING

FINAL DESIGN

iteration 2

hi-fi protoype

Recommendation on choosing New the timing mode

REASONING

FINAL DESIGN

iteration 2

Time area on the left

hi-fi protoype

Revised

The area showing recorded times and the area of racers are swapped, because it is more logical to see which time has been assigned with which racers in this way.

Findings from the (pre)-pilot test showed that users do not have time to think about which is the best mode to use in a stressful situation. They will do what they are used to. Therefore, we decided to make one-step timing and two-step timing separate from each other. The system will recommend the user the best mode for the timekeeper based on the number of racers and the user can only use the chosen mode of timing during the race.

Hints for the user

With one-step timing mode the user can time the racers efficiently. The onestep list timing mode is recommended to use in a race with less than nine racers, because one screen can fit eight racers maximum. For the same reason, the onestep grid mode can be best used for races with less than 33 racers. With two-step timing mode the user can time the racers accurately. They are suggested to be used in the race with more than 32 racers, because there will be a bigger chance to have the situations in which multiple racers are passing by at the same time or closely to each other.

New

When the user is navigated to the timer dashboard, the actual time will first show in the empty time area. There will also be a hint for the timekeeper that the registered times will be shown here. Instead of having a navigation button to the result page collected in more options, we decided to have a popout window which invites the user to the result page after all the racers are timed.

( continue on the next page ) Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

( follow the last page )

REASONING

FINAL DESIGN

iteration 2

Color code

hi-fi protoype

Revised

The intended two-step timing has now four steps to do. The confirmation step is removed to shorten and simplify the timing process. In order to make it easier to reassign, the time boxes will not disappear after assigning the racers. In this way, the user can easily go back to a specific time box and check if the racers are assigned correctly or reassign the time to another racer.

The color code for the time box is also changed in a way that makes the system more consistent. The time boxes have a dark blue color as the default color, so that the times and racers can be easily distinguished from each other. The selected time box and the selected racers are highlighted in orange. In this way, there is a stronger connection between the time and corresponding racers.

Vibration feedback

FINAL DESIGN

iteration 2

4-step to 3-step

Revised

The color changes for the approaching racers can cause confusion for the user. In order to make the interface more simple, we decided to leave out the color indicators for approaching racers. There are only three states for racers now. White is the default color, orange means being assigned, and gray means timed. 26

REASONING

Undo/reasssign

New

There are no undo and reassign buttons anymore, because it is more logical to deselect and select racers for undoing and reassigning racers. Deselecting participants needs a double-tap to prevent wrong presses.

In a stressful situation the user will not have time to look down at the phone to check if the time is successfully registered. Therefore, haptic feedback after pressing the volume button enables the user to register time with no need to look at the screen. In our hi-fidelity prototype, there will be a default vibration every time the volume button is pressed.

( continue on the next spread ) Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

Iteration 3 USABILITY INSPECTION

Usability inspection

REASONING

Show assigned time next to the name

A usability inspection was executed by ourselves in which we tested the final setup before the user test.

FINAL DESIGN

iteration 3

hi-fi protoype

Revised

During the usage inspection we found out that the connection between the time and its corresponding racer is not clear enough. To quickly fix this problem, we added the assigned time next to the name of the racer when the racers is selected.

3-step to 2-step

Revised

In order to make the timing process shorter, we decided to make the last recorded time to be automatically selected. In this way, users can start assigning immediately after recording the time.

Team A1 | RaceClocker

UXAD 2021 | Phase two

( follow the last page )

REASONING

FINAL DESIGN

iteration 3

Vibration mode

hi-fi protoype

Revised

After the pilot test we changed the vibration mode from default to tap on the Android phone, because the default vibration is too intense and too long which results in hindrance for quickly recording times after each other.

Next we exectued the user tests. An elaborated description of the final user tests can be read in chapter 4. FIrst we will explain the Final redesign in detail in the next chapter.

Undo

Revised

After the usage inspection we decided to change the double tap to one tap for deselecting the racer, because it is more logical to have the same operation for selecting and deselecting in conventional usage.

Team A1 | RaceClocker

UXAD 2021 | Phase two

3. Final redesign. The design brief functions as a guideline to get started with the redesign process of RaceClocker. This chapter describes the final redesign. We present the context of use, intended interaction and use flow, detailed user interfaces of our redesign. We closed this chapter by summarizing how we solve the problems we stated in the first phase through design. The design reasoning can be found in chapter two.

Metaphor

We use a stopwatch as the metaphor for our redesign (Figure 6). Because Raceclocker’s target races are those who are organized by some clubs chips or transponders using a stopwatch to record time is the most natural and comfortable approach for timekeepers to feel in charge.

Interaction - Physical button Physical interaction to record the time will strengthen the feeling of control and confidence when recording the time. As this gives the user the ability to track the times while looking at the race, instead of having a constant eye on the phone. Flow- Time first Instead of an one-button-click on the racer, we use a two-step way. Similar to using a stopwatch, record the time and then assign this to the corresponding racer. Again giving the user the ability to track the time, keep their eyes on the race and then assign it to the corresponding racers. Figure 6 Physical stopwatch interaction

Team A1 | RaceClocker

UXAD 2021 | Phase two

Storyboard A storyboard (Figure 7) is created with the user journey of the redesign of RaceClocker.

Figure 7 Storyboard of use Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

Use flow To let users feel more in control and confident while tracking times in the race, we ensured to simplify the user flow and eliminate unnecessary steps (figure 8). There are mainly two phases in the use flow: going to the right split point before the race and timing the racers during the race.

Onboarding page In the onboarding page (race introduction in figure 8), users are welcomed with a message communicating basic information about the race, their role and the split point location. A map with guidance towards this split point will ensure them to reach the split point.

UXAD 2021 | Phase two

Timerdashboard For recording time, we simplify the process and divide the recording time action into “record time” and “assign racer” to fit the different scenarios (figure 9-11). Different modes are available within the timerdashboard, like grid or list mode. These modes are recommended for specific race types and thus meet the needs of the users.

Figure 8 Use flow of timekeeper Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

A flexible system for a wide variety of races Because races can vary greatly in the type of sport, influencing the speed of the racers and the number of racers, the use of a product as such can also vary greatly. Therefore, we intend to provide either one-step timing mode or two-step timing mode within one race according to the number of racers. We also ensured that our redesign can fit in multiple scenarios that may happen during a race. Figure 9 shows the scenario of racers approaching one by one. Figure 10 shows the scenario of racers finishing at the same time. Figure 11 shows the scenario of racers finishing quickly after one another. We provide an easy and quick solution to tackle those different scenarios in two-step timing mode. The detailed use flow of these scenarios can be found at appendix 1.

Figure 9 One by one

Figure 10 Finishing at the same time Team A1 | RaceClocker

Figure 11 Finishing quickly after each other Team A1 | RaceClocker

UXAD 2021 | Phase two

Information architecture We design the timekeeper’s information architecture and make it as simple as possible to let the timekeeper focus on the recording time process. (Figure 12).

Figure 12 Information architecure chart Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

Design Guidelines

User interfaces

A design guideline was created in order to create a smooth and coherent style within our redesign. The design guideline contains two parts: 1) the UI guideline; 2) the component state guideline. The interface with details measurements can be found in appendix 6.

Onboarding When the user opens the link from the race manager, he/she will get directed to the onboarding page on the app. The timekeeper will get welcomed by the app, and get reminded of his/her role in this race.

Figure 13 > UI guidance

UI guidance In the UI guideline, we followed Raceclocker’s original iconic orange to maintain its energetic visual style. (Figure 13) Components To make full use of the screen space and improve the efficiency and accuracy of recording the time, we carefully consider the dimensions and different states of each button (Figure 14).

Tap Continue to get to the page with detailed race information.

Figure 14 > Components

Figure 15 Welcome page Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

Timer dashboard Unfold the information box to see the full map and follow the route to the right place where the timekeeper needs to be (Figure 16).

The current time is shown on the left side. There is also a hint for the user that this area is used for registering the time.

After checking all the information, tap here to get to the next step Selecting the timing mode (Figure 17).

A list of racers is shown on the right side of the screen. In order to help the user find racers efficiently and easily, it is desirable to have more racers fitting on the screen. Therefore, the racers are divided into two columns.

Tap Back to go back to the race details. Based on the number of racers the system will recommend the best timing mode to use (iteration 2). 44

An animation of using the selected timing mode will be shown here as an instruction for the new user.

Figure 18 Timerdashboard

Figure 16 Race details and map

The menu icon enables users to view the race information and switch between Grid mode and List mode (Figure 19). The interfaces showing grid modes can be found in appendix 2.

The recommended timing mode will be listed as the first one. The user is also able to choose another timing mode which he/she prefers. Tap Okay! to get navigated to the Timer dashboard (Figure 18).

Figure 17 Timing mode selection Team A1 | RaceClocker

Figure 19 Side menu Team A1 | RaceClocker

UXAD 2021 | Phase two

If the volume button is pressed again, the new registered time will slide in and turn into the ‘selected’ state automatically. In this way, the user can start with selecting the racer(s) directly (Figure 22).

Press the volume button at the moment that the racer(s) is/are passing by. There is vibration feedback when pressing the volume button. At the same time, the registered time box slides in from the top side of the screend (Figure 20).

The already assigned racers for other times will be grayed out.

Figure 20 Volume button pressed

Figure 22 Volume button pressed and participant greyed out

Tap the right racer to assign that time to him/her (Figure 21). The selected racer is highlighted with the same orange color as the time box, and the same time will appear next to the name of the racer.

Click on the gray racers to jump to the linked time. It can also be the other way around, click on the time box to jump to the assigned racer(s) for that time. The user can easily select and unselect the racers for the time in the highlighted state (Figure 23).

The user can just simply deselect the racer to undo the wrongly assigned racer(s).

Figure 21 Participant selection

Figure 23 Second participant assigned Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

The time part and the racers part can be scrolled separately.

In order to make searching more easily, tap on the eye icon to hide the already assigned racers. Figure 24 Scrolling two sides

After assigning all the racers, there will be a pop up window to confirm the completion and invite the user to the result page.

Figure 25 Hiding passed participants

Figure 26 Results pop-up Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

5 design solutions In our redesign, we intended to solve the problems we identified in phase 1, including the confusing hierarchy, complicated use flow, lack of feedback, lack of clarity in icons and buttons, and difficult operation on the product. Accordingly, they constitute 5 features of our redesign.

Flat Hierarchy

Smooth Use Flow

Timely Feedback

Clear Visual Language

Efficient Operation

We aim to keep the hierarchy of timer dashboard simple and flat by 1) reducing the unnecessary digital buttons and collecting not-often-used features in the menu; 2) making the main interactions happen on the same interface and avoiding unnecessary jumping.

We intend to smoothen the use flow by limiting users’ choices. For example, we recommend timekeepers to choose the timing mode that most suits the race beforehand, which helps to offload the pressure of choosing the best timing mode and also avoid distractions when keeping track of the time during the race.

We use various types of output as the feedback, including the visual feedback (the color changing) and the haptic feedback (the vibration of the phone). We carefully designed our color code to make it intuitive. We also decide the form of feedback according to the level of importance and the context. For example, during the phase of timing, we provide visual and haptic feedback as a double confirmation of certain actions. After timing all participants, a pop-out window will show the timekeepers they have already finish their job.

We standardized our design according to a set of UI design guideline, which includes the size of eachibutton, the color code, the font and font size, and the state changing code. We set up the guideline to make sure each element on the interface is clearly presented, logically laid out at the appropriate location and also meets the aesthetic need.

We got inspired by the use of the stopwatch during timekeeping - a traditional but efficient way of timekeeping without the problem of operation - and decided to use the hard keys of the mobile phone to replicate the familiar feeling of timekeeping with a stopwatch thereby solving the problem of hand-eye discoordination.

Team A1 | RaceClocker

UXAD 2021 | Phase two

4. User testing.

For phase two we mainly focussed on building our prototype and generating our user tests. In this chapter we will elaborate on the user tests. Within the process of setting up our user tests, a pre-pilot and a pilot test have been executed before the official test. In total we tested a pre-pilot with two participants, a pilot with one and the final test with six participants. As for our scope we decided to focus on the timing part. The goal of these user tests is to evaluate our redesign for keeping track of time during the race. The user test is prepared and explained in this chapter.

Team A1 | RaceClocker

UXAD 2021 | Phase two

Purpose of the test

Questions & Criteria

Goal

Constraints

To tackle the current problems of RaceClockers’ timing tool with our redesign, we set up research questions for each problem identified. In this way we could set up a user test to get answers to these questions and evaluate the redesign according to the criteria. Methods to answer these research questions and to check the criteria can be found in section ‘Methods for data analysation’.

As mentioned in chapter 1 we want to make frequent users feel in control when using RaceClocker. Time recording during a race should be intuitive, efficient and fast. Besides that, users should feel confident when using and navigating through the functions RaceClocker offers.

•

• •

•

Since we will simulate the sport race with an animation, the experience of timing will be different from the real world. Influences of external factors, like weather conditions, road users and race audiences, are not involved. The participants may not have any timing experience. We did not interchange the order of tasks within the user tests. This could have an influence on the results as understanding of the product has grown after performing a task. The participants are acquaintances of some of us. This might have an influence on their opinion towards the product, as they might not want to offend us.

Design criteria

Research questions

Sub-criteria

Hierarchy

How do people navigate through the app?

The interface should be simple and clear (C 1.1) Users clearly know in which status of the process they are (C 1.2)

Useflow

How well does the flow of the app match the user’s expectations?

The layout of the interface and the interactions should comply with conventional using habits, which means it should be logical and intuitive for users (C 2.1) The interface system should be flexible and adjusted to the preference of the user (C 2.2) The amount of functions for the timekeepers needs to be decreased (C 2.3) The right information needs to be given at the right time (C 2.4)

Feedback

Is the feedback provided to the user understandable and useful?

User should automatically get instant and correct feedback after each actions (C 3.1)

Information of buttons

Is the first time user able to perform the tasks without errors?

User can finish their task with RaceClocker without wrong attempts (C 4.1) The meaning of the buttons is understandable for users without resorting to external sources (C 4.2) The interface of the website should be easily readable (C4.3)

Operations

How efficient does the user feel when operating on this app?

The product enables users to time the racers without having hand-eye coordination problems (C 5.1) Users are able to undo the wrong

Table 1 Research questions and criteria > Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

Participants & schedule

The pre pilot is conducted with our two coaches Arnold and Marina. After this pre pilot test, we made some changes to the test and did a usage inspection with our group members before the actual pilot test on Thursday may 27th. We recruited six participants for the final user test. We scheduled an hour for each participant on Tuesday June 1st (see table 2).

Test content

Date

What

Time

Participant

Thursday May 20st

Pre Pilot

8:45-10:00

Marina & Arnold

Thursdat May 27th

Usage inspection 9:00-10:00

Group A1

Pilot

11:45-12:45

Suzanne

Test

9:00-10:00

Elke

Test

10:15-11:15

Danielle

Test

11:30-12:30

Britt

Test

13:30-14:30

Sven

Test

14:45-15:45

Sophia

Test

16:00-17:00

Irene

Tuesday June 1st

Table 2 Participant schedule Team A1 | RaceClocker

The test takes about 30/45 minutes per participant. Each participant executes the test on a smartphone enabling optimal use of the prototype created in protopie. The user test knows two races. Race one simulates a small squale rowing race event with ten racers, race two simulates a bigger cycling event with fifty racers. The test was split up in these two race as the amount of racers in a race varies widely and the size of the race has a great effect on the difficulty of tracking times within the race. These race sizes were chosen based on Raceclockers data’s average number of racers in a race and the ‘average’ of a high sample.

The test consists of three parts with a few sub tasks, part one is the rowing race and part two and three are representing a cycling race (Figure 27). The facilitator will guide the participant through the different tasks with a session script (Appendix 3). We are simulating a race that is created by a race manager and the participant will keep track of the time during this race. So the user acts as the timekeeper and uses our redesign prototype of RaceClocker.

Figure 27 Tasks of the user tests > Team A1 | RaceClocker

UXAD 2021 | Phase two

Set-up

Figure 28 illustrates the setup of the user test. We conducted the user tests with all of us, each contributing with a different role. The facilitator is the leader of the user test and executes the test supported with a detailed session script (Appendix 3), she is placed closely next to the participant. Observer FEEL is placed on the other side of the table, diagonally opposite to the participant, to note down how the participant is behaving and what kind of emotions he or she is expressing. Observer DO is observing online, via lookback.io [9] to see how the participant interacts with

the interfaces of the prototype. The remaining two team members are acting as observers SAY and are sitting behind the facilitator to write down what the participant is saying. Furthermore, the participant is asked to perform the tasks on the prototype, a smartphone with a protopie prototype, supported by a race animation on a bigger screen to simulate the context. The whole session is recorded with a camera and with lookback.io as a back up.

Figure 28 Set-up of test and roles > Figure 29 Conducting the user tests

Team A1 | RaceClocker

UXAD 2021 | Phase two

Methods for data collection Various methods were used to gather data during the user tests. In order to gather qualitative data, we used thinking out loud, debriefing interviews and observation of say, do and feel. For qualitative data we used a smeq scale [7] to rate the different tasks and after finishing the tasks a questionnaire with SUS [8] and Attrakdiff [6].

Thinking out loud

SMEQ

Observering

Interview

In advance, the participant is asked by the facilitator to share every thought on their actions during the tasks. The participant is assured that nothing is wrong and is requested to be honest with the feedback.

After each task, we asked the participant to rate the task difficulty on a rating method that has been found to be easy to use: the Subjective Mental Effort Questionnaire (SMEQ) [7]. It consists of a single scale with nine labels from “Not at all hard to do” to “Tremendously hard to do” (Figure 30).

During the user test we will have three types of observers. Observer SAY notes down what the user says during the session and how difficult each task is rated on the SMEQ scale. Observer FEEL notes down how the participant behaves and feels while performing the tasks. Observer DO notes down the steps the user takes within the prototype and how many attempts the user needs to complete the task.

After the tasks are completed, the facilitator asks some debriefing questions to discuss the prototype of RaceClocker, see appendix 4.

Figure 30 SMEQ scale which is used to rate the task difficulty > Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

Attrakdiff

SUS

Attrakdiff [6] is a tool designed to understand how users rate a product or service’s usability and design. We used a part of this existing tool to let the participants rate our redesign of RaceClocker (Figure 31). Using the same scale as the user test of phase one, where we evaluated the current service of RaceClocker, enables us to compare the redesign’s usability to the current RaceClocker.

System Usability Scale (SUS) provides a quick and dirty reliable tool for measuring the usability of a product or service [8]. It consists originally of 10 questions with five response options; from strongly disagree to strongly agree. We tested all 10 questions in the usability test of the current RaceClocker (Report Phase 1), but found out that not all five questions are relevant to test our design criteria. So in order to be effective and efficient, we asked the participants only five questions (Figure 32) that are relevant to our design criteria (see chapter 1).

Figure 31 Attrakdiff template >

Figure 32 SUS template Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

5. Data analysis.

In chapter five, data analysis, we describe how we collected all the data during the user tests, how we analyzed these data of each method and how we translated these into the results for the five problem statements of the current RaceClocker.

Team A1 | RaceClocker

UXAD 2021 | Phase two

Data collection

Results

UXAD 2021 | Phase two

Data results In order to compare and use the data to draw conclusions and findings, we visualised and presented the data in graphics. We will describe the data gathered per method.

Observation As described within chapter four, user testing, we use qualitative and quantitative methods, Figure 33 shows an overview. For each qualitative method (do, feel and say notes, interview and open questions in the questionnaire) quotes and notes were gathered in a table per task and participant [5]. After gathering all quotes we captured these quotes and notes into understandable summarized themes.

During each task, four team members took notes in different roles. All these notes were combined in a big table in Miro to draw conclusions from [5]. Observation notes, do say feel, showed us that when users have to time a lot of racers, stress is experienced. Especially when all racers arrive at the same time and close to one another. It was very clear that the ease of use increased as more experience was gained.

All of our quantitative data, such as attrakdiff, SUS, SMEQ and task completion, was gathered in a google form and visualized in graphics. These interpretations and graphics were again clustered and summarized per design criteria. Figure 33 Data collection during the test Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

SMEQ

In figure 34 the rating of each task with the SMEQ scale [7] is shown. The small orange dots represent the individual SMEQ scale rating of each participant, the big orange dots show the average of all six participants for each task. From the SMEQ rate 30 or higher, it has the label ‘a bit hard to do’ and a few tasks are on or above this rating. Task 1.1, 1.2, 1.3, 2.2, 2.4, 3.1, 3.2 and 3.3 are tasks in which the participants had trouble with.

Figure 34 Average rating of each task (big orange) with the individual ratings of each participant (small orange) > Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

Task completion

In figure 35 the completion of each task is visualized. Some participants didn’t succeed in completing tasks 1.2, 1.3, 2.4, 3.1, 3.2 and 3.3. Furthermore, task 1.1, 1.4, 2.1 and 2.2 needed multiple attempts to fulfill the job. Comparing the task completion (Figure 35) and SMEQ scale (Figure 34) to each other, they show a similar pattern. The SMEQ scale could act as an overlay on the task completion graphic and show the same flow.

Figure 35 Task completion from each participant > Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

Interview results The interview notes where gathered, clustered and analyzed in the Miro board [5] Some interesting summarizing insights and conclusions are shown on this page.

Interview after test asks Navigation within the interface was experienced as clear. Most time users knew where to click, however, operation and navigation did get easier over time. Scrolling within a long racer list took a lot of time, which made operation of the system more difficult.

The interview shows that the little amount of functions creates a simple and clear interface. Also the color contrast makes it clear for the user where to click. The volume button and the order of timing (first timing then assigning) was not always understood directly. Furthermore, a long list of racers made assigning harder. The feedback provided while timing was experienced as comfortable and supporting. Users noted that they would prefer to receive more feedback as they select a racer. The feedback when finishing timing was experienced as satisfying. However, users would need some more feedback and instructions within the app on how to operate, as some options were not clear directly. For example the hide - eye button was not understood right away. However it was clear to users that they could scroll within the list of racers.

The scrolling took too long. - Elke

That was really clear, it was all clear and not too much information. ... You do not want too many options. You already have a lot to do. - Elke

It was nice. Just clear and not too much information. Just the information that you need. - Danielle

Timing was experienced as stressful, as there sometimes were a lot of racers in a race and approaching at the same time. The welcoming page was experienced as valuable guidance and gave a personal touch. Grid mode was evaluated as most useful for most users. As this allows for less scrolling in case of big racer lists and shows a greater overview. On the other hand, the list mode was mentioned to be more appealing and suitable for small-scale races, as it also allows for a more personal touch as names are shown. Most often landscape mode would be prefered, as this is experienced as more intuitive regarding pressing the volume button. Portrait mode would perhaps allow for a more comfortable reach to all racer numbers.

It was difficult sometimes as you had to do a lot of things at the same time. - Irene

That was all clear, yes I think it felt personal to see your own name. - Elke

I like it because it is less scrolling. - Irene

With the volume button I think landscape. - Sophia

Reasoning SUS questionnaire The product is experienced as easy to use and understand, when using it for a while usage gets easier. The (physical) buttons needed some learning, but usage gets easier after a few uses. The app was reviewed as consistent and straight forward. Stress is experienced as multiple racers approach close after each other or at the same time. However confidence within this will grow over time. Feedback helped the user while operating but could be more direct, for example an extra buzz when pressing a racer. Overall the application is experienced as easy to use. In stressful situations the ease of use reduces. The interface is clear and clean, and ease of use improves over time.

I did not get a notification of feedback when I pressed the number. - Sven

Team A1 | RaceClocker

If you practice only once, I think it is easy to understand and use. - Irene

I needed to learn which buttons to use in the application, so I had to learn something but not much. - Sven

The explanation of the app was clear, except for the part that I could press multiple times when multiple participants finished shortly after each other. - Danielle

It is easy to understand how to use it. However, it is difficult and stressfull when a lot of participants are approaching at the same time. - Elke Team A1 | RaceClocker

UXAD 2021 | Phase two

Attrakdiff The results of the Attrakdiff [6] questionnaire can be found in figure 36, where the grey dots represent the average ratings of the six participants in the user tests of the current RaceClocker website. The orange dots are the average rating of our redesign by the six participants of the user test.

Figure 37 shows the spread of the Attrakdiff score of the redesigned RaceClocker timer. The spread of unpredictable-predictable, cheappremium and unimaginative-creative shows a wider range compared to the other scales in this figure, while other dimensions clearly show obvious improvements. If we compare this graphic to Figure 36, we can see that the improvement in these three scales is not as significant as it is in the other scales.

The graph shows that on each level, the redesign is rated higher than the current RaceClocker. The biggest differences can be seen in between the confusing-clearly structured scale and the tacky-stylish scale. The slightest difference can be noticed in the cheap-expensive scale.

Figure 36 Results of Attrakdiff of both current RaceClocker (grey) and our redesign (orange) >

< Figure 37 Attrakdiff results showing the average (dark orange) and the spread of the participants rating (light orange) Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

SUS In figure 38 the averages of the System Usability Scale, as known as SUS [8], of both the current (grey) and redesigned (orange) RaceClocker are plotted. In all five statements, we see strong improvements in comparison to the current RaceClocker. The participants could imagine using this tool if they needed to track time during a race, found it easy to use and were confident when using the product. The system was really consistent and they had less to learn during the use of the application in comparison to the current RaceClocker.

Figure 39 shows the average rating for each SUS statement. No unexpected divergent ratings have been given.

Figure 38 Results of SUS questions from the current RaceClocker (grey) and the redesigned RaceClocker (orange).

Figure 39 SUS results showing the average (dark blue) and the spread of the participants rating (orange). Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

Hierarchy CRITERIA

Data interpretation

CONCLUSION

Hierarchy

As we gathered all data within our Miro board we could start clustering and interpreting our data towards each criteria. Per criteria some space was created for strengths and weaknesses. For each method we clustered our results within the criteria it belonged to, together with a reference to the method used to gather this insight. These detailed tables can be found in appendix 5, the main conclusions are written down in this section.

Team A1 | RaceClocker

Hierarchy

C1.1 The interface should be simple and clear.

C1.2 Users clearly know in which status of the process they are.

How do people navigate through the app?

Interviews showed that the navigation within the interface is described as simple and clear. The user understands where to click. Besides, the SMEQ results (Figure 34) showed that the use of the product gets easier over time. Attrakdiff (Figure 36) also showed that simple and clearly structured aspects of the redesign are rated significantly higher than the original version of RaceClocker. Therefore, it can be concluded that this criteria is fulfilled.

This criteria is unfortunately not fulfilled as expected. During the interview, users said that they did not always know what the next page would be.

Navigating through the app is easy and clear, but some participants still want to know what pages they can expect next. For example, instead of having “I’m ready” on the button we can change it to “Ready to time”, so that the user can expect the timer dashboard in the next screen.

Team A1 | RaceClocker

UXAD 2021 | Phase two

Use flow CRITERIA

CONCLUSION

Use flow

C 2.1 The layout of the interfaces and the interactions should comply with conventional using habits, which means it should be logical and intuitive for users.

The SUS and Attrakdiff results showed that the application was easy to use in general. In comparison with the SUS results of the original RaceClocker, it is obvious that the scores increased significantly. However, not everything within the app is intuitive. Some detailed interactions, such as the process of assigning two racers arriving at the same time, is not intuitive enough right away. Two out of the six participants clicked the volume button twice for registering time, because they thought that one time can only belong to one racer. Moreover, for inexperienced timekeepers the sequence of timing is not always understood directly. We designed the timing function in a way similar to using a stopwatch where the user can register multiple times in a row and then assign them to corresponding racers. This will be very useful in the situation when multiple racers are quickly approaching after each other. However, all of the participants thought that timing can only be done in the order of “first time then assign” because of the lack of appropriate instructions. In stressful situations, they do what they are used to. Besides, functions such as undo and reassign show a learning curve, but are very easy to learn (SUS). Team A1 | RaceClocker

Use flow

C 2.2 The interface system should be flexible and adjusted to the preference of the user.

C 2.3 The amount of functions for the timekeepers needs to be decreased.

C 2.4 The right information needs to be given at the right time.

How well does the flow of the app match the user’s expectations?

During the interview, participants said that they want to switch between landscape and portrait mode based on the number of racers. Landscape grid mode is the most suitable one for a race with a large number of racers. In our app, users can switch the timing mode in the setting page, and change the screen mode by rotating the phone.

The interview showed how the decrease of functions improves the structure of the app. “It is really nicely structured. Not too many options”, said Sophia.

The welcome page provides users with useful information before the race starts. During the interview it has been said that the welcoming screen gave a personal touch. “The second page I got was very clear. I saw the places on the map and how long the race would be.”, said Danielle Navigating to the result page has been rated as not at all hard to do, this is because the result window only pops out at the moment when all the racers have been timed.

Actions such as assigning multiple racers at one time are not intuitive for the user, because the way of timing is hard to know without instructions . Therefore, aclear guidance before first time using is necessary.

Team A1 | RaceClocker

UXAD 2021 | Phase two

Feedback

Information of buttons CRITERIA Feedback

UXAD 2021 | Phase two

CONCLUSION

CRITERIA

Feedback

CONCLUSION

Information of buttons

C 3.1 Users should automatically get instant and correct feedback after each actions.

Is the feedback provided to the user understandable and useful?

C 4.1 Users can finish their task with RaceClocker without wrong attempts.

C 4.2 The meaning of the buttons is understandable for users without resorting to external sources.

C 4.3 The interface of the website should be easily readable.

Is the first time user able to perform the tasks without errors?

Vibration feedback assures the users. It gives a reassuring feeling when the volume button is pressed. Also the color contrast between timed and untimed racers helps within the process of tracking times. The pop-out window after finishing timing all racers makes users feel satisfied.

The provided feedback is valuable, however, more feedback for assigning racers is needed. Since the haptic feedback for both the volume button and selecting racers might lead to confusion, we will add some textual feedback after assigning the racer successfully.

The task completion table (Figure 35) showed that there are six uncompleted tasks. They are all about the timing functions. One cause could be that the instruction page is not clear enough. Participants cannot understand the process of timing thoroughly.

According to the participant, the buttons in our app are similar to other apps. It was clear for them where to press. The scrolling bar on the timer dashboard makes it clear that the list is scrollable.

During the user test, none of the participants had problems with reading. The color contrast of the button clearly indicates the state of the racer.

The first time user is able to perform at least 30% of the tasks without errors. The most errors are caused by the misunderstood way of timing and operating the system. A learning curve is shown in performing difficult tasks. Therefore, we can conclude that our app is easy to learn.

On the other hand, some participants (3 out of 7) think that the feedback for assigning racers is not enough. They want to have haptic feedback for it as well. Besides, they also want to have stronger visual feedback to show the connections between the time and corresponding racers. Sophia: I need more confirmation to make sure it really has been selected. Elke: I do not know which time is connected to which name.

Team A1 | RaceClocker

However, the “eye” button is not clear enough for the participants (3/6). It gets clear after trying this function. Besides, inexperienced timekeepers do not understand the terms such as the start type of the race.

Team A1 | RaceClocker

UXAD 2021 | Phase two

Operations CRITERIA

CONCLUSION

Operations

C 5.1 The product enables users to time the participants without having hand-eye coordination problems.

C 5.2 Users are able to undo the wrong action easily.

How efficient does the user feel when operating on this app?

Participants still have a lot of things to do at the same time. They need to look at the coming racers, find the racer in the list in advance, and then press the time button. Most difficulty is encountered during scrolling and finding the right racer. When multiple racers are approaching, remembering the numbers is hard.

There are different opinions on undoing. The participants’ opinions on the function to undo vary. The undo function has been integrated into the list of participants. It can be done by deselecting the participant. This interaction feels natural, but at the same time it has a high risk of misoperation. Sophia: I am a little afraid I will press it again accidentally.

Actions that are performed with this app are natural and easy, however, there might be a high risk of misoperation. The racers might be selected or deselected accidentally and sometimes the volume button is hard to press because of little press force.

Some participants like using the volume button, because timing by pressing the volume button reminds them of the feeling of using a stopwatch. They think that our app is a good alternative for the stopwatch. Instead of writing down names and times, the user can click the racer in the app to assign time. There were still some participants who had problems with the physical buttons. Two participants missed a couple of times because they didn’t press the volume button hard enough. One participant pressed the on/off button accidentally during timing.

Team A1 | RaceClocker

UXAD 2021 | Phase two

6. Recommendations.

We defined a few recommendations and changes to our redesign after analyzing and processing all of the data from the usability tests.

Team A1 | RaceClocker

UXAD 2021 | Phase two

Quick Fixes and Interfaces The onboarding page is modified to better guide the timekeeper to select the most suitable timing mode. The timing mode is preselected. Timekeepers can also click to override it. The recommendation page shows the reason why a certain mode is recommended.

Team A1 | RaceClocker

UXAD 2021 | Phase two

The quick scrolling bar is added to assist timekeepers to quickly find the participant when the participant list is long. The scrolling bar can be numeric (for bib/bow numbers) or alphabetic (for names).

We add the result button at the top right corner. Timekeepers can check the result anytime.

When the timekeepers assign a time to a participant for the first time, there will be a confirmation text ‘successfully assigned!’ appearing below the selected participant. It will disappear automatically after 3 seconds. The text will only show up at the first time. We suggest adding a digital timing button. This can be activated in the setting. It is designed for those cases where the volume button can’t work or timekeepers still need a digital button as an assistive button.

We add number indicators below the time buttons. Timekeepers can have a quick overview of participants who have been assigned to the corresponding time. Team A1 | RaceClocker

Team A1 | RaceClocker

UXAD 2021 | Phase two

Future recommendations that need testing Beginner Guidance We suggest adding a brief guidance for the first time users. But how the guidance should look like still needs testing.

Two-step Keypad Mode

Team A1 | RaceClocker

[1/5]

[2/5]

[4/5]

[5/5]

[3/5]

Suggested Guidance Contents [1/5] Press the volume button to record the time. [2/5] Click multiple times to record several times in a row. Newly recorded time will appear here. [3/5] Select the time and assign the racer(s) to it. [4/5] Click on the racer you want to assign the time to. [5/5] You can assign multiple racers to a same recorded time.

Team A1 | RaceClocker

We suggest integrating the third submode inside the two-step mode - a two-step keypad mode. It will solve the pain point of finding the racer by directly typing in the number.

Photo Backup We suggested adding a new feature of taking a photo as a backup. It will solve the problem of forgetting which racers belong to what time.

UXAD 2021 | Phase two

References. 94

Team A1 | RaceClocker

[1]

Official website RaceClocker https://www.raceclocker.com

[2]

Miro board 1 of team A1 https://miro.com/app/board/o9J_lUyEEbA=/

[3]

Miro board 2 of team A1 https://miro.com/app/board/o9J_lQDFPtw=/v

[4]

Miro board 3 of team A1 https://miro.com/app/board/o9J_lJRkX1E=/

[5]

Miro board 4 of team A1 https://miro.com/app/board/o9J_lARKlks=/

[6]

Attrakdiff http://attrakdiff.de/index-en.html

[7]

SMEQ scale https://measuringu.com/papers/Sauro_Dumas_CHI2009.pdf

[8]

SUS https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html

[9]

Lookback.IO https://lookback.io

Team A1 | RaceClocker

UXAD 2021 | Phase two

Appendices. Appendix 1 Detailed useflow of multiple scenarios.

Team A1 | RaceClocker

UXAD 2021 | Phase two

Appendix 2 Interfaces GRID mode

Team A1 | RaceClocker

UXAD 2021 | Phase two

Task 1 | Delft Rowing Race (10 participants) The test consists of three parts and I will guide you through it. We are simulating a race that is created by a race manager and you will keep track of the time during this race. So you are the timekeeper. We will first start with a rowing race in Delft. Let’s start with part one.

Appendix 3 Session script for user test

100

TO DO list before the test: • • • •

Print consent form, race details and smeq scale Open website/phone on the first page Open questionnaire Set up lookback.io for online observation

Session introduction Thank you for agreeing to take part in our research study. My name is [....], and they are my team members who will take notes. I first would like to ask you to read and sign this consent form before we start with the test.

101

During the rest of the session, I’ll be working from a script to ensure that my instructions to all participants are similar. We are very eager to get some feedback on the redesign of RaceClocker for race managers and timekeepers. I am an independent researcher and we are very curious to find out exactly what you think. So please be honest in your feedback. During the session I will ask you to perform several tasks. As you perform these tasks, please try to do whatever you would do in normal circumstances. We encourage you to think out loud while performing the tasks. Just tell me whatever is going through your mind. Please know that we are not testing you and there is no such thing as a wrong answer. We ask you to perform the tasks in order to understand what works, and what does not work within this system. After each task you will be asked to rate the difficulty of it on a scale. The whole session will take about 30-45 minutes. We will record the whole test via zoom as a back-up analysis since we will all take notes. Do you have any questions before we begin?

Team A1 | RaceClocker

UXAD 2021 | Phase two

Task 2 | Cycling round (50 participants) We will now let you partcipipate in another race, a big scale cycling round. This race knows more race-participants, however we will not ask you to time all racers. We will just focus on small bits and will guide you within each bit. Let’s start with part two.

UXAD 2021 | Phase two

Imagine that you are now present at a different split point within this same race, the finish. Let’s start with part 3.

102

103

Team A1 | RaceClocker

UXAD 2021 | Phase two

Appendix 4 Interview questions

Interview questions after the test 1. How did you experience navigating through the app in general? • Was it easy or difficult? What was easy?

UXAD 2021 | Phase two

Appendix 5 On the next four pages, the Miro tables for data intepretation are showned.

2. To what extent did the system fit your preferences and needs? • Were all needed functions available? • How did you consider the information presented in the app?

104

3. Did you receive feedback when using the app? • And how did you feel about this? • Do you feel that you have used the app correctly?

105

4. Did you understand all the buttons in the interfaces? • What was not clear? • Could you find a needed button easily? 5. How did you experience tracking the time? • How did it make you feel? 6. What do you think about the beginner’s guidance? • Is it clear? • Is it necessary? 7. Which mode do you prefer (list/grid) for timekeeping? • Why? 8. In which direction would you like to hold your phone during timekeeping? • Why?

Team A1 | RaceClocker

UXAD 2021 | Phase two

Strengths

106

107

Team A1 | RaceClocker

UXAD 2021 | Phase two

Weaknesses

108

109

Team A1 | RaceClocker

UXAD 2021 | Phase two

Appendix 6 Interfaces with detailed measurements.

110

Team A1 | RaceClocker