I
d e a
Wo
r l d
Fi
t n e s s
Co
n v e n t i o n
Conference website usability study plan
2016
The team for Spartan UX Researchers
Divya Narayan divyanarayan24@gmail.com
Animesh Tripathi animeshtripathi@gmail.com
Michael Chambers mjchambersjr@gmail.com
Jennifer Quan Jenni.Quan@gmail.com
Gala Nikolaeva litavtor@gmail.com
Table of Content
Introduction 4 Product Description 5 The Objective 6 Study Design 11 Participants 13 Usability Testing Facility 16 Measures 20 Specific Tasks 24 Session Checklist 47 Procedure 49 Findings by Protocol 56 Task Success and Failure Rate 98 Heuristic Evaluation 102 Global Usability Issues 102 Local Usability Issues 116
Scripts Appendix Consent form Participant Screener Recruitment Ads Fake log-on info
137 170
170 171 172 175
4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Introduction
The interface analysis team at Spartan UX was contacted by IDEA Health and Fitness Associ-
ation to conduct a usability study of their 2016 IDEA World Fitness Convention website. As 2016 is projected to be the biggest year for the IDEA World Fitness Convention, they wanted a professional evaluation of the website before the conference takes place to ensure that the attendees can easily sign up for the conference, provide payment, find accommodations and select their sessions within the conference website. It is essential to facilitate proper user experience when interacting with a conference website, as it is the first exposure to the conference that the user encounters. As IDEA World Fitness Convention is an internationally renowned fitness conference that is extremely popular among fitness professionals, it is imperative that the website interactions are seamless, as it affects how the users perceive the company IDEA Health and Fitness Association as a whole. Spartan UX will take a scientific approach to usability testing, using principles from engineering psychology and human factors to assist IDEA in creating a conference website that revolves around user centered design.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 5
Product Description IDEA Health and Fitness Association is an organization for professionals within the realm of fitness and wellness, ranging from personal trainers to group fitness instructors to nutritionists. The goal of this company is to provide a large breadth of informational resources that professionals can utilize in order to influence and motivate millions of health-minded individuals worldwide. IDEA Health and Fitness Association puts out journals, newsletters, educational materials to obtain CECS for fitness certifications, as well as multiple conferences throughout the year. IDEA prides itself on being the most visited fitness professional website in the world.
One of the biggest conferences that IDEA puts on yearly is the IDEA World Fitness Conven-
tion. Fitness professionals from around the world attend the conference in order to get informed about the latest research within the realm of fitness, to get certified in different training modalities, to experience different styles of workouts, to network with industry professionals and to obtain CECs. Additionally, IDEA World Fitness Convention has a free expo that runs in conjunction with the conference. This expo displays the latest training equipment, fitness technology and nutritional supplements for the public to purchase at a discounted rate. Conferences like IDEA World are important, as they inspire fitness professionals to consistently hone their craft, and expand their repertoire. Additionally, professional conferences and expos foster a sense of camaraderie throughout the field, which in turn motivates professionals when they return to their job.
6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Executive Summary
As a team, we began by familiarizing ourselves with the website, specifying typical goals
and tasks for target users. We then identified potential local and global usability issues through a heuristic evaluation. Upon observing users interact with the website, we confirmed the specific usability issues that occur upon interaction, consequently impeding successful completion of the aforementioned goals and tasks. Following an analysis of the data that was compiled during user testing, redesign recommendations were provided according to principles from interaction design and engineering psychology.
Upon watching users interact with the IDEA World Conference website during usability testing,
their inability to carry out several key tasks elucidated problem areas within the website interface, which interfere with achieving the perfect conference experience. Since conference websites are not visited as frequently as other websites, ensuring its intuitiveness is imperative to simplify interactions. Moreover, since a large sum of money is typically spent on attending a conference, guaranteeing that the users have no doubts or questions about what they sign up for is also essential. These problems will be elaborated upon in the paragraphs below, and throughout the report.
Participants in the usability study cited unanimously that one of the main sources of difficulty
stems from the “Session Selection Tool�, which left users frustrated and unable to plan their schedule. Viewing a list of presentations that are pertinent to individual interest and planning out a proposed schedule beforehand is crucial for a streamlined conference experience. A poorly designed session selection tool can contribute to attendees being unaware of the breadth of sessions offered, as well as feeling overwhelmed while parsing through each individual session to find the few that are relevant to their interest. Bad design is especially apparent when users have difficulty in interacting with a component after reading the instructions, which are shown below. Our metrics showed that 60% of users failed in creating a session schedule for Friday, indicating the necessity to redesign the session selection tool to make it more user friendly.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 7
Another problem area lies in the fact that the website lends itself towards experienced
attendees rather than first time attendees. While there is a video assigned for first time attendees, this does not assist the user in navigating through the poorly explained website with confusing user interface or educate the user about the different components of the conference. For example, many new attendees could not discern the difference between the IDEA World Fitness Convention and the IDEA World Club & Studio when registering for the conference (as shown below). As they are making a large monetary investment when checking these options, they should feel reassured that they are making the right choice relative to their needs and interests. Similarly, many first time attendees were unable to discern the difference between the free Expo pass and the conference.
8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
While seasoned IDEA World attendees would understand how the conference works based
on their past experiences, the website needs to do a better job in conveying this information to first time attendees so they feel confident that they are making informed choices. Providing more explanations of the different conference components is recommended in the redesign, to facilitate a greater understanding for both new and intermittent users.
Finally, another significant usability problem that many users commented on is the shear
breadth of information provided on the website, and the lack of salience for the prominent features that are most relevant to attending a conference. Subsequently, the user has to spend excess time on the website to complete tasks. The homepage itself has more information than the user can possibly process, thus causing an overwhelming feeling of confusion (as shown below). Performance measures such as time to complete task during the usability test bolster this claim. For example, one participant took about four minutes to even find the date and location of the conference (one of the most basic tasks), citing that it was difficult because “it’s too much stuff, bam-bam advertising to my face, I didn’t know where to look”. The poor use of screen real-estate coupled with the lack of minimalist design creates a busy feel. This is not made easier by the fact that banner blindness exists, which can cause the user to miss out on pertinent information, such as early bird specials. Therefore, a redesign of the homepage which consists of minimalist design, salience of prominent and relevant features is recommended
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 9
The users rated their overall impression of the websites usability as mediocre (a 3 on a scale
from 1-5). While the user’s perception of a conference website is seemingly trivial when compared to their impression of the actual conference experience, the user’s ratings of the IDEA World website should be strongly taken into consideration. Ensuring proper website usability is imperative, especially when taking into consideration that IDEA World is currently the premier fitness event for professionals. In order to maintain this status, the conference website should mirror the excellence that is presented at the event. To have a subpar conference website could theoretically deter potential attendees, or steer them in the direction of other fitness conferences with a more appealing, usable website, such as the up and coming conference NASM Optima (website shown below). If the data presented and the recommendations are taken seriously, we can ensure that IDEA World will remain the premier conference within the realm of fitness, providing superior services from the start of registration, to the final claps at the post conference session.
1 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
The Objective
The objective of this usability test is to analyze the efficacy of the IDEA World Fitness
Convention homepage. It is imperative that the conference webpage is usable, as efficient interaction is technically the beginning of the users’ experience in attending the IDEA World Fitness Convention. Their ease in registering for the conference, finding accommodations, and choosing their schedule all affect their overall impression of their conference experience. Additionally, ensuring a usable website could increase attendees, as they will easily be able to find everything that the conference has to offer. The 2016 IDEA World Fitness Convention webpage will be analyzed, and tested among users and potential users before the conference takes place in Los Angeles in July. A heuristic evaluation will be performed to initially familiarize the team with the website and identify any usability issues that hinder proper user experience. Then 24 participants who are students, fitness professionals, volunteers or exhibitors will be taken through a usability test, where they interact with the website at the Spartan UX lab, performing certain tasks which coincide with their specific goals. Half of these participants will be new to interacting with the IDEA World Fitness Convention website, and the other half have already interacted with the website in previous years. Then, potential redesign recommendations will be given to resolve any usability issues. In executing a usability study of the 2016 IDEA World Fitness Convention, the website’s effectiveness and efficiency, along with user satisfaction will be enhanced. This ensures that the IDEA World Fitness Convention will remain the premier fitness conference that professionals choose to attend.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 1
Study Design STUDY DESIGN OVERVIEW
The study is designed to look at overall usability of the website. The website aims to it educates
and enables the visitors to learn about the conference, register and attend the conference. In this research we aim to find the main challenges to achieve these tasks. The factors which were considered when designing the study are outlined below. The usability study was conducted in a controlled environment, with 20 participants from different categories. According to the client, the primary focus is to ensure the ease of use when registering and paying for the days that the user plans to attend the conference.
Primary Goals
• Get information about the conference
• Buy tickets/pass
• Plan a visit
Secondary Goals
1. Pricing
1 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
2. Cheapest
3. Travel (nation & international)
4. Accommodation
5. Volunteering Opportunities
Metrics: Three measure would be considered when conducting this study. PERFORMANCE: This measure would help the researchers understand participant’s performance on specific tasks and give numerical measurement. This measurement would be collected via the recording taken from the screen recording software.
• Time on task
• Accuracy
SUBJECTIVE: This measure would be derived via their response on subjective measure after tasks, or through revisiting the specific segment of the session on video and asking what they were going through something specific.
• Understanding purpose of the site
• Successfully or unsuccessfully completing tasks,
• Navigation
• What the site makes them feel
BEHAVIORAL: These measure are exhibited behavioral cues that help researchers understand user mental model and to help correlated the displayed behavior to actions performed.
• Successful Task Completion: Each scenario requires the participant to obtain specific data that would be used in a typical task. • Critical Errors: Critical errors are deviations at completion from the targets of the scenario. If the participant chooses to • Non-Critical Errors: Non-critical errors are errors that are recovered by the participant and do not result in the participant’s ability to successfully complete the task • Error-Free Rate: Error-free rate is the percentage of test participants who complete the task without any errors (critical or non-critical errors).
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 3
Participants Participants
In conducting a usability study, one of the main goals is to enhance user satisfaction when
interacting with the website interface. In studying the user satisfaction when interacting with a product or website, it is essential to determine who the users are, and what their goals are. After outlining specific user groups who comprise the typical user base of a website, they are then recruited from the general population to be a part of the usability study, and given a screener to determine eligibility for the study based on inclusionary and exclusionary criteria, which can be found in appendix 2A. This ensures that the efficacy of the website is determined by the typical (and potential) users of the website, as they are well acquainted with the ideas and goals which are needed when interacting with the website. For the IDEA World Fitness Convention website, the test participants are outlined into fitness professionals, companies who are exhibitors, students, and volunteers. These groups are further divided by experience level with the conference website in the past, either being a novice or experienced user. Descriptions of each category are further outlined below.
Participant Categories
Dependent on the participant category, different tasks were given that were relevant to the
users typical interaction with the IDEA World Fitness Convention website. These tasks were generated while keeping in mind the groups specific goals and requirements from the website. The protocol sheet that was created outlines the tasks which each specific participant group completed and can
t 1 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
be found in appendix 1A.
Fitness Professionals:
The fitness professionals include personal trainers, group fitness instructors, studio owners
and other wellness professionals who wish to either attend the conference or the free expo. Typical interactions with the site include finding the free expo pass, paying for the conference, selecting sessions to attend, viewing CEC values for each course, finding accommodations while staying in Los Angeles.
Exhibitors:
Exhibitors include companies who are looking to showcase their products during the IDEA
World Fitness Expo, which runs in conjunction with the IDEA World Fitness Convention. They use the website in order to get more information about what the expo entails and how to become an exhibitor at the next IDEA World Fitness Expo.
Students:
Students include people who are currently enrolled in a university and are interested in
pursuing the profession of fitness, usually with a major like Kinesiology, Recreation, Sports Science, or Sports Medicine. When students attend conferences, they generally like to view opportunities that are unique to them, as they can network with students from other universities, and gain more insight from professionals as to what it takes to enter the profession. IDEA Fitness offers an IDEA World Discovery College Program, which is exclusive to students who will be attending the IDEA World Fitness Convention. Student attendees should be able to successfully learn about the IDEA World Discovery College Program upon looking at the conference website, as well as sign up for the program.
Volunteers:
At many conferences, some potential attendees wonder if there are any volunteer opportunities.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 5
They usually volunteer a portion of their time to help ensure the conference runs smoothly, and in return can attend the conference at a cheaper rate, or ideally for free. IDEA World Fitness Conven-
Participant Type
Experience Level (with the confer- Number of Participants ence website)
Fitness Professional
Novice
6
Fitness Professional
Experienced
6
Exhibitors
Novice
2
Exhibitors
Experienced
2
Students
Novice
2
Students
Experienced
2
Volunteers
Novice
2
Volunteers
Experienced
2 Total 24
Who should attend according to Idea Fit
1 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Facilities Enterance The reception and waiting area will include a receptionist desk facing the entrance. The desk will have a single computer for the receptionist and two tablets for participants to sign in and fill out consent forms. There is a sofa, several chairs, and a coffee table covered with magazines in the participant waiting area. There is a water cooler, a coffee pot, disposable cups, a trash can, and an assortment of unhealthy snacks near the sofa.
Waiting and signing in
The door to the side of the receptionist’s desk leads to a hallway. The first door in the hallway
leads to the client room. The second door leads to the testing room. The last door leads to the control room.
The client room is a small room with a large one way mirror that allows observers to look into
the testing room. In front of the one way mirror is a large table that spans the width of the mirror. On top of the desk, to one side, is a large monitor, a small speaker, a jar of pencils and pens, and a small stack of copy paper. Three comfortable office chairs are in front of the desk. Two more chairs are against the wall, also facing the mirror. At the far wall, adjacent to the mirror, is a table with a coffee dispenser and a tray of unhealthy snacks. Beneath the table is a wastebasket and a
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 7
mini fridge filled with water, juice, and soda.
The testing room has one way mirrors on the two walls adjacent to the door. Video cameras
are set into the ceiling rails, allowing them fairly free movement around the room. A speaker is set into a corner of the ceiling. A three framed landscape paintings are on the walls. The walls are a soft cream color and the ceiling and its rails are a dark brown. The floor is tiled but there is a neutral patterned rug on the floor. A book shelf filled with boring looking books is against the wall, next to the door. A floor lamp is to the side of the bookshelf. A wooden office desk with an adjustable keyboard tray is against the far wall. An adjustable leather office chair is behind the desk. The 20 inch anti-glare computer monitor with web cam is on the desk 24 inches away from the front edge. The monitor shows a windows desktop with a single internet browser icon. The keyboard, mouse, and mouse pad are on the keyboard tray. The CPU is beneath the desk, away from the participant’s kicking radius. On the desk, to the side of the monitor, is a framed picture of a cute animal, a coffee mug, and a pencil holder with two pens. The computer has software that will capture every-
1 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
The control room has two desks facing the one way mirror. One desk has multiple computer
monitors and video viewing and controlling equipment. The monitors show the information from the eye tracker, the web cam, the ceiling cameras, a real time transcript of everything picked up by a the microphone in the test room, and a mirror of the computer in the testing room. The speakers play audio from the microphones in the testing room. There is also a microphone that allows the controller to talk to the people in the testing room. The second desk also has a computer but its input controls are mainly for data logging. There are two office chairs behind the desks. Behind the chairs and desks is a cabinet with equipment replacements, office supplies, a tool box, and a first aid kit.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 9
2 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Measures During this study, three types of measures were examined, including: • Performance • Subjective • Behavioral
Performance Measures Performance measures are used to analyze participants’ behaviors as they perform specific implicit and explicit task on the IDEA Fit conference website. This data is then quantified and used to measure time and clicks while performing task and goals within the site. In order to measure the performance of these tasks, several methods were used as well as recorded during and after the usability test. Participants were observed in-person and performance was recorded for data collection. The details are as follows: • Time on task • The total amount of time it takes to complete a task • Accuracy • Were they able to complete the task? • If not, what path was chosen? • Did they perform the task correctly? • If not, where did they falter?
Subjective Measures During and after the study, specific questions and probes were directed at the participants. after the study questions will be asked of the participants. The goal of these questions are to ascertain the participants’ thoughts and feelings in regard to the IDEA Fit conference website. These measures are an important method in order to understand whether the site felt intuitive, created value, and is of course easily used. Some questions asked include:
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 2 1
•
Did the participant understand the purpose of the site?
•
Did the participant understand the tasks and what they were expected to do?
•
For nascent users, what were their initial impressions of the site?
•
Did they perceive this as a site where they find all the information necessary for the IDEA
Fit Conference?
•
Were the participants successfully able to complete the tasks we asked them to do?
•
Did they know where to start with the interface to get to what they need?
•
Did the participants feel that the site guides them in completing the tasks?
•
Did it take them a long time to perform or find a function?
•
Do they seem frustrated during this process?
•
Did the participants become more comfortable with the site the longer they used it?
•
Did it become easier to navigate?
Behavioral Measures Behavioral measures help describe a participant's’ thoughts, and feelings through both verbal and nonverbal communication. In non verbal communications such as facial expressions and gestures, a large amount of information about the participants reaction towards the site can be measured. More often than not, participants’ do not communicate effectively or over analyze while explaining. Participants were observed live on-site, while also being recorded for review later. Some behavioral measures include:
•
Verbal comments made by the participants’ during the study
•
Time participants spent reading explanations on the screen
•
Locating key information when needed
•
Confidence in their ability to complete each task
•
Nonverbal behavior showing hesitation, confusion, happiness, sadness, etc.
•
Did the participant feel that they could perform each task independently, or did they look
for help in completion of the tasks?
2 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Tasks
The evaluation of this usability test is based on several key findings, tasks and measures that
were evaluated relative to the various participant groups. There was a list of general sample tasks that each participant completed during the study. Each task represents key interactions and items that most users, in specific user groups, would want to accomplish while using this site.
Initial Feelings
Participants were shown the homepage of the IDEA Fit conference website and asked to
share their initial feelings and insights. Nascent users are encouraged to share their initial feelings while more experienced users will share their thoughts and perceptions. Participants were also asked to show their most common tasks that they perform while on the IDEA Fit conference site. This provided us with certain insights into expectations and the overall value proposition for the use of the website. In most happenstance if participants are too far removed from what the site offers, then they usually become confused, irritated, and dejected. These tasks are measure by analyzing their verbal responses and body language. All verbal and nonverbal communication are recorded and analyzed during and after the study.
Navigation
All participants are to be given tasks that require them to navigate the website to help deter-
mine how intuitive and effective the flow and navigation schema of the website is. Some specific tasks are considered common and most users would wish to accomplish these on a regular basis. We will then collect data on how well participants were able to navigate through the website while completing various task. This is important because navigation needs to be seamless in order to accommodate a positive user experience. Users are more likely to abandon a poorly made website than to muddle through it.. Navigation will be measured based on a participant's’ ability to successfully navigate through tasks within a reasonable amount of time. A successful navigation is not only
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 2 3
defined as completing the task within a certain time frame but also without becoming flustered and dejected.
Task Completion
Very similar to navigation, we measure how well the user accomplish a given task. This includes
confidence, moods, as well as the pathing used to perform the given tasks.
Emotions
During the test, researchers will be observing the participants as they complete their goals
and tasks for signs of frustration, confusion, joy, and dejectedness. Although many participants often struggle pride and other systems interfere with the ability for the participants’ to verbally express these issues. This often leads in missed information or the report of false information. By monitoring facial expressions and the seemingly minute sighs, gasps, and grunts we can get an appropriate estimation of the emotions user feel from the site itself. These expressions give the researchers a window into the moment-by-moment feelings that were not confounded by more visible emotions (e.g. a participant feeling extremely accomplished at the end of completing an unnecessarily difficult task). All comments and facial expressions are to be recorded and analyzed during and after the study.
Survey
Participants are to be asked a short series of questions prior to the start of the study and
again at the end of the study in order to assess their thoughts and feeling sin regards to the IDEA Fit conference page. The pre-survey provides us researchers with a better understand of the participant’s web habits, their immediate perception of the IDEA Fit Conference site and allow us to recognize changes in their feelings toward the site at the end. The post survey questions give us feedback on how their time interacting with the site can change their perception and whether or not a positive or negative user experience is portrayed. The surveys will help us researchers to better understand the needs, wants, and aspirations the IDEA Fit users want with the site. This allows us to suggest optimal redesigns that are specifically tailored to improve the future user experience. All participants’ answers are collected and recorded via verbal feedback. All answers are to be recorded and
2 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
analyzed during and after each study and again after the entire study is done.
Specific User Tasks
Detailed below are examples of tasks from each protocol sheet with detailed steps for
completing each task. These tasks represented common tasks that our users would want to accomplish using the site and what we believed would provide more detailed information about how users interact with the site. The steps to accomplish the tasks are detailed here, but each provided several different ways to accomplish the same task. We observed the path each user attempted in order to understand any difficulties users might face using the IDEA Fit conference website. Each task tested the different knowledge each user type brought to this study and was well suited for measuring the intuitiveness of the site. Please find out if being exhibitor give you any privileges on the conference? (esp.: tickets discounts, free passes, other extras).
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 2 5
2 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 1B: Accommodation task (goal level ) Find appropriate accommodations (according to your preference) while attending the conference.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 2 7
Task 1C: Planning your visit (goal level ) Workshop must be from any of next categories: PT techniques Group exercise Workshop must be with any CEC certification
2 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 2 9
3 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 2A: Workshops categories . Filter . (task level ) Let’s now imagine you want to attend certain workshops, could you please find the day and time to attend at least one of workshop from next categories: Nutrition Group exercise
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 3 1
3 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 2B: Exhibitor ’s benefits (task level ) Please find out if being exhibitor give you any privileges on the conference? (esp.: tickets discounts, free passes, other extras).
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 3 3
Task 2C: “Session Selection ” tool (task level ) Using Session Selection tool, please, create a schedule for day N, so you will attend one workshop in the
3 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 3 5
Task 2D: Particular trainer (task level ) Let’s assume that you are big fan of personal trainer Benjamin Degenhardt and want to attend all workshops/ lectures/talks/meetings that he gives on conference.
3 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 3 7
Task 2D: Programs , student discounts (task level ) There are a few options, discount and special student’s program on the site. Please, try to find all of them,
3 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 3 9
Task 3A: Certification (task level ) Let’s now imagine that your goal is renew or get AFAA CEC certification, can you find the way to do it during the conference?
4 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 3B: Free pass (task level ) Please find all available options for attending the conference for free
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 4 1
Task 3C: Volunteering (task level ) One of the options to attend conference for free is to become a volunteer, please find what should you do to become one.
4 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 4 3
Task 3D: Membership (task level ) There are two types of prices on the site: for nonmembers and for IDEA members. Please find out how to
4 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 4 5
4 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 5A: How to become an exhibitor (task level ) As a member of a small business that specializes in fitness equipment, you would like to exhibit at the Expo which takes place in conjunction with the conference. Please find out how you can get more information about exhibiting at the expo.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 4 7
Session Checklist To maintain a consistent procedure across all of the participants, a checklist was created which consisted of all the tasks that the team had to do before the participants arrived, as well as during and post-session.
Before the Participant Arrives • Call the participant and confirm where he is and if he is arriving on time • Reserve a parking spot in the building for the participant • Make sure the computer is running well • Are the cameras ready to record? • Make sure the computers used to collect data are running • Open the web browser and go to IDEAFit conference website. • Test that the website is working • Go back to the main page • Clear the cookies • Confirm the team is using the right protocol • Prepare the task sheets for the participant • Get the consent form and a pen ready • Prepare the pre-survey • Prepare the task’s package • Get water, snacks, and other things the participant may need • Prepare participant compensation • Team meeting • Is everyone ready to start?
4 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Once the Participant Arrives • Greet the participant and Introduce yourself and the team • Does the participant have any questions? • Give the participant a tour of the facilities (bathroom, water, etc.) • Is participant seated comfortably? • Remind the participant to silence any mobile devices • Present the consent form to the participant • Did the participant sign the consent form? • Collect the consent form • Pre-survey the participant • Is the pre-survey completed? • Give the task cards to the participant • Follow the tasks in the protocol • Give the post-survey to the participant • Is the post-survey completed? • Debrief the participant • Pay the participant • Dismiss the participant
Once the Participant Has Been Dismissed • Team meeting • Are the video/picture files correctly labeled? • Are the video/pictures saved?
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 4 9
Our Procedure Participants Recruitment 6 participants were recruited through personal connections and contacts. Due to time and budget constraints, no previous attendees of IDEA World Fitness Convention were found. Instead, 6 users who were either fitness enthusiasts, attendees of other conferences, as well as those who have never attended a conference were chosen.
5 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Type This study tested users who had no experience using the Idea Fit website and were asked to assume few roles to facilitate the upcoming tasks in the session. For example, some participants were asked to imagine if they were wanted to be an exhibitor at the conference and wanted to make business arrangement with IdeaFit. While others were asked to assume they would want to volunteer for the event to reduced the overall conference costs.
Participant Type Student
Professional
Experience
# of
Novice
2
Expert
2
Novice
1
Expert
1
Total: 6
Payment Our participants participated in the study voluntarily and were not paid.
Facility & Equipment This study was conducted in Human Factors and Ergonomics Lab using 2 cameras, screen recording and face cam. The screenrecorder was recorded locally on the they testing laptop and also used in to record in conjunction with the two external video feeds, to help sync action to body language.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 5 1
Procedure The following is the sequence of events that occurred when the six participants took part in this study.
5 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Procedure Prior to starting the study, the participants were given an informed consent form to read, sign, and date. This form explained an overview of the study, listed any potential risks or benefits, and instructions for which the participant can follow should they have any questions. In addition, participants were given an initial session introduction.
The moderator then explained the general process of the study to the participant.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 5 3
The moderator then began to proceed through the study protocol and provided task sheets to the participant.
All participants were run through the tasks step by step.
5 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
During each session, the data logger recorded participant interactions, verbal responses, and ratings following each trial in real time.
At the end of the session each participant was thanked for their participation and offered some treats.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 5 5
<Blank on purpose>
5 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Findings by Protocol Study Observation and Findings The following sections include findings, observations, and verbal comments made by participants during the testing sessions. Each task is listed in sequential order of the protocol. Performance, subjective, and behavioral data is listed with each task. We noted any steps that caused confusion or frustration below. Here are our findings and observations.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 5 7
Task 1- Conference location and dates. Participants were asked, â&#x20AC;&#x153;Using the website, find out when and where the conference takes place.â&#x20AC;?
5 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
When asked if it was difficult to find the information. Even though this participant had found the location and date on the front page, he was unable to discern the fact that EXPO general information was different from the conference information.
Participants were asked, “How difficult would
Participants were asked, “How satisfied are you
you rate it on a scale of 1-5?”
with the result on a scale of 1-5?”
Rating
Rating
Participants N = 6
Easy (4-5)
5
Satisfied (4-5)
4
Neutral (3)
1
Neutral (3)
2
Difficult (1-2)
Participants N = 6
Not Satisfied (1-2)
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 5 9
Observations Since a user’s rating upon task completion does not necessarily match their actual experience, it is important to use other metrics, measurements and probes in order to fully understand the user's thought process and impression after completing a task.
Participant’s Verbal Comments during task: Moderator: “Using the website, find out when and where the conference takes place.?” P3: “very easy” P5: “looking for a map, couldn't find it for awhile” P6: “It’s obviously in LA… There’s way too much stuff… O, there it is!…”
Moderator: “How do you feel about this process?” P2: “Looking for a map, couldn’t find it for a while” P6: “It took me a while to figure out where it was. Because it’s too much stuff, bam-bam, advertising to my face, I didn’t know where to look”.
Task Completion Did they finish the task
Participants N = 6
Yes
4
No
2
6 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 2 - Conference tickets and accommodations. Participants were asked, “Choose the tickets and accommodations as though you are planning on attending the conference this year.” Participants were asked, “How difficult would you rate it on a scale of 1-5?”
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 6 1
Participants were asked, “How difficult would
Participants were asked, “How satisfied are you
you rate it on a scale of 1-5?”
with the result on a scale of 1-5?”
Rating
Participants N = 4
Rating
Participants N = 4
Easy (4-5)
2
Satisfied (4-5)
2
Neutral (3)
2
Neutral (3)
2
Difficult (1-2)
Not Satisfied (1-2)
Observations Participants took time to understand where and how the information was spread across the website. Even though finding information was seemingly easy for some of the participants, it was more difficult for those who were instructed to keep a budget in mind. There was a general lack of visual hierarchy that was missing in the website, thus making it difficult for users to discriminate based on price. With the same information present over and over again with a different layout, the user latched onto the way that they were first exposed to. This lead to incomplete information, since the user latched onto finding information in an incomplete way, since they did not want to take the cognitive effort to look at the same information twice.. This makes sense as the user here here on a mission to the tickets as quickly as possible, presenting the same information again and again with different layouts would confuse them.
Did they finish the task? Yes No
Participants N = 2 2
6 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Participant’s Verbal Comments during task: Moderator: “Using the website, choose the tickets and accommodations as though you are planning on attending the conference this year.”
P1: “It was really weird… there are two options and I have no idea how those are related” P4: “Registration is pretty okay. [Only] rich people would look into add-on, that [is] too much info” P6: “Tickets are right here… And accommodation… ” Moderator: “What were your search criteria?”
P1: “I was just looking for something that says registering or days or price” P2: “I want to attend the event except for the extra stuff...” P3: “I’m a busy guy. I’ll only go Saturday.” Moderator: “Did you notice any discounts or deals?”
P2: “Yes” P3: “What discounts?... You said there was a deal but I didn’t see a deal.” Moderator: “Using this scale how difficult was finding this information?” P2: “It was quite easy” P3: “Mostly easy”
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 6 3
Task 3 - Cheapest 3-day pass. Participants were asked, â&#x20AC;&#x153;Find the cheapest option for a 3-day pass.â&#x20AC;?
6 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Participants were asked, “How difficult would you
Participants were asked, “How satisfied are
rate it on a scale of 1-5?”
you with the result on a scale of 1-5?”
Rating
Participants N = 5
Rating
Participants N = 5
Easy (4-5)
2
Satisfied (4-5)
2
Neutral (3)
1
Neutral (3)
1
Difficult (1-2)
2
Not Satisfied (1-2)
2
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 6 5
Observations Similar to previous issues, the lack of visual hierarchy and/or mismatch of the mental model and Similar to the previous issues, the lack of visual hierarchy and the mismatch of the mental model and the design of the UI caused the users to take a long time, or left them unable to discriminate the information they were looking for. . This is highlight in the verbal remarks they made during
Participant’s Verbal Comments during task : Moderator: “Using the website, find the cheapest option for a three-day pass.” P2: “Does it exist?” P3: “I only see one option” P4: “ I felt cheated because I could have saved $50 had I known that becoming an Idea members gets you a discount. I would have made my plans accordingly during such an event.” P4: “ Since I am on a budget I was going to look for the cheapest hotel they mentioned, and then Google search which one was cheaper than that, I was also hoping to find information to cab fares from all this hotel locations”
Moderator: “Using this scale how difficult was finding this information?” P2: “It says pricing there, [that] is where I thought it should be but it wasn’t clear” P3: “very easy” P4: “Finding the information is easy, it’s pretty standard, but I was missing information when I made the decision to buy my tickets”
Did they finish the task?
Participants N = 5
Yes
2
No
3
6 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 4 - Free Expo ticket. Participants were asked, “There is a free Expo ticket available on the site. Find a free 1-day Expo ticket.”
Participants were asked, “How difficult would you Participants were asked, “How satisfied are you with the result on a scale of 1-5?”
rate it on a scale of 1-5?” Rating Easy (4-5)
Participants N = 2 2
Rating Satisfied (4-5)
Neutral (3)
Neutral (3)
Difficult (1-2)
Not Satisfied (1-2)
Participants N = 2 2
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 6 7
Observations The Expo passes were clearly visible but there was a misunderstanding of the word “Expo”. Participants assumed that are going to a conference and attending an Expo for “X” number of days, and somehow they it meant that they could get a free expo pass for one of these days. As participant number 3 points out “What’s the difference between the expo pass and the conference?”. However since participants would clearly see the menu option of “Expo” they would easily gravitate towards the page and find the information. This confusion, however, would be compounded after they were introduced to the concept of Exhibit or Exhibitor. Participants confused the session exhibits like Yoga or Pilates with exhibits like “latest ab roller”.
Participant’s Verbal Comments during task : Moderator: “There is a free expo ticket available on the site. Please find it.” P2: “It was easily found.” P3: “What’s the difference between the expo pass and the conference?”
Moderator: “How do you feel about this process?” P3: “Maybe clarify the difference between the expo and the conference”
Completion Rates Did they finish the task?
Participants N = 4
Yes
4
No
0
6 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 5 - Volunteer. Participants were asked, â&#x20AC;&#x153;One of the options to attend the conference for free is to become a volunteer. Find out how to become a volunteer.â&#x20AC;?
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 6 9
Participants were asked, “How difficult would you Participants were asked, “How satisfied are you with the result on a scale of 1-5?”
rate it on a scale of 1-5?” Rating
Participants N = 2
Easy (4-5) Neutral (3)
Rating
Participants N = 2
Satisfied (4-5) 2
Difficult (1-2)
Neutral (3)
2
Not Satisfied (1-2)
Observations In our initial test, it was very difficult and annoying to find information on volunteering. However, by the time of our test, the company decided to put up an easy access infographic at the “seven ways to save” section.” [Screenshot above]. The participants who were asked to find opportunities with respect to going and attending the conference for free were having a hard time finding the information. Since students are a potential target audience for the conference, obscuring information jeopardizes the acquisition of a future customer.
Participant’s Verbal Comments during Task 1: Moderator: “One of the options to attend the conference for free is to become a volunteer. Find out how to become a volunteer.” P2: “Couldn’t find the option to become a volunteer but found information about it” P3: “Am I supposed to know you can get into the conference for free?” After searching in FAQ: “No one gives a fuck about the FAQ!”
Moderator: “Using this scale how difficult was finding this information?” P1: “Should make it obvious you can volunteer”
7 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 6 - Membership. Participants were asked, â&#x20AC;&#x153;Ticket prices are cheaper if you are an IDEA Fit member. Find out how to become an IDEA member (find the registration form).â&#x20AC;?
Observations Participants had to find the information regarding becoming a member using Google, instead of using any internal links. This is crucial issue with regards to helping people become a member and brand with the organization. The cognitive load for an average user increases as the user has to look up the conference membership, then discriminate whether the conference they are referring to is the same as the one google has pulled up. Since they are on a different design language page they feel hesitant even trusting the information presented. We had to stop the participants from wasting too much time trying to verify information, and inform them that they are indeed on the correct page and this is how they would have to register if they fail to find the link. At that point they would let out a nervous laughter and inform the moderator about their disbelief. This would be a major concern for the organization, as it is setting up the foundation of mistrust with visual cues. One of the participants chose to Google the conference reviews on online forums to check if the conference he was going to attend legitimate. Completion Rates
Did they finish the task?
Participants N = 6
Yes
2
No
4
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 7 1
7 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Participant’s Verbal Comments during task: Moderator: “Find out how to become an IDEAFit member” P1:“I would google it” P2:“not possible...” P3: “Oh I saw that!” P6: “I remember seeing something about becoming a member, but now I can’t find it.”
Moderator: “Using this scale how difficult was finding this information?” P1: “easy with google… hard on site” P2: “it’s not possible on site..” P3: “ easy only because I saw it earlier”
Moderator: “How do you feel about this process?” P1: “Once I used google it was amazingly fast but trying to do it on the website was very difficult” P2: “I would just use google to become an IDEAFit member”
Participants were asked, “How difficult would you
Participants were asked, “How satisfied are
rate it on a scale of 1-5?”
you with the result on a scale of 1-5?”
Rating
Participants N = 6
Rating
Easy (4-5)
1
Satisfied (4-5)
Neutral (3)
1
Neutral (3)
Difficult (1-2)
4
Not Satisfied (1-2)
Participants N = 6 2
4
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 7 3
Task 6b - Student Discounts. Participants were asked, “You’re a student and there are always student discounts for conferences. Find out what student discounts are available.”
Participants were asked, “How difficult would you
Participants were asked, “How satisfied are you
rate it on a scale of 1-5?”
with the result on a scale of 1-5?” Rating
Rating
Participants N = 4
Easy (4-5)
N=4
Satisfied (4-5) Neutral (3)
Neutral (3)
1
Difficult (1-2)
3
Not Satisfied (1-2)
4
7 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Observations After observing the participants’ behavior, it was clear that they had no idea that student discounts were available. Finding this information, which changed based on membership, amongst all the overloaded pages was difficult. There is no clear feedback in the session selection or checkout pages that would inform the participant how to access student discounts or even that these discounts are available.
Participant’s Verbal Comments during task: Moderator: “Find out what student discounts are available” P1: “Not possible to find student discount” P6: “O that is how I can become a member! That’s all I can find so far. (Points to the FAQ section)
Moderator: “How do you feel about this process?” P6: “ I wouldn’t know <by myself> that here is a student discount.”
Completion Rates
Did they finish the task?
Participants N = 4
Yes
1
No
3
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 7 5
Task 6c - Exhibitor. Participants were asked, “You’re a small business owner and you want to be an exhibitor at the conference. Find out how to do so.”
Participants were asked, “How difficult would
Participants were asked, “How satisfied are you
you rate it on a scale of 1-5?”
with the result on a scale of 1-5?”
Rating
Participants N = 2
Easy (4-5) Neutral (3) Difficult (1-2)
Rating
Participants N = 2
Satisfied (4-5) 2
Neutral (3) Not Satisfied (1-2)
2
7 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Observations The two participants who played the role of exhibitors for the study were thoroughly unsatisfied with how the information was presented. They felt that if they wanted their business interests to be valued, then they would have to make a personal connection with the organization to get better treatment.
Participant’s Verbal Comments during task: Moderator: “Find out how to be an exhibitor at the conference”
P1: “Looks like you have to call or email to do that” P3: “All they say is sign up… all they give you is a form, they should give more information”
Moderator: “Do you think you have enough information on being an exhibitor?” P1: “Nope” P3: “Of course not”
Moderator: “Using this scale how difficult was finding this information?” P1: “All they give you is a form, they should give more information” P2: “I want a little more information”
Moderator: “Do you feel safe sharing your information on this page?” P2: If I as an individual business owner had to put information on this page I would not cause I do not know where this information would land. But if I was part of bigger organization I wouldn’t care. Completion? Yes No
Participants N = 2 2
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 7 7
Task 7 - Interesting workshop. Participants were asked, “What fitness activity are you interested in? Find a workshop on it.”
Participants were asked, “How difficult would
Participants were asked, “How satisfied are you
you rate it on a scale of 1-5?”
with the result on a scale of 1-5?”
Rating
Participants N = 6
Rating
Participants N = 6
Easy (4-5)
4
Satisfied (4-5)
1
Neutral (3)
2
Neutral (3)
3
Not Satisfied 1-2)
2
Difficult (1-2)
7 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Observations There were multiple issues when we asked participants to find an interesting workshop, or predetermined workshop in this tool. Some participants had hard time find the filter capabilities in the session by presenter. Almost all participants found that the information was too dense, consequently leading them to avoid looking for any details. They did this throughout the website, as they knew they would be navigating the same problem again and again with the session selection tool. One version of the task setup was to urge the user to scan multiple pages to find the key information that they were asked to look for. This led to frustration by the end of the task.
Participant’s Verbal Comments during task: Moderator: “What fitness activity are you interested in? Find a workshop on it.”
P1: “Holy crap this it’s painful. There are more than 9 pages and you have to go through them one by one, the navigation is not exactly useful.. Not very clear how it’s organized. This one is a little bit less fun to use.” P2: “Looking in session info. Should be included in session handout, but wasn’t found” P3: “Oh that’s horrendous really… oh jesus...now I see why you’re doing this [study]... are they out of their minds?... 21 different things from just 7-7:45, no one will read these things. This is awful…. Oh, you have to highlight it.”
Moderator: “Using this scale how difficult was finding this information?” P2: “Most of the terms used are not normal to the people” P3: “It is fucking terrible.”
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 7 9
Completion Rate Completion?
Participants N = 6
Yes
5
No
1
8 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 8 - Workshop on nutrition. Participants were asked, “Find a workshop on nutrition.”
Participants were asked, “How difficult would
Participants were asked, “How satisfied are you
you rate it on a scale of 1-5?”
with the result on a scale of 1-5?”
Rating
Participants N = 6
Rating
Easy (4-5)
5
Satisfied (4-5)
Neutral (3)
1
Neutral (3)
Difficult (1-2)
Not Satisfied (1-2)
Participants N = 6 6
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 8 1
Observations Having used the tool once, participants had no issues finding the information. However, they did point out that it was information dense and that it looked as though there were only lectures.
Participant’s Verbal Comments during Task Moderator: “Find a workshop on nutrition.”
P2: “there we go, that wasn’t so bad.” P6: “Are they only lectures, or there are other types too? ” Moderator: “How do you feel about this process?”
P1: “I first thought that session info would be too general so I overlooked it” P6: ”I went to the wrong place first. It would be nice to be able to filter by lecture”
Completion Rates Completion?
Participants N = 6
Yes
5
No
1
8 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 9 - Favorite Trainer. Participants were asked, â&#x20AC;&#x153;You are a big fan of personal trainer Benjamin Degenhardt. You want to attend every lecture/ workshop/meetings he presents. Can you find all information on him and his sessions?â&#x20AC;?
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 8 3
Participants were asked, “How difficult would you
Participants were asked, “How satisfied are
rate it on a scale of 1-5?”
you with the result on a scale of 1-5?”
Rating
Participants N = 4
Rating
Participants N = 4
Easy (4-5)
3
Satisfied (4-5)
3
Neutral (3)
1
Neutral (3)
1
Difficult (1-2)
Not Satisfied (1-2)
Observations This task was easy for the participants. However, when they used the filter, they verbalized that they wished the information would be presented with a higher same value. This meant that the presenter bio be attached to the same filtered info instead of having to move to another page and look for the information all over again.
Participant’s Verbal Comments during Task Moderator: “You are a big fan of personal trainer Benjamin Degenhardt. You want to attend all his lectures and sessions. Can you find all the information on him and his sessions?”
P2: “A sort by alphabet would be nice” P6: “What? That’s annoying. Ok. That’s these two then. ” Moderator: “Using this scale how difficult was finding this information?” P1: “It’s pretty easy”
Moderator: “Do you feel like you’ve found all information on him and his sessions?”
P1: “ I guess so…” P2: “Yes, because they list all his stuff here”
8 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Completion Rates Did they finish the task?
Participants N = 6
Yes
5
No
1
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 8 5
Task 10 – Timetable Participants were asked, “You know that you want to attend two workshops on Friday. Create a non-conflicting schedule with two workshops you would enjoy.”
8 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Participants were asked, “How difficult would you Participants were asked, “How satisfied are you rate it on a scale of 1-5?” Rating
with the result on a scale of 1-5?” Participants N = 5
Rating
Participants N = 6
Easy (4-5)
3
Satisfied (4-5)
1
Neutral (3)
1
Neutral (3)
3
Difficult (1-2)
1
Not Satisfied (1-2)
1
Observations Using the session info tool the participants were asked to created a timetable for themselves. The maximum time went in scanning and finding the task. This meant that eventually they would give up on what they were looking for. For those who chose to stick and find something interesting would be irritated on having to redo the task when using the session selection tool.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 8 7
Participant’s Verbal Comments during task: Moderator: “You know you want to attend two workshops on Friday. Create a non-conflicting schedule with two workshops you would enjoy.”
P3: “Why can’t I actually select it? Am I supposed to be able to select it? I didn’t realize those actually did anything.” P6: “O, it’s time blocks. Ah? I thought there was overlapping stuff.... Ah, they are!” Moderator: “Using this scale how difficult was finding this information?” P6: “It wasn’t obvious that time blocks were there. Blocks were……. All over. If it would be obvious it would be way easier. Once I figured out time blocks it made it way easier”
Completion Rates Completion?
Participants N = 6
Yes
2
No
3
8 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 11 - Session selection tool. Participants were asked, “Using the Session Selection tool, recreate the same schedule for Friday.”
Participants were asked, “How difficult would you Participants were asked, “How satisfied are you rate it on a scale of 1-5?” Rating Easy (4-5)
with the result on a scale of 1-5?” Participants N = 4 2
Neutral (3) Difficult (1-2)
4
Rating
Participants N = 4
Satisfied (4-5)
1
Neutral (3)
1
Not Satisfied (1-2)
4
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 8 9
Observations This was amongst the most confusing segment of our task list. All participants chose to ignore the instructions on the page. They were unsuccessful at achieving any of the task requirements.. The readability was an issue, the information was not visually represented. There was no feedback or dynamicity to the interaction. When we revisted the task with them, and informed them that had they read the instruction there would have a login feature that allows them to do things more easily, to which they responded, “That makes sense but I would have never arrived to that conclusion.”
Participant’s Verbal Comments during task: Moderator: “Use the section selection tool to select workshops.” P2: “It’s all blends together, and it is hard to find a particular thing without scrolling all the way through it. It would be nice if I can select everything that I am interested in and add it to a queue or something. And then go and check a queue.” P3: “Is that any other tools so that you can pick different sessions? Or it is only this list?” P6: “You need to login, which I am not going to do.”
Moderator: “How do you feel about this process?” P6: “ For the tool I went back to the menu, it should be a link to the tool on the session page. If you wouldn’t mention there is a tool, I wouldn’t have no idea there is a tool.” P6: [about instructions] “It was clear, but I do not like how it looks.”
9 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Completion Rates Did they finish the task?
Participants N = 4
Yes No
6
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 9 1
Task 12 - Certification. Participants were asked, “Find out if you can get certification points by attending a workshop. (Continuing Education Credit (CEC))”
9 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Participants were asked, “How difficult would you
Participants were asked, “How satisfied are you
rate it on a scale of 1-5?”
with the result on a scale of 1-5?”
Rating
Participants N = 5
Rating
Participants N = 5
Easy (4-5)
2
Satisfied (4-5)
2
Neutral (3)
1
Neutral (3)
2
Difficult (1-2)
2
Not Satisfied (1-2)
1
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 9 3
Observations The readability of the CEC chart was the biggest issue on this task. The second concern is that because this is a very specific task, technical industry terms should be understood.. Then be aware of what their itinerary is and be able to cross refer to the table presented. This is an excruciating experience for Mr. Scamalot, and he was very upset to use this information.
Participant’s Verbal Comments during task: Moderator: “Find out if you can get certification points by attending a workshop”
P1: “Oh my eyes! … would have been easier to find if it wasn’t in a pdf!” P3: “I don’t know… it’s not very clear where it is… nothing… Whoa! Holy... fuck... ok.” P6: Went to the FAQ page “I am not sure…all classes have a certification?” Moderator: “Using this scale how difficult was finding this information?” P3: “No, I can’t read it… you need the Hubble Telescope”
Completion Rates Did they finish the task?
Participants N = 5
Yes
3
No
2
9 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Final Questions Moderator: Now that you have finished all of the tasks, I’d like to ask you some final questions.
Q1. How satisfied are you with navigating through this site on a scale of 1-5? Rating
Participants N = 6
Satisfied (4-5)
1
Neutral (3)
3
Not Satisfied (1-2)
2
P1: “It’s not the worst I’d ever dealt with. It’s very far from the best.” P2: “The information was misleading and I’m not sure what you’re asking is actually available” P3: “Registering was easy” P6: “So much is going on and it’s attracting me, but all stuff that I need to know just blended into background. Like on homepage, so much going on there, so much text and big pictures and you think each of this things is important, but it doesn’t take me what I want. And a lot of stuff that I want is hidden under the menu. Like the session tool on the session menu. It would be nice to have session as it’s own big thing.”
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 9 5
Q2. How can the site change to improve your satisfaction rating?
P1: “Make the agenda, make the scheduling a lot more easier to see at a flance. Group by day and time that would make it easier… there was no way to get to section selector from the agenda it was kinda lame.” P2: “An information page that has a better explanation” P3: “Certs need to be apparent. They can speed up the website make things more open” P4: “Only menu is dynamic, looking for more feedback and help. Give me offers when I stay and look! … I’m not a valued visitor’ P6: “ Take stuff away! It is too distracting! They need to prioritize! They have all info there, but they need to prioritize, what do they want? Do they want people to sign in? Do they want to advertise what they do? Why people come here? What the first thing they want them. Instead of put all the things out.”
Q3. Overall, how difficult was it to navigate the site on a scale of 1-5? Rating
Participants N = 6
Easy (4-5)
1
Neutral (3)
2
Difficult (1-2)
3
9 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Q4.In comparison to other conference websites, how did you feel this website stacks up? P1: “At Def Con conference they made it so the agenda was labeled clearly so you can see any event on each day and time. It made scheduling a breeze” P2: “Lots of information. Very guided” P3: “Worse! It was very easy on other [conference sites], they also had a map”
Q5. What was your favorite part of the website, if any?
P1: “the registration page is pretty straightforward all one page” P2: “Nothing in particular’ P3: “Don’t have one’ P4: “Hotel and registration’ P6: “Time block, I really like it. I wish they made them more obvious”.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 9 7
Q6. What was your least favorite part of the website, if any? P1: “Certification chart and the agenda” P2: “The landing page should be minimal and talk to the point… too much information’ P3: “the certification page’ P4: “Hardest was section selection… go to tab for different days’ P6: “ Navigation.”
Q5. Was there anything in the conference site that you thought was missing?
P1: “More info on expo page without having to contact some person”
Q7. Overall, how did you feel about the site? P1: “yeah, pretty satisfied. Most of them, google was there to help me” P2: “It’s good enough if you’re already familiar but something that is more guided would be better.’ P3: “make info easier to find… Expo makes it seem hush hush….session info with 10 pages is bullshit. P4: “Pretty standard besite. So much information, it’s too hard P6: “If it is conference site… most people want to go to the conference… 75-95% information should be about conference and registering.”
9 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task Success & Tasks (success / fail)
P1
You are considering attending the IDEAFit conference for the first time. Using the website, find out when and where the conference takes place?
S
Register as though you want to attend the general conference for all three days.
S
Choose accommodations (hotel) as though you are planning to attend the conference this year.
Find the cheapest option for a 3-day pass. Ticket prices are cheaper if you are an IDEAFit member. Find out how to become and IDEA member (find the registration form). Youâ&#x20AC;&#x2122;re a student, and there are always student discounts for conferences.
N/A
F
F
Find out what student discounts are available.
F
Find a workshop on nutrition.
S
You are a big fan of personal trainer Benjamin Degenhardt. You want to attend every lecture/ workshop/meetings he presents. Can you find all information on him and his sessions?
S
You know that you want to attend two workshops on Friday. Create a non-conflicting schedule with two workshops you would enjoy.
F
Using the Session Selection tool, recreate the same schedule for Friday.
F
Find out if you can get certification points by attending a workshop.
S
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 9 9
& Failure Rate P2
P3
P4
P5
P6
Failure %
F
S
S
S
F
33.33%
S
S
S
F
S
16%
S
NA
S
NA
NA
-
F
S
F
N/A
S
60%
S
F
S
F
F
50%
S
N/A
F
N/A
F
75%
S
S
F
S
S
16%
S
S
F
N/A
S
20%
N/A
S
F
F
S
60%
F
N/A
F
F
F
100%
S
F
F
S
S
33.33%
1 0 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task Satisfication (S) and Tasks (success / fail)
S
D
S
D
You are considering attending the IDEAFit conference for the first time. Using the website, find out when and where the conference takes place?
4
5
3
5
Choose the tickets as though you are planning on attending the conference this year.
3
5
5
5
N/A
N/A
3
3
Find the cheapest option for a 3-day pass.
2
2
1
1
Ticket prices are cheaper if you are an IDEAFit member. Find out how to become and IDEA member (find the registration form). Youâ&#x20AC;&#x2122;re a student, and there are always student discounts for conferences.
1
1
1
2
Find out what student discounts are available.
2
3
1
2
Find a workshop on nutrition.
4
5
5
5
You are a big fan of personal trainer Benjamin Degenhardt. You want to attend every lecture/ workshop/meetings he presents. Can you find all information on him and his sessions?
4
3
5
5
You know that you want to attend two workshops on Friday. Create a non-conflicting schedule with two workshops you would enjoy.
3
5
N/A
N/A
Using the Session Selection tool, recreate the same schedule for Friday.
1
1
3
4
Find out if you can get certification points by attending a workshop.
3
2
5
3
Find accommodations for when you attend the conference this year.
N
N
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 0 1
d Difficulty (D) Rating S
D
S
D
S
D
S
D
Avg
Avg
5
5
5
5
5
5
3
3
4.17
4.67
4
4
4
5
3
2
5
4
4.00
4.17
N/A
N/A
3
3
5
5
5
4
4
3.75
5
5
3
3
N/A
N/A
4
4
3.00
3.00
5
5
5
3
1
2
1
1
2.33
2.33
1
1
N/A
N/A
1
1
1.25
1.75
5
5
4
5
4
5
4
4
4.33
4.83
5
5
N/A
N/A
N/A
N/A
4
4
4.50
4.2
5
5
3
3
2
3
4
0
3.00
3.80
N/A
N/A
1
1
4
4
0
0
2.25
2.50
1
1
N/A
N/A
3
4
5
5
3.40
3.00
1 0 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Heuristic Evaluation Several known usability and engineering psychology principles were violated by the IDEA Fit conference website. Below are the Global and Local usability issues and our suggestions on how to overcome them.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 0 3
Global Usability Issues
Global usability issues are a representation of systemic problems within a website’s interface. Serious violations of usability on a system wide scale often pose numerous issues which designers must resolve in order to provide an optimal experience for the user. The global usability principles that were located on the IDEA Fit conference website are as follows: • Visibility • Consistency • Feedback & Feed-forward • Navigation
1 0 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Visibility Visibility refers to “whether or not a user can perceive all the available features of a product up front”. In other words, a product with high visibility provides visual cues so users can understand what options are available to them. This principle is the most commonly violated heuristic in interface design and the Idea Fit Conference website was no exception.
Participant expression when trying to find info in obscure text
The actual Footer for the Idea Fit website is the one listed above, but randomly throughout the footer disappears. This doesn’t allow access to many features present on the page.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 0 5
Impact: The continual violation randomly throughout the website only confuses the user. Not only have you not provided them with useful tools to complete their task, this violation coupled with the fact it appears on some pages but not on others hurts the user experience.
Recommendation: Keep the footer on all pages or provide a way to access this information upfront.
1 0 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Consistency Consistency is the level to which familiar interfaces, features, and other aspects a user has previously experienced continue across a product.
Throughout the entire Idea Fit Conference website whether it be navigation, sidebars, or links there is a huge amount of inconsistency between what can be clicked and what cannot.
Impact: With an already crippling slow website, inconsistent links will frustrate the user creating an almost unbearable experience.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 0 7
Recommendation: Based on your frequency matrix of what you want the users to see and do. Make the un-clickable tabs in your navigation point to the most appealing one.
Impact: A non consistent footer that contains useful and in many cases the only way to access that information is detrimental to the usability of the site.
Recommendation: Keep the footer consistent on every page. A footer should be an artifact on a website and never change throughout the entire site.
1 0 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Three completely different font sizes, fonts, colors and schemes are used throughout the IdeaFit Conference
Impact: The inconsistency makes the conference website look tacky and in-turn makes the Idea Fit look tacky.
Recommendation: Use no more than two font styles. Use one serif and the other san serif for contrast. Keep it consistent throughout all the pages.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 0 9
Throughout the IdeaFit website there are numerous things you can click. However, many elements that can be clicked donâ&#x20AC;&#x2122;t go anywhere. They are dead links.
Recommendation: Carefully comb the site for dead links and provide a destination or remove the element if itâ&#x20AC;&#x2122;s not necessary.
1 1 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Feedback & Feed-Forwarding One of the most basic guidelines to improving user experience is proper feedback. Feedback is communication between the user and the product to narrate what’s happening, what has happened, and what can happen. Feed-forwarding is the concept that interface elements – buttons, links, tools, etc. – should always indicate the implication of their actions before a user commits to execution.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 1 1
Impact: Itâ&#x20AC;&#x2122;s really annoying to the user. Although itâ&#x20AC;&#x2122;s not detrimental and the user may re-do their entire thought process again, you will have people that opt out due to frustration.
Recommendation: Instead of a giant jpg that links you to the registration page, feedforward the data using specific links to make the experience more enjoyable.
1 1 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
When selecting workshops that have additional Feeâ&#x20AC;&#x2122;s attached to them, only the latest workshop is added to the cart.
Impact: User will narrow their option to lessen their workload unless the workshop is an absolute must. Money is being lost and/or you are causing the user to keep track of things they usually donâ&#x20AC;&#x2122;t need to keep track of on other sites.
Recommendation: Be sure to forward all the workshops they wish to select. Also wait until the user is done selecting before automatically taking them to checkout.
Expression when trying to use the session selection tool
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 1 3
Navigation There is a convenient and obvious way to move between related pages and sections and it is easy to return to the home page.The information that users are most likely to need is easy to navigate to from most pages. Navigation choices are ordered in the most logical or task-oriented manner. The major sections of the site are available from every page (persistent navigation) and there are no dead ends. There is a site map that provides an overview of the siteâ&#x20AC;&#x2122;s content and it is linked from every page. Good navigational feedback is provided (e.g. showing where you are in the site). Category labels accurately describe the information in the category. Links and navigation labels contain the â&#x20AC;&#x153;trigger wordsâ&#x20AC;? that users will look for to achieve their goal. Product pages contain links to similar and complementary products to support cross-selling. There are clearly marked exits on every page allowing the user to bale out of the current task without having to go through an extended dialog.
Impact The navigation draws away from what is capable to do at the event, due to lack of guidance and facilitating the experience of the user feels lost and unsure how to progress.
Recommendation Revisit the information architecture to ensure there is logical and conceptual flow to the navigation paradigm.
1 1 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Multiple styles of nagivation systems cross different segment presenting similar information doesnâ&#x20AC;&#x2122;t make sense.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 1 5
1 1 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Local Usability Issues The following is a list of local usability issues exemplified through screen shots pertaining to specific sections of the IDEA Fit conference website. Each section of the website was analyzed before and after participants went through each task. We have listed the user interface problems on individual screens that inhibit a user friendly experience when interacting with the website. Lastly, a recommended redesign is provided for each problem, along with a score that evaluates the criticality.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 1 7
Critical Rating Scale The critical rating scale below is provided for each usability principle violation that is listed. The goal is to prioritize the violations to help prioritize for future successful redesign.
â&#x20AC;˘ High - Appropriate re-design solution should be implemented immediately. â&#x20AC;˘ Medium - Appropriate re-design solution should be implemented soon. â&#x20AC;˘ Low - Appropriate re-design solution should be implemented if time and resources allows for it.
1 1 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Home Page The homepage is important as it will be the first interaction that the user has with the website. Most of the information that the user is looking for should be displayed on the website in a salient manner, so the potential attendee can immediately know when the conference is, where the conference will be held and how much it costs to attend the conference.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 1 9
Principle Violated
Recommended Redesign
Poor Use of Screen Real-estate The screen real estate refers to the amount of space available on the display, and whether the positioning and layout of the various screen elements are laid out efficiently relative to the space available. Efficient screen layout makes the website more organized, and aesthetic. Viewing the IDEA homepage through the lens of Screen Real Estate elucidates the notion that the lack of efficient screen real estate makes the website even more congested, confusing and unnecessarily lengthy.
Creating a more minimalistic design with key components that users are looking for displayed in a salient fashion would be recommended. Criticality Rating: High
Steep learning Curve to navigate the website A good interface should be intuitive, so even a first time user does not have too much trouble navigating the website. It is up to the interface to facilitate easy navigation, rather than forcing the user to learn the website. Users typically are in a hurry, donâ&#x20AC;&#x2122;t want to think and will not put in the mental effort to learn a website that they only occasionally use.
Create a more intuitive interface with a simple navigation structure. The proper use of grammar and a focus on the minimum viable product allocates only a certain amount of objects, actions and attributes. Following these principles will ensure a simpler navigation. Criticality Rating: High
Lack of visibility/salience of prominent features forces the user to do more worK An interface with good visibility should ensure that the user is able to see all the available features and options that are available for them right up front. The user should not have to waste time searching for what they need, or using trial and error clicks to obtain what they need. Highlighting important features and making sure they are salient is important because that is what the users will pay attention to.
Highlighting salient information such as date of conference, location of conference, accommodations, price of conference etc. A good example from the NASM Optima conference is provided below. Criticality Rating: High
Banner blindness forces user to ignore relevant information Since relevant information is presented in a manner that looks like an advertisement, users will subconsciously ignore this information. Moreover, creating banner like ads with relevant information makes the website look more busy and cluttered
Get rid of these banners, and present the information in a design that looks more visually appealing and noticeable for the user. Criticality Rating: High
1 2 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Registration This page is important because the user has to easily interact to both pay for the conference and select which days and pre/post conference sessions that they are attending. Therefore creating a registration process with plenty of feedback and no confusion is necessary.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 2 1
The participants had few task that brought them to this page, this page confused them initially and other were unable to understand the hierarchy of the information presented. We explore the reasons below.
Principle Violated
Recommended Redesign
Poor color contrast makes it hard to distinguish certain elements, like the step process in registering for the conference. In design, making all elements on the registration page readable is important
Make this part a darker gray so the user knows how many steps are there, what step they are currently on, and what the future steps entail. CRITICALITY: LOW
Lack of explanation causes user confusion.
Distinguish the difference between 1A and 1B for new users so they understand what they get when they sign up for the session. CRITICALITY: MEDIUM
The back to conference link at the top blends in with the blue banner.
Make this more salient CRITICALITY: LOW
1 2 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Session Selection If the user are to make their own itinerary, this is the tools they would be using. All the participants failed to complete the task on using this tool. All the participnats ignored the instruction presented in the first screen. The heuristics reasons for these issues are expored below.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 2 3
Principle Violated
Recommended Redesign
Readability: The prioritization of information across the page is difficult to understand. The information on the left is too big but the crucial information with the radio buttons are too small. As a result the page is bloated and does not help find the information the visitor is seeking.
The font size has to be balanced. CRITICALITY: MEDIUM
The check boxes on the top which act as filter donâ&#x20AC;&#x2122;t work. The standard mental models users assume is where the filters would make the active options shift and change move up the list.
Bringing the option up in the list on the basis of filter would reduce the amount of scanning the user had to do. Criticality Rating: High
No feedback: The information presented here does help make decisions. User are expecting to have the information carry forward across the website so that it remembers what they wanted.
Adding a basket where the information is clearly presented next to the tool. Criticality Rating: High
1 2 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Session Info: This page is essential to users as it is the main way to plan out the conference experience without logging in. Users use this to find workshops, lectures, and presenters that fit their schedule.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 2 5
Principle Violated
Recommended Redesign
Use of negative space Proper use of negative allows for the ease of visual delineation. Without it, the page looks cluttered with too much text. This causes the user to have to work to differentiate sections.
Increase the amount of space between each section to clearly define section separation Critical Rating: Low
Consistency and emphasis Especially when there is a large amount of similar information displayed, it is vital that the formatting is consistent between sections. Different parts of key information should be sufficiently segregated or emphasized so that users can pick up relevant information quickly.
Keep information consistent between sections. Keep the formatting consistent between sections. Have clear indications as to what is being displayed (e.g. bold subsection titles). Critical Rating: High
Logical Order Important and relevant information should be prioritized so that users can get what they need from a site as quickly as possible. This is especially true for pages with large amounts of text. Here, it is more important that the user knows the time and date of the event than the certification points. Furthermore, having this information higher up is consistent across interfaces the users will be familiar with.
Move the time and date up either just be below or just above the presenter name. The location of event is also missing from session information. Critical Rating: Low
Efficiency of use Filters, sorts, and searches are important whenever this are large amounts of information displayed. Here there are enough sessions to take up and additional 37 pages. Although there is some filtering available, it is insufficient. Users who are looking for a specific session with have to go through each page to find their session. Users who are looking for only lectures or workshops will have to go through page by page. Users who want to attend a session not named in the filter will have to go through page by page.
Implement a search function or increase the options for filtering and sorting. This may include by presenter, by name, by additional categories, or by time block. Critical Rating: High
1 2 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Using negative sepace effectively.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 2 7
Pricing
:
This is one of the most important pages within the website. Conference prices and deals are one of the main contingency factors as to whether a professional will attend the conference or not. Therefore, the prices should be displayed in a clear manner, and the user should be able to easily discern the early bird discount and when these deals expire.
Principle Violated Hyperlink with register now is meaningless, as the whole chart is one big image that clicks to the registration pag.
Recommended Redesign Make it so that upon interaction only clicking the hyperlinked register now text links the user to the registration page. This complements the user's schema about what a hyperlink actually does. CRITICALITY: LOW
1 2 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Special Events: This page shows any networking or other concurrent events that occur in conjunction with the IDEA conference. Properly displaying the events that an attendee can attend, as well as explaining what the event entails and how you will gain different experiences than you would if visit another page.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 2 9
Principle Violated Banner blindness forces user to ignore relevant information Since relevant information is presented in a manner that looks like an advertisement, users will subconsciously ignore this information. Moreover, creating banner like ads with relevant information makes the website look more busy and cluttered. Self exploratory, intuitiveness, and learnability principles. Lack of explanation causes user confusion Confusion with the main page. Special Events Two pages have the same main menu; however, in fact there two different menus with different submenus and functionality/navigation. This confusion annoys users, ruined searching experience, and prolong search time.
Recommended Redesign Get rid of these banners, and present the information in a design that looks more visually appealing and noticeable for the user. Criticality Rating: High
Add more explanation, mission statement. CRITICALITY: MEDIUM Make a different menu with it’s own logic, navigation, and design, if it is a completely different event or Re-organize special events menu, in a matching to main page format/navigation. CRITICALITY: MEDIUM
Two way to improve the site: Dead end Link <back to site> leads to the Idea Assosiation not back to conference site. Page leaves user stranded, with no choice but to start from home again by using <back> button or retype the whole URL address. Dead ends frustrate users and give them a reason to a site. They waste valuable time that a user will spend on a page.
Include the main navigation on every single page, so it is always available. Add home button, breadcrumbs, or “previous page” button on every page Example: Facebook logo always leads to the Criticality Rating: High
1 3 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
V.S
How to use a grid system on a website.
Confusing Navigation system
Dead link, is not paid attention too
A way to get back to main site is always available.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 3 1
Expo Become an Exhibitor: This page is for companies that are interested in exhibiting at the IDEA Fit expo. It is crucial that they understand the benefits of exhibiting, what it entails and how to get more information about exhibiting.
Principle Violated Lack of feedback/confirmation when submitting info. Instead of letting the user know that their information has been received, it just takes the user to a completely different page.
Recommended Redesign Create feedback upon submission such as â&#x20AC;&#x153;Thank you for being interested in exhibiting at the IDEA Expo a representative will be contacting you shortlyâ&#x20AC;? Criticality: High
1 3 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
There needs to be feedback to let the user know that their actions has made a response in the system.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 3 3
Expo. General Information: This page includes general information about the Expo and what it entails, what times it will be held and where at the convention it will be held. Additionally, users can claim their free expo pass on this website.
1 3 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Principle Violated Aesthetic and minimalistic design of text Content and headers should not have extraneous or irrelevant information. Every bit of extra text competes with the rest of the text causing clutter and diminishes visibility.
Logical Order Related information should be adjacent to each each other. This prevents the user to overlooking important content or having their attention broken. Logical Order Important and relevant information should be prioritized so that users can get what they need from a site as quickly as possible. This is especially true for pages with large amounts of text. Here, it is vital that the user knows that they need to bring their confirmation id to the on-site desk in order to pick up the badge but it is buried within a wall of text.
Recommended Redesign Use short concise headers to define points. Specific features of category should be bullet pointed below rather than expressed on the header. Content should minimize the use of lists that are separated by commas with over 3-5 items. Bullet points should only contain key information. Keep them to a short sentence or a fragment. Critical Rating: High Group all of the information about the Expo features together. Move the Expo hours to the side, bottom, or top to prevent it from breaking the flow. Critical Rating: Low
Move the vital information up to the top of the page. Knowing how to get the badge is more important than knowing the features of the pass which they have already obtained. Critical Rating: High
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 3 5
FAQ
:
This page was often visited by the users due to the fact that they had issues in navigating the website. As no search bar is provided, they visited the FAQ page upon difficulty most of the time.
1 3 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Principle Violated
Recommended Redesign
Informational Organization Violation No organization of the information of the FAQ questions causing confusing for the user making it harder to find what they need to accomplish their goal.
Make sections with logical order Critical Rating: LOW
Aesthetic and design Violation Information provided on this FAQ page is too dense, the font size is too small and too many different fonts used. Critical Rating: LOW
Improvement ideas below
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 3 7
IDEAFIT.COM - STUDY SCRIPT SESSION INTRO
Fitness Professional Novice SESSION INTRO Today you will be using the IDEAFit Conference. IDEAFit is an annual fitness conference in which students, fitness professionals, and other interested parties from all over the world attend. Our goal is to investigate how efficient, useful, and satisfactory this website is, and also to find any problems or challenges that might occur during interactions with the website. This should take about an hour. First, we will be asking you to fill out a questionnaire. The questionnaire will contain a few demographic and experience questions. Afterwards, we will move to the testing room where we will ask you to complete several tasks. After each task, we will ask a few questions about how you felt. You will also have 5 minutes to explore the website. Last, we will finish our usability test with a few general questions about your overall experience. If at any time, you are uncomfortable or wish to skip a task, please let us know and we will move on to the next task or discontinue the session. Please remember that we are not testing you, your skills, or your knowledge. If you struggle or fail to complete a task, that is not your fault, it is because the website design was inadequate. Your comments will help us figure out how the website can be improved. QUESTIONNAIRE Please fill out the questionnaire as accurately as possible. COMPUTER Please come with me to the testing room. You will be recorded by the cameras up there and by the microphone on the computer. Yes, they can see you through the one way mirror. Please have a seat. This is the IDEAFit Conference homepage. You can use either the mouse or the track pad. Please try to complete all tasks by using the IDEAFit Conference website. Task 1- Conference location and dates. Let’s begin with the first task. As fitness professional, you are considering attending the IDEAFit conference for the first time. Using the website, find out when and where the conference takes place.
How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why?
1 3 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
How do you feel about the process?
Task 2- Conference tickets and accommodations. Choose the tickets and accommodations as though you are planning on attending the conference this year. What were your search criteria?
Did you notice any discounts or deals? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Task 3- Cheapest 3 day pass. Find the cheapest option for a 3-day pass. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 4- Free Expo ticket. There is a free Expo ticket available on the site. Find a free 1 day Expo ticket. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 5- Volunteer. One of the options to attend the conference for free is to become a volunteer. Find out how to become a volunteer. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 6- Membership.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 3 9
Ticket prices are cheaper if you are an IDEAFit member. Find out how to become an IDEA member (find the registration form). How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 7- Interesting workshop. What fitness activity are you interested in? Find a workshop on it. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 8- Workshop on nutrition. Find a workshop on nutrition. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 9- Favorite Trainer. You are a big fan of personal trainer Benjamin Degenhardt. You want to attend every lecture/ workshop/ meetings he presents. Can you find all information on him and his sessions?
Do you feel like you’ve found all information on him and his sessions? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Task 10- Timetable You know that you want to attend two workshops on Friday. Create a non-conflicting schedule with two workshops you would enjoy. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
1 4 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 11- Session selection tool. Using the Session Selection tool, recreate the same schedule for Friday.
Did you read the instructions on the page? Is the information displayed clearly? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Task 12- Certification. Find out if you can get certification points by attending a workshop. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? FINAL QUESTIONS Now that you are done with all of the tasks, I just have a few questions about your overall experience with the IDEAFit website. How satisfied are you with navigating through this site on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
How can the site change to improve your satisfaction rating? Overall, how difficult was it to navigate the site on a scale of 1-5? 1 = very difficult, 5 = very easy. Why?
Compared to other conference website, how was this website?
What was your favorite part of the website, if any?
What was your least favorite part of the website, if any?
Overall, how did you feel about the site?
Was there anything in the conference site that you thought was missing?
END SESSION Thank you for taking the time to participate in our usability test. We gained a lot of insight from your comments and suggestions. If you have any questions or concerns, please ask us now, or you can contact us later through email or phone. Thanks again. Have a good day.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 4 1
Fitness Professional Experienced SESSION INTRO Today you will be using the IDEAFit Conference. IDEAFit is an annual fitness conference in which students, fitness professionals, and other interested parties from all over the world attend. Our goal is to investigate how efficient, useful, and satisfactory this website is, and also to find any problems or challenges that might occur during interactions with the website. This should take about an hour. First, we will be asking you to fill out a questionnaire. The questionnaire will contain a few demographic and experience questions. Afterwards, we will move to the testing room where we will ask you to complete several tasks. After each task, we will ask a few questions about how you felt. You will also have 5 minutes to explore the website. Last, we will finish our usability test with a few general questions about your overall experience. If at any time, you are uncomfortable or wish to skip a task, please let us know and we will move on to the next task or discontinue the session. Please remember that we are not testing you, your skills, or your knowledge. If you struggle or fail to complete a task, that is not your fault, it is because the website design was inadequate. Your comments will help us figure out how the website can be improved. QUESTIONNAIRE Please fill out the questionnaire as accurately as possible. COMPUTER Please come with me to the testing room. You will be recorded by the cameras up there and by the microphone on the computer. Yes, they can see you through the one way mirror. Please have a seat. This is the IDEAFit Conference homepage. You can use either the mouse or the track pad. Please try to complete all tasks by using the IDEAFit Conference website. Task 1- Conference location and dates. Let’s begin with the first task. As fitness professional, you are considering attending the IDEAFit conference again this year. Using the website, find out when and where the conference takes place.
How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
How do you feel about the process?
Task 2- Conference tickets and accommodations. Choose the tickets and accommodations as though you are planning on attending the conference this year.
1 4 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
What were your search criteria?
Did you notice any discounts or deals? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Have you used this website to find tickets or accommodations before?
Task 3- Cheapest 3 day pass. Find the cheapest option for a 3-day pass. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Have you used this website to get a 3-day pass before?
Task 4- Free Expo ticket. There is a free Expo ticket available on the site. Find a free 1 day Expo ticket. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used this website to get the free Expo ticket before? Task 5- Volunteer. One of the options to attend the conference for free is to become a volunteer. Find out how to become a volunteer. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Have you used this website to find out how to volunteer before?
Task 6- Membership. Ticket prices are cheaper if you are an IDEAFit member. Find out how to become an IDEAFit member (find the registration form). How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why?
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 4 3
How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Are you an IDEAFit member? Did you use this website to sign up to be a member? Task 7- Interesting workshop. What fitness activity are you interested in? Find a workshop on it. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used this website to find workshops before? Task 8- Workshop on nutrition. Find a workshop on nutrition. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 9- Favorite Trainer. Pretend that you are a big fan of personal trainer Benjamin Degenhardt. You want to attend every lecture/ workshop/meetings he presents. Can you find all information on him and his sessions?
Do you feel like you’ve found all information on him and his sessions? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Task 10- Timetable You know that you want to attend two workshops on Friday. Create a non-conflicting schedule with two workshops you would enjoy. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
1 4 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 11- Session selection tool. Using the Session Selection tool, recreate the same schedule for Friday.
Did you read the instructions on the page? Is the information displayed clearly? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used the session selection tool before?
Task 12- Certification. Find out if you can get certification points by attending a workshop. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used this website to find out about certification points for workshops before? FINAL QUESTIONS Now that you are done with all of the tasks, I just have a few questions about your overall experience with the IDEAFit website. How satisfied are you with navigating through this site on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
How can the site change to improve your satisfaction rating? Overall, how difficult was it to navigate the site on a scale of 1-5? 1 = very difficult, 5 = very easy. Why?
Compared to other conference websites, how was this website?
What was your favorite part of the website, if any?
What was your least favorite part of the website, if any?
Overall, how did you feel about the site?
Was there anything in the conference site that you thought was missing?
END SESSION Thank you for taking the time to participate in our usability test. We gained a lot of insight from your comments and suggestions. If you have any questions or concerns, please ask us now, or you can contact us later through email or phone. Thanks again. Have a good day.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 4 5
Exhibitor Novice SESSION INTRO Today you will be using the IDEAFit Conference. IDEAFit is an annual fitness conference in which students, fitness professionals, and other interested parties from all over the world attend. Our goal is to investigate how efficient, useful, and satisfactory this website is, and also to find any problems or challenges that might occur during interactions with the website. This should take about an hour. First, we will be asking you to fill out a questionnaire. The questionnaire will contain a few demographic and experience questions. Afterwards, we will move to the testing room where we will ask you to complete several tasks. After each task, we will ask a few questions about how you felt. You will also have 5 minutes to explore the website. Last, we will finish our usability test with a few general questions about your overall experience. If at any time, you are uncomfortable or wish to skip a task, please let us know and we will move on to the next task or discontinue the session. Please remember that we are not testing you, your skills, or your knowledge. If you struggle or fail to complete a task, that is not your fault, it is because the website design was inadequate. Your comments will help us figure out how the website can be improved. QUESTIONNAIRE Please fill out the questionnaire as accurately as possible. COMPUTER Please come with me to the testing room. You will be recorded by the cameras up there and by the microphone on the computer. Yes, they can see you through the one way mirror. Please have a seat. This is the IDEAFit Conference homepage. You can use either the mouse or the track pad. Please try to complete all tasks by using the IDEAFit Conference website. Task 1- Conference location and dates. Let’s begin with the first task. As a potential exhibitor, you are considering attending the IDEAFit conference for the first time. Using the website, find out when and where the conference takes place.
How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
How do you feel about the process?
Task 2- Conference accommodations. Choose the accommodations as though you are planning on exhibiting at the conference this year.
1 4 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
What were your search criteria?
Did you notice any discounts or deals? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Task 3- Exhibitor. You’re a small business owner and you want to be an exhibitor at the conference. Find out how to do so.
Do you think you have enough information on being an exhibitor?
(Not logged in): Do you think you can obtain more information by logging in?
How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 4- Interesting workshop. What fitness activity are you interested in? Find a workshop on it. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used this website to find workshops before? Task 5- Workshop on nutrition. Find a workshop on nutrition. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 6- Favorite Trainer. Pretend that you are a big fan of personal trainer Benjamin Degenhardt. You want to attend every lecture/ workshop/meetings he presents. Can you find all information on him and his sessions?
Do you feel like you’ve found all information on him and his sessions?
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 4 7
How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 7- Timetable You know that you want to attend two workshops on Friday. Create a non-conflicting schedule with two workshops you would enjoy. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 8- Session selection tool. Using the Session Selection tool, recreate the same schedule for Friday.
Did you read the instructions on the page? Is the information displayed clearly? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used the session selection tool before?
Task 9- Certification. Find out if you can get certification points by attending a workshop. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used this website to find out about certification points for workshops before? FINAL QUESTIONS Now that you are done with all of the tasks, I just have a few questions about your overall experience with the IDEAFit website. How satisfied are you with navigating through this site on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
1 4 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
How can the site change to improve your satisfaction rating? Overall, how difficult was it to navigate the site on a scale of 1-5? 1 = very difficult, 5 = very easy. Why?
Compared to other conference website, how was this website?
What was your favorite part of the website, if any?
What was your least favorite part of the website, if any?
Overall, how did you feel about the site?
Was there anything in the conference site that you thought was missing?
END SESSION Thank you for taking the time to participate in our usability test. We gained a lot of insight from your comments and suggestions. If you have any questions or concerns, please ask us now, or you can contact us later through email or phone. Thanks again. Have a good day.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 4 9
Exhibitor Experienced SESSION INTRO Today you will be using the IDEAFit Conference. IDEAFit is an annual fitness conference in which students, fitness professionals, and other interested parties from all over the world attend. Our goal is to investigate how efficient, useful, and satisfactory this website is, and also to find any problems or challenges that might occur during interactions with the website. This should take about an hour. First, we will be asking you to fill out a questionnaire. The questionnaire will contain a few demographic and experience questions. Afterwards, we will move to the testing room where we will ask you to complete several tasks. After each task, we will ask a few questions about how you felt. You will also have 5 minutes to explore the website. Last, we will finish our usability test with a few general questions about your overall experience. If at any time, you are uncomfortable or wish to skip a task, please let us know and we will move on to the next task or discontinue the session. Please remember that we are not testing you, your skills, or your knowledge. If you struggle or fail to complete a task, that is not your fault, it is because the website design was inadequate. Your comments will help us figure out how the website can be improved. QUESTIONNAIRE Please fill out the questionnaire as accurately as possible. COMPUTER Please come with me to the testing room. You will be recorded by the cameras up there and by the microphone on the computer. Yes, they can see you through the one way mirror. Please have a seat. This is the IDEAFit Conference homepage. You can use either the mouse or the track pad. Please try to complete all tasks by using the IDEAFit Conference website. Task 1- Conference location and dates. Let’s begin with the first task. As business owner, you are considering exhibiting at the IDEAFit conference again this year. Using the website, find out when and where the conference takes place.
How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
How do you feel about the process?
Task 2- Conference tickets and accommodations. Choose accommodations as though you are planning on exhibiting at the conference this year.
1 5 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
What were your search criteria? Did you notice any discounts or deals? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Have you used this website to find tickets or accommodations before?
Task 3- Exhibitor. You’re a small business owner and you want to be an exhibitor at the conference. Find out how to do so.
Do you think you have enough information on being an exhibitor?
(Not logged in): Do you think you can obtain more information by logging in?
How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used this website to find out how to be an exhibitor before? Task 4- Interesting workshop. What fitness activity are you interested in? Find a workshop on it. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used this website to find workshops before? Task 5- Workshop on nutrition. Find a workshop on nutrition. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 5 1
Task 6- Favorite Trainer. Pretend that you are a big fan of personal trainer Benjamin Degenhardt. You want to attend every lecture/ workshop/meetings he presents. Can you find all information on him and his sessions?
Do you feel like you’ve found all information on him and his sessions? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Task 7- Timetable You know that you want to attend two workshops on Friday. Create a non-conflicting schedule with two workshops you would enjoy. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 8- Session selection tool. Using the Session Selection tool, recreate the same schedule for Friday.
Did you read the instructions on the page? Is the information displayed clearly? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used the session selection tool before?
Task 9- Certification. Find out if you can get certification points by attending a workshop. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used this website to find out about certification points for workshops before? FINAL QUESTIONS Now that you are done with all of the tasks, I just have a few questions about your overall experience with the
1 5 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
IDEAFit website. How satisfied are you with navigating through this site on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
How can the site change to improve your satisfaction rating? Overall, how difficult was it to navigate the site on a scale of 1-5? 1 = very difficult, 5 = very easy. Why?
Compared to other conference websites, how was this website?
What was your favorite part of the website, if any?
What was your least favorite part of the website, if any?
Overall, how did you feel about the site?
Was there anything in the conference site that you thought was missing?
END SESSION Thank you for taking the time to participate in our usability test. We gained a lot of insight from your comments and suggestions. If you have any questions or concerns, please ask us now, or you can contact us later through email or phone. Thanks again. Have a good day.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 5 3
Student Novice SESSION INTRO Today you will be using the IDEAFit Conference. IDEAFit is an annual fitness conference in which students, fitness professionals, and other interested parties from all over the world attend. Our goal is to investigate how efficient, useful, and satisfactory this website is, and also to find any problems or challenges that might occur during interactions with the website. This should take about an hour. First, we will be asking you to fill out a questionnaire. The questionnaire will contain a few demographic and experience questions. Afterwards, we will move to the testing room where we will ask you to complete several tasks. After each task, we will ask a few questions about how you felt. You will also have 5 minutes to explore the website. Last, we will finish our usability test with a few general questions about your overall experience. If at any time, you are uncomfortable or wish to skip a task, please let us know and we will move on to the next task or discontinue the session. Please remember that we are not testing you, your skills, or your knowledge. If you struggle or fail to complete a task, that is not your fault, it is because the website design was inadequate. Your comments will help us figure out how the website can be improved. QUESTIONNAIRE Please fill out the questionnaire as accurately as possible. COMPUTER Please come with me to the testing room. You will be recorded by the cameras up there and by the microphone on the computer. Yes, they can see you through the one way mirror. Please have a seat. This is the IDEAFit Conference homepage. You can use either the mouse or the track pad. Task 1- Conference location and dates. Let’s begin with the first task. As a student, you are considering attending the IDEAFit conference for the first time. Using the website, find out when and where the conference takes place.
How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
How do you feel about the process?
Task 2- Conference tickets and accommodations. Choose the tickets and accommodations as though you are planning on attending the conference this year.
1 5 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
What were your search criteria? Did you notice any discounts or deals? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Task 3- Cheapest 3 day pass. Find the cheapest option for a 3-day pass. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 4- Free Expo ticket. There is a free Expo ticket available on the site. Find a free 1 day Expo ticket. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 5- Volunteer. One of the options to attend the conference for free is to become a volunteer. Find out how to become a volunteer. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 6- Membership. Ticket prices are cheaper if you are an IDEAFit member. Find out how to become and IDEA member (find the registration form). How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 5 5
How do you feel about the process? Task 7- Student Discounts. You’re a student and there are always student discounts for conferences. Find out what student discounts are available. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 8- Interesting workshop. What fitness activity are you interested in? Find a workshop on it. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 9- Workshop on nutrition. Find a workshop on nutrition. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 10- Favorite Trainer. You are a big fan of personal trainer Benjamin Degenhardt. You want to attend every lecture/ workshop/ meetings he presents. Can you find all information on him and his sessions?
Do you feel like you’ve found all information on him and his sessions? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Task 11- Timetable? You know that you want to attend two workshops on Friday. Create a non-conflicting schedule with two workshops you would enjoy. //write down their choices// How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why?
1 5 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 12- Session selection tool. Using the Session Selection tool, recreate the same schedule for Friday.
Did you read the instructions on the page? Is the information displayed clearly? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Task 13- Certification. Find out if you can get certification points by attending a workshop. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? FINAL QUESTIONS Now that you are done with all of the tasks, I just have a few questions about your overall experience with the IDEAFit website. How satisfied are you with navigating through this site on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
How can the site change to improve your satisfaction rating? Overall, how difficult was it to navigate the site on a scale of 1-5? 1 = very difficult, 5 = very easy. Why?
Compared to other conference website, how was this website?
What was your favorite part of the website, if any?
What was your least favorite part of the website, if any?
Overall, how did you feel about the site?
Was there anything in the conference site that you thought was missing?
END SESSION Thank you for taking the time to participate in our usability test. We gained a lot of insight from your comments and suggestions. If you have any questions or concerns, please ask us now, or you can contact us later through email or phone. Thanks again. Have a good day.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 5 7
Student Experienced SESSION INTRO Today you will be using the IDEAFit Conference. IDEAFit is an annual fitness conference in which students, fitness professionals, and other interested parties from all over the world attend. Our goal is to investigate how efficient, useful, and satisfactory this website is, and also to find any problems or challenges that might occur during interactions with the website. This should take about an hour. First, we will be asking you to fill out a questionnaire. The questionnaire will contain a few demographic and experience questions. Afterwards, we will move to the testing room where we will ask you to complete several tasks. After each task, we will ask a few questions about how you felt. You will also have 5 minutes to explore the website. Last, we will finish our usability test with a few general questions about your overall experience. If at any time, you are uncomfortable or wish to skip a task, please let us know and we will move on to the next task or discontinue the session. Please remember that we are not testing you, your skills, or your knowledge. If you struggle or fail to complete a task, that is not your fault, it is because the website design was inadequate. Your comments will help us figure out how the website can be improved. QUESTIONNAIRE Please fill out the questionnaire as accurately as possible. COMPUTER Please come with me to the testing room. You will be recorded by the cameras up there and by the microphone on the computer. Yes, they can see you through the one way mirror. Please have a seat. This is the IDEAFit Conference homepage. You can use either the mouse or the track pad. Please try to complete all tasks by using the IDEAFit Conference website. Task 1- Conference location and dates. Let’s begin with the first task. As a student, you are considering attending the IDEAFit conference again this year. Using the website, find out when and where the conference takes place.
How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
How do you feel about the process?
Task 2- Conference tickets and accommodations. Choose the tickets and accommodations as though you are planning on attending the conference this year.
1 5 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
What were your search criteria? Did you notice any discounts or deals? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Have you used this website to find tickets or accommodations before?
Task 3- Cheapest 3 day pass. Find the cheapest option for a 3-day pass. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Have you used this website to get a 3-day pass before?
Task 4- Free Expo ticket. There is a free Expo ticket available on the site. Find a free 1 day Expo ticket. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Have you used this website to get the free Expo ticket before?
Task 5- Volunteer. One of the options to attend the conference for free is to become a volunteer. Find out how to become a volunteer. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Have you used this website to find out how to volunteer before?
Task 6- Membership. Ticket prices are cheaper if you are an IDEAFit member. Find out how to become an IDEAFit member (find the registration form).
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 5 9
How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Are you an IDEAFit member? Did you use this website to sign up to be a member? Task 7- Student Discounts. You’re a student and there are always student discounts for conferences. Find out what student discounts are available. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used this website to find student discounts before? Task 8- Interesting workshop. What fitness activity are you interested in? Find a workshop on it. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used this website to find workshops before? Task 9- Workshop on nutrition. Find a workshop on nutrition. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 10- Favorite Trainer. Pretend that you are a big fan of personal trainer Benjamin Degenhardt. You want to attend every lecture/ workshop/meetings he presents. Can you find all information on him and his sessions?
Do you feel like you’ve found all information on him and his sessions?
1 6 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 11- Timetable You know that you want to attend two workshops on Friday. Create a non-conflicting schedule with two workshops you would enjoy. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 12- Session selection tool. Using the Session Selection tool, recreate the same schedule for Friday.
Did you read the instructions on the page? Is the information displayed clearly? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used the session selection tool before?
Task 13- Certification. Find out if you can get certification points by attending a workshop. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used this website to find out about certification points for workshops before? FINAL QUESTIONS Now that you are done with all of the tasks, I just have a few questions about your overall experience with the IDEAFit website. How satisfied are you with navigating through this site on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 6 1
How can the site change to improve your satisfaction rating? Overall, how difficult was it to navigate the site on a scale of 1-5? 1 = very difficult, 5 = very easy. Why?
Compared to other conference websites, how was this website?
What was your favorite part of the website, if any?
What was your least favorite part of the website, if any?
Overall, how did you feel about the site?
Was there anything in the conference site that you thought was missing?
END SESSION Thank you for taking the time to participate in our usability test. We gained a lot of insight from your comments and suggestions. If you have any questions or concerns, please ask us now, or you can contact us later through email or phone. Thanks again. Have a good day.
1 6 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Volunteer Novice SESSION INTRO Today you will be using the IDEAFit Conference. IDEAFit is an annual fitness conference in which students, fitness professionals, and other interested parties from all over the world attend. Our goal is to investigate how efficient, useful, and satisfactory this website is, and also to find any problems or challenges that might occur during interactions with the website. This should take about an hour. First, we will be asking you to fill out a questionnaire. The questionnaire will contain a few demographic and experience questions. Afterwards, we will move to the testing room where we will ask you to complete several tasks. After each task, we will ask a few questions about how you felt. You will also have 5 minutes to explore the website. Last, we will finish our usability test with a few general questions about your overall experience. If at any time, you are uncomfortable or wish to skip a task, please let us know and we will move on to the next task or discontinue the session. Please remember that we are not testing you, your skills, or your knowledge. If you struggle or fail to complete a task, that is not your fault, it is because the website design was inadequate. Your comments will help us figure out how the website can be improved. QUESTIONNAIRE Please fill out the questionnaire as accurately as possible. COMPUTER Please come with me to the testing room. You will be recorded by the cameras up there and by the microphone on the computer. Yes, they can see you through the one way mirror. Please have a seat. This is the IDEAFit Conference homepage. You can use either the mouse or the track pad. Task 1- Conference location and dates. Let’s begin with the first task. You are considering volunteering for the IDEAFit conference for the first time. Using the website, find out when and where the conference takes place.
How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
How do you feel about the process?
Task 2- Accommodations. Choose the accommodations as though you are planning on volunteering for the conference this year.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 6 3
What were your search criteria?
Did you notice any discounts or deals? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? How do you feel about the process?
Task 4- Free Expo ticket. There is a free Expo ticket available on the site. Find a free 1 day Expo ticket. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 5- Volunteer. One of the options to attend the conference for free is to become a volunteer. Find out how to become a volunteer. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 6- Membership. You are considering becoming an IDEA member. Find out how to become an IDEA member (find the registration form). How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 8- Interesting workshop. What fitness activity are you interested in? Find a workshop on it. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
1 6 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Task 9- Workshop on nutrition. Find a workshop on nutrition. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 10- Favorite Trainer. You are a big fan of personal trainer Benjamin Degenhardt. You want to attend every lecture/ workshop/ meetings he presents. Can you find all information on him and his sessions?
Do you feel like you’ve found all information on him and his sessions? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Task 11- Timetable? You know that you want to attend two workshops on Friday. Create a non-conflicting schedule with two workshops you would enjoy. //write down their choices// How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 12- Session selection tool. Using the Session Selection tool, recreate the same schedule for Friday.
Did you read the instructions on the page? Is the information displayed clearly? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Task 13- Certification. Find out if you can get certification points by attending a workshop. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why?
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 6 5
How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? FINAL QUESTIONS Now that you are done with all of the tasks, I just have a few questions about your overall experience with the IDEAFit website. How satisfied are you with navigating through this site on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
How can the site change to improve your satisfaction rating? Overall, how difficult was it to navigate the site on a scale of 1-5? 1 = very difficult, 5 = very easy. Why?
Compared to other conference website, how was this website?
What was your favorite part of the website, if any?
What was your least favorite part of the website, if any?
Overall, how did you feel about the site?
Was there anything in the conference site that you thought was missing?
END SESSION Thank you for taking the time to participate in our usability test. We gained a lot of insight from your comments and suggestions. If you have any questions or concerns, please ask us now, or you can contact us later through email or phone. Thanks again. Have a good day.
1 6 6 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Volunteer Experienced SESSION INTRO Today you will be using the IDEAFit Conference. IDEAFit is an annual fitness conference in which students, fitness professionals, and other interested parties from all over the world attend. Our goal is to investigate how efficient, useful, and satisfactory this website is, and also to find any problems or challenges that might occur during interactions with the website. This should take about an hour. First, we will be asking you to fill out a questionnaire. The questionnaire will contain a few demographic and experience questions. Afterwards, we will move to the testing room where we will ask you to complete several tasks. After each task, we will ask a few questions about how you felt. You will also have 5 minutes to explore the website. Last, we will finish our usability test with a few general questions about your overall experience. If at any time, you are uncomfortable or wish to skip a task, please let us know and we will move on to the next task or discontinue the session. Please remember that we are not testing you, your skills, or your knowledge. If you struggle or fail to complete a task, that is not your fault, it is because the website design was inadequate. Your comments will help us figure out how the website can be improved. QUESTIONNAIRE Please fill out the questionnaire as accurately as possible. COMPUTER Please come with me to the testing room. You will be recorded by the cameras up there and by the microphone on the computer. Yes, they can see you through the one way mirror. Please have a seat. This is the IDEAFit Conference homepage. You can use either the mouse or the track pad. Please try to complete all tasks by using the IDEAFit Conference website. Task 1- Conference location and dates. Let’s begin with the first task. You are considering volunteering for the IDEAFit conference again this year. Using the website, find out when and where the conference takes place.
How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
How do you feel about the process?
Task 2- Accommodations. Choose the accommodations as though you are planning on volunteering for the conference this year.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 6 7
What were your search criteria? Did you notice any discounts or deals? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Have you used this website to find tickets or accommodations before?
Task 4- Free Expo ticket. There is a free Expo ticket available on the site. Find a free 1 day Expo ticket. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Have you used this website to get the free Expo ticket before?
Task 5- Volunteer. One of the options to attend the conference for free is to become a volunteer. Find out how to become a volunteer. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Have you used this website to find out how to volunteer before?
Task 6- Membership. You are thinking of becoming an IDEAFit member. Find out how to become an IDEAFit member (find the registration form). How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Are you an IDEAFit member? Did you use this website to sign up to be a member? Task 7- Interesting workshop. What fitness activity are you interested in? Find a workshop on it.
1 6 8 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used this website to find workshops before? Task 8- Workshop on nutrition. Find a workshop on nutrition. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 9- Favorite Trainer. Pretend that you are a big fan of personal trainer Benjamin Degenhardt. You want to attend every lecture/ workshop/meetings he presents. Can you find all information on him and his sessions?
Do you feel like you’ve found all information on him and his sessions? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process?
Task 10- Timetable You know that you want to attend two workshops on Friday. Create a non-conflicting schedule with two workshops you would enjoy. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Task 11- Session selection tool. Using the Session Selection tool, recreate the same schedule for Friday.
Did you read the instructions on the page? Is the information displayed clearly? How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 6 9
How do you feel about the process? Have you used the session selection tool before? Task 12- Certification. Find out if you can get certification points by attending a workshop. How difficult would you rate it on a scale of 1-5. 1 = very difficult, 5 = very easy. Why? How satisfied are you with the result on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why? How do you feel about the process? Have you used this website to find out about certification points for workshops before? FINAL QUESTIONS Now that you are done with all of the tasks, I just have a few questions about your overall experience with the IDEAFit website. How satisfied are you with navigating through this site on a scale of 1-5? 1 = very dissatisfied, 5 = very satisfied. Why?
How can the site change to improve your satisfaction rating? Overall, how difficult was it to navigate the site on a scale of 1-5? 1 = very difficult, 5 = very easy. Why?
Compared to other conference websites, how was this website?
What was your favorite part of the website, if any?
What was your least favorite part of the website, if any?
Overall, how did you feel about the site?
Was there anything in the conference site that you thought was missing?
END SESSION Thank you for taking the time to participate in our usability test. We gained a lot of insight from your comments and suggestions. If you have any questions or concerns, please ask us now, or you can contact us later through email or phone. Thanks again. Have a good day.
1 7 0 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Appendix 1 A
Consent Form
Usability Test Consent Form Idea World Convention Website The purpose of this usability study is to evaluate the usability of the Idea World Convention website. We are interested in determining if people can accomplish common tasks and easily find information using this site. The session will not test you or your ability, rather the session will test the website to obtain information on areas that might be improved. Please be advised that there are no risks associated with participation in this session. You will be compensated $100 for participating in this study. During this session, you will be asked to complete several tasks using the website. As you complete the tasks, five students of San Jose State Universityâ&#x20AC;&#x2122;s Human Factors and Ergonomics graduate program in the Usability Testing class will observe and take notes. In addition, the session will be captured on video for future review and use in the usability report. The session will last no longer than an hour. If, for any reason, you do not wish to complete a task, you may say so and we will move on to the next task. In addition, if you do not wish to continue the session, you may end the session and leave at any time. Results from the session will be included in a usability report to be used for academic and nonacademic purposes. Your name and other personal identifiable information will not be included in the report nor will you be associated with any session data collected. If you wish to speak with someone about your participation in this study, or if you have any other concerns, please contact us at SpartanUX@gmail.com. I, ______________________________________________, have read and fully understand the extent of the study and any risks involved. All of my questions, if any, have been answered to my satisfaction. My signature below acknowledges my understanding of the information provided in this form and indicates my willingness to participate in this usability testing session. I have been given a copy of this consent form for my records.
______________________________ _________________ Signature Date
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 7 1
Appendix 1 B
Participants Screener 1) What is your occupation?
If they answer is within the health and wellness field, such as personal trainer, group exercise instructor, mind-body coach, fitness studio owner, nutrition professional, or fitness/wellness program director or manager then skip to question 4. These people will be placed in the Fitness Professional category. If they answer no, continue on to question 2. 2) Are you currently a university student? If yes, what major?
If they answer with a major such as Kinesiology, Sports Science, Physical Education, Recreation etc. then skip to question 4. These people will be placed in the studentâ&#x20AC;&#x2122;s category. If they answer no, continue to question 3. 3) Do you own a company or work with a company that would potentially benefit from exhibiting your product at a fitness expo?
If yes, go to question 4. These people will be placed in the exhibitorâ&#x20AC;&#x2122;s category.
4) Have you ever attended a conference put on by IDEA Health and Fitness Association, such as the IDEA World Fitness Convention or the IDEA World Expo?
If yes, go to question 5. If no, these people will be placed in the novice category.
5) In attending a previous year of the IDEA World Fitness Convention or IDEA World Expo, did you interact with the website to pay for the conference, schedule your sessions and get more informa-
1 7 2 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Fitness professional, coaches, gym owners. $100 for 1 hour (San Jose) Reply to: spartanux@gmail.com Date: March 28th 2016, 3:00 PM PST
We are looking for your opinion and feedback on a fitness conference website. If you are interested in making $100 in an hour, continue reading to see if you qualify for this research.
We at SpartanUx are looking for people in the fitness industry who are Gym owners, personal trainer, coaches and students who are working toward getting into the fitness industry.
In this study we would look at how you use the website, ask for your feedback and opinion.
If you think you qualify and would like to participate, please contact Scamalot at +1 415-555-1234 or email us at spartanux@gmail.com.
Please feel free to forward this advert to people who you may know are in the fitness industry and visit fitness conference.
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 7 3
Email Recruitment Dear Previous Participant,
We are looking for your opinion and feedback on a fitness conference website. If you are interested in making $100 in an hour, continue reading to see if you qualify for this research.
We at SpartanUx are looking for people in the fitness industry who are Gym owners, personal trainer, coaches and students who are working toward getting into the fitness industry.
In this study we would look at how you use the website, ask for your feedback and opinion.
If you think you qualify and would like to participate, please contact Scamalot at +1 415-555-1234 or email us at spartanux@gmail.com.
Regards,
WEBSITE RESEARCH
Spartan UX
Use a website for 60 mins and get paid $100. Must be a fitness professional or gym owner looking to network with other similar professionals. Must be 18 years and older. Call +1 415-555-1234 or email spartanux@gmail. com
1 7 4 | U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N
Appendix 2A
Interact with a website and give us your feedback and opinion about the site.
Fitness professional, coaches, gym owners. $100 for 1 hour (San Jose)
Website Research (Flyer)
To participate, call SpartanUx at +1 415-555-1234 or email at spartanux@ gmail.com
Website Research $100 for 1 hour (415)-555-1234 Website Research $100 for 1 hour (415)-555-1234 Website Research $100 for 1 hour (415)-555-1234 Website Research $100 for 1 hour (415)-555-1234 Website Research $100 for 1 hour (415)-555-1234 Website Research $100 for 1 hour (415)-555-1234 Website Research $100 for 1 hour (415)-555-1234 Website Research $100 for 1 hour (415)-555-1234
U S A B I L I T Y E VA L U AT I O N S T U D Y P L A N | 1 7 5
Exhibitor Fake Information
Appendix 3A
First name: Dana Last name: Smith Email: danafitness@email.com Phone: 800-342-9976 Company: Perform Better City: San Ramon State: CA
Students Fake Information
Email: samfitnessnerd@sjsu.edu First Name: Sam Last Name: Jones Phone: 408-222-2939 School: San Jose State Major: Kinesiology