TABLE OF CONTENTS
1. EXECUTIVE SUMMARY ................................................................................ 2 RESULTS................................................................................................................................. 3
2. NIKEiD: CUSTOMIZATION OPTION FOR NIKE PRODUCTS ........................ 4 3. METHODOLOGY ........................................................................................... 4 PROCEDURE.......................................................................................................................... 5 SESSION STRUCTURE......................................................................................................... 5 PARTICIPANTS ...................................................................................................................... 6 EQUIPMENTS AND MATERIALS ........................................................................................ 6
4. EVALUATION TASKS LIST AND SCENARIOS ............................................. 7 5. RESULTS AND FINDINGS............................................................................. 7 TASK RATINGS...................................................................................................................... 7 FINDABILITY .......................................................................................................................... 8 EASE OF USE OF CUSTOMISATION TOOL ..................................................................... 8 LEARNABILITY....................................................................................................................... 9 SATISFACTION USING THE SITE ...................................................................................... 9 SUMMARY OF DATA ............................................................................................................ 9
6. RECOMMENDATIONS ................................................................................ 11 SEVERITY RATING .............................................................................................................. 11
7. CONCLUSION.............................................................................................. 14 8. APPENDIX A: TASK SCRIPT........................................................................ 15 FINDINGS: ............................................................................................................................ 17
9. APPENDIX B: POST EVALUATION QUESTIONNAIRE ................................ 18 10. APPENDIX C: INFORMED CONSENT FORM ............................................ 21 11. REFERENCES ........................................................................................... 22
1
1. EXECUTIVE SUMMARY
The following study has been carried out to identify the usability issues with Nike’s website, NIKEiD customisation page in particular. Nike’s customisation option (NIKEiD) is an important feature on Nike’s website and it is essential to evaluate its easy of use, findability and learnabiliy. This evaluation will therefore aim to answer the following concerns. • • • •
Do users know there is a customisation option from the homepage? Can users easily find NIKEiD page from the homepage? Once users are on the customisation page do they know how to use it? Are participants interested in returning for future purchase?
A representative sample of site users were recruited to attempt to perform tasks typical to those users would attempt in the real world. A total of 6 participants were involved in the usability test to ensure a constant result. Studies have shown that 4-5 participants can show upto 80% of usability issues in a product (Rubin, 1994). The individual session lasted approximately 60 min. Each participant was presented with a task scrip (Appendix A) to direct him/her to complete a typical shopping scenario that included finding the right item and customising it to their liking. A short pre and post task questionnaire was also included to capture users outlook on using NIKEiD and rate the process of their customisation. (Appendix B) Two competitor websites (Adidas and Reebok) with similar customisation options, were also presented to participants to compare their results with Nike’s website. Participants were encouraged to think aloud as they undertook each task. Notes were taken by the moderator of participant’s thoughts about using the website, why they were performing a certain action and how they made decisions. Silverback software was used to video participants whilst they undertook the task and was later used to analyze participant interactions, facial expressions and verbal comments.
2
RESULTS
Task 1 I’d like you to show me how you would find a pair of running shoes from the homepage. You should make sure the shoes you choose can be customised to your liking.
Nike
Finding
By just looking at the homepage 66% of participants didn’t realise there was a customisation option. Participants weren’t familiar with the term “NIKEiD” and “NIKE+” and were reluctant to check them.
Adidas
Reebok
By just looking at the homepage all of the participants identify the customisation option. The word “Customise” on the homepage was a helpful indication
By just looking at the homepage all of the participants identify the customisation option. The word “Design your own” on the homepage was a helpful indication.
The shortcut to most popular sports listed in the navigation bar was very helpful.
The price filter on the product page was helpful.
The description of “NIKEiD” doesn’t stand out well.
Task 2 I’d like you to show me how you would Customise the body, lace and sole (or the mid-sole) of the shoes, to Red, Blue and White colour scheme.
Finding
Nike
Adidas
Reebok
It made more sense to participants to have the “Start Customising” option before the “Add to basket” option. One participant didn’t notice the “Add to cart” option after completing the customisation.
5/6 participants preferred the customisation tool to be on the right hand side of the screen.
The ID personalisation option was hard to find.
The slight change of colour when hovering on the shoes was useful.
Having a separate option for ID personalisation was useful.
3
The ID personalisation option was hard to find.
The circles icons on the customisation tool panel doesn’t have lables next to them therefore, wasn’t that helpful.
The circles icon on the right of each customising section changes colour after customising, this was very useful to distinguish which sections have been customised.
Some of the text in the customisation section was too small.
The opaque image on the customisation page distracted
2. NIKEiD: CUSTOMIZATION OPTION FOR NIKE PRODUCTS
NIKEiD allows customers to customise their desired product from scratch. Customers can express their style by choosing their favourite colours, material, the level of cushioning and personalized touch on the shoes tongue. This service can be accessed both online from Nike’s homepage and in physical NIKEiD studios available in different countries around the world. (Wikipidia, 2013) To best facilitate users to find, customise and purchase their selected product it is important to evaluate the usability of the website in order to discover any possible interface issues.
3. METHODOLOGY
User testing approach was used to evaluate NIKEiD page and two competitor websites. This approach involved testing a relatively small, representative group of users in order to yield reliable results and reduce the amount of time spent conducting and analysing test results. This methodology allowed a better understanding to why the participant may have interacted with NIKEiD and competitor websites in a certain way. Questionnaires also helped to gauge the participants’ subjective reactions and solicit ideas for improvements. (Appendix B)
4
Participants were recruited and tested over a three-day period. All participants were tested in private and quiet room to insure they wouldn’t be disturbed by external factors.
PROCEDURE Each session lasted approximately 60 min. During the session, the moderator introduces the test and asks a few open questions (Appendix A). The participants were then presented with a task scenario and asked to complete the tasks. (Attachment A). Upon completing each task, the moderator asked a few post-task questions (Appendix A). Some of which included: •
How easy it was to find the information from the home page.
•
What the participant liked most.
•
What the participant liked least.
Finally, at the end of each session participants were asked to rate the websites on a 5point Likert Scale with measures ranging from Strongly Disagree to Strongly Agree. (Attachment B)
Some of the measures included: • Ease of use •
Learn ability - how easy it would be for most users to learn to use the website
•
Information facilitation
•
Look & feel appeal
SESSION STRUCTURE To insure constant results, all session were structured in the following way: •
The participant reads and signs the Informed Consent Form (Appendix C).
•
The moderator introduces the test and the process. (Appendix A)
•
The participant is asked to ‘think aloud’ while performing the tasks in order to get a more realistic assessment of their behaviour.
•
The participant is presented with the tested website.
•
The participant is requested to complete a total of three tasks for each website. (Appendix A). These tasks are representative of the tasks that typical users would be attempting in the real world.
•
The participant is asked to complete a short questionnaire upon each task completion. (Appendix A).
5
•
After completing the tasks the participant is given a questionnaire to complete (Appendix B). The questionnaire collects subjective feedback regarding the features and functionality that were considered most useful, desired features/functionality, problem areas, etc.
•
The participant’s on-screen interactions, facial expressions and verbalizations are observed for later analysis.
PARTICIPANTS Participants in the user testing evaluations were required to perform tasks and participate in a one-to-one session. A total of 6 participants were recruited for this evaluation. Of the six participants, three were male and three were female. All participants were aimed to reflect Nike’s core customers. The participants recruited for the research had to have the following profiles: Criteria
Details
Gender
Not specific
Age
18-35
Location
London
Internet Usage
Intermediate to expert: Must use a variety of sites on a regular basis. Must use the Internet at least 4 hours per week every week on average. Must be an online shopper, or have shopped in the past.
Other
NONE to have used Nike’s website before. Interested in sports and sport wear.
All participants were offered refreshments and snacks for their participation.
EQUIPMENTS AND MATERIALS Bellow is a list of materials used during the testing sessions: • • • • • • •
Test laptop Silverback software Participant instructions sheet Informed consent form Task list Pre and Post-task questionnaire Post-evaluation questionnaire
6
4. EVALUATION TASKS LIST AND SCENARIOS
A total of nine task, three for each website, has been designed for this evaluation. The tasks were devised based on Nike’s aims and concerns. Each task has completion criteria, which have been established in order to determine if the goal state for each task is reached. Additionally, participants were required to complete a short questionnaire at the end of each task, and one at the end of the final task. A short summary of the tasks complete by participants is as follows: •
Find a pair of running shoes that you can customise to your liking
•
Customise the body, lace and sole (or the mid-sole), to your Red, Blue and White colour scheme.
•
Inscribe the word “HAPPY” on your shoes.
(See Attachment A&B for complete test scenarios/tasks and questionnaires)
5. RESULTS AND FINDINGS
TASK RATINGS Participants were asked to rate the tasks they completed based on two factors: •
Ease of completing the task. Which was rated on a scale of 5 ranging from (Very difficult) to (Very easy)
•
Satisfaction in the time spent to complete the task. Which was rated on a scale of 5 ranging from (Strongly disagree) to (strongly agree) A summary of the finding is listed in the charts below:
7
Task difficulty! 5 4.8 4.6 4.4 4.2 4 3.8 3.6
Satisfaction in time of completing the task!
Task 1! Task 2! Task 3!
Nike!
Adidas!
Reebok!
6 5 4 3 2 1 0
Task 1! Task 2! Task 3! Nike!
Adidas!
Reebok!
(See Appendix A for detailed data table)
FINDABILITY Almost all of the participants agreed it was easier to find a product on Adidas website (mean agreement rating = 4.4). • • •
The navigation bar at the top of the page was clear and useful The shortcut to most popular sports on the navigation bar was also useful The label “Customise” on the homepage navigation bar was used by all participants
While 86% of the participants agreed that it was easy to find the product they were looking for on the Nike website. • • •
Participants mainly used the shop or sport icon from the navigation bar to select their categories. Most of the participants didn’t know the meaning of “NIKEiD” and “NIKE +” and therefore where reluctant to check them Participants found the filters very useful and easy to use.
80% of the participants agreed it was easy to find a product on Reebok website. • • •
The label “Design your Own” on the homepage navigation bar was useful However, when participants chose a category they weren’t sure which shoes was customisable and which one was unless they click on each individually. The price filter was useful
EASE OF USE OF CUSTOMISATION TOOL An average of 92% of the participants found it easy to customise their selected shoes on Adidas website. • •
The customisation tool was easy and clear. Participants found it useful to have the label of each option next to the circle icon. 8
•
•
They also found it helpful when the icons would change colour once an element was customised. This enabled them to distinguish between customised elements and non-customised elements. Participants felt the opaque image in the background wasn’t as good as Nike’s white background
Participants also found Nike’s customisation tool easy to use scoring a mean agreement rate of 4.5. • •
• •
Participants liked the clean and sleek design of the page, and thought the white background gave a good contrast to view the shoes. However once they have chosen a product to customise participants felt that having to click on “Start customising” button was an extra click and preferred it to be ready for customisation. The mouse hover on the shoes works well and almost all of the participants used it to customise their item. The slight change in colour of any element participants hovered on was also helpful.
Participants found Reebok also easy to use scoring an average of 86% on the ease of customisation but preferred the other two websites. • • •
Some of the text on the page was too small There were too many pop-up windows when customising. The mouse hovering on the shoes was also easy to use.
LEARNABILITY All of the participants (an average of 100%) agreed both Nike and Adidas’s customisation tool is easy to learn. Although few had difficulties starting to use the tool they quickly learn their way around it. They also stated that if they revisit the site again they wouldn’t have any issues using the tool. Reebok however scored a mean of 4.6 in learnability.
SATISFACTION USING THE SITE Adidas scored the highest with an average of 96%. Participants enjoyed the user-friendly interface and simple terminology. Nike scored second with an average of 92%. All participants loved the sleek layout of the website. Participants felt the amount of information give about each product was just right. There wasn’t too much unnecessary information given. Reebok scored a mean of 4.5. Participants didn’t enjoy the experience as much as they did with Nike and Adidas. The excessive details and small text repelled participants from revisiting the site.
SUMMARY OF DATA At the end of each session paticipants were asked to rate the sites for four overall measures (see Appendix X for full data) 9
The measures where as follows: • • • •
Findability Ease of use Learnability Satisfaction
Most of the participants (86%) agreed that it was easy to find their desired item on Nike’s website. 90% of the participants also agreed that it was easy to customise a product. They all (100%) agreed that it was easy to learn. And 92% said they would return in the future. While 96% of participants agreed that it was easy to find a product on Adidas’s website. 92% stated it was easy to customise an item and they all approved that it is easy to learn. A majority of participants (96%) ensured they would use the site again 80% of participants however, found Reebok site easy to find a product. While 86% believed the customisation tool was easy to use. 92% of participants said it was eay to learn and 90% claimed they would return in the future. The chart below shows a summary of the data gathered. (See Appendix B for full detail of the gathering). By findability we mean success in finding the desired item, Ease of use relates to the customisation tool, learnability is the ability to learn how to use the site and customisation tool quickly, and satisfaction refers to the desire to return to the website for future shopping.
Means task Rating and Percentage! 6! 5! 4! Nike!
3!
Adidas!
2!
Rebook!
1! 0! Findability!
Ease of use!
Learnabiliy!
Satisfaction!
10
6. RECOMMENDATIONS
SEVERITY RATING To evaluate the findings of the usability test, issues have been classified by severity rating. Severity ratings are a way of indicating the seriousness of a usability issue and are provided to help client prioritise the order in which issues are to be resolved. The severity ratings are determined by three influencing factors, these being: 1. Frequency of occurrence 2. Persistence of issue (one-time problem that can be overcome vs. recurring issue) 3. Impact of issue (the level of ease or difficulty with which the user is able to overcome the issue)
The following table describes each level of defect severity. Severity Rating
Explanation
N/A usability issue ✩✩✩✩✩
No action is required. Often these types of issues are positive findings of interest
Minor usability issue
Often linked to the look & feel or minor functionality.
★★✩✩✩
Should be fixed if time and budget allow.
Moderate usability issue
Often linked to the experience with the site.
★★★✩✩
Easier to overcome than major issues. Should be rectified as soon as time and budget allow.
Major usability issue ★★★★★
Seriously impact the user experience, often leading to abandonment of the session. Hinder conversion, as they are difficult to overcome if encountered. It is crucial that this is rectified immediately.
11
Page Home page
Product page
Issue
Recommendation
Homepage is simple, clutter free and straight to the point. Participants found the navigation bar simple and helpful to start their shopping.
Severity
✩✩✩✩✩
Most of the participants weren’t familiar with the term “NIKEiD” and “NIKE+” and were reluctant to check them. Therefore, they couldn’t find out the customisation option.
Add a customisation icon or a short word like “customise” to indicate what “NIKEiD” is
The description of “NIKEiD” doesn’t stand out well.
Move the description to the right hand side to be viewed first, and make the text bolder to be seen. Adding a splash of colour may help.
Filter menu is very easy and clear to understand.
★★★✩✩
★★★✩✩
✩✩✩✩✩
Participants didn’t know difference between narrow, regular, and wide in the “width” filter
A short description or a table explaining the difference between narrow, regular, and wide.
★★★✩✩
Its not possible to choose a price range
Add a price filter
★★✩✩✩
12
Page Customisation page
Issue
Recommendation
Severity
If the participant is on the customisation page there is no need for asking to start customisation. The customisation tool should be automatically open and ready.
The customisation tool should be automatically open and ready.
★★✩✩✩
It made more sense to participants to have the “Start Customising” option before the “Add to basket” option. One participant didn’t notice the “Add to cart” option after completing the customisation.
Move the “Add to Cart” and present it after the completion of the customisation process.
There isn’t an undo option.
Add an undo option.
The circle icons are very vague and don’t help much.
Add a title next to each describing their functionality.
★★★✩✩
The title of each section and its navigation arrows aren’t very useful.
Remove the title and arrows and add the titles next to the circular icons.
★★✩✩✩
The “personalised ID” option isn’t easily seen as it is in the same list as the colour and material customisation choice.
Create a separate section for “Personalised ID”.
The white and grey background creates a good contrast against the image displayed.
★★★★★
★★✩✩✩
★★✩✩✩
✩✩✩✩✩
13
7. CONCLUSION
Most participants found Nike’s website sleek, clutter free, and easy to use. Finding products was straight forward, either by clicking on the shop, sport, or NIKEiD icon. The customisation option was easy to learn and fun. The variety of option to customise a product amazed many participants. Continuing to focus on a user centred design and by implementing the recommendations Nike can continue to take the lead as an ecommerce website.
14
8. APPENDIX A: TASK SCRIPT
Moderator Script Thank you for agreeing to participate in this usability study. Please keep in mind that you are not being evaluated. The purpose of this study is to identify issues with the usability of various sport websites. Therefore, there are no wrong answers and any comments you have should be made freely. You will be given a list of tasks, to perform using Nike, Adidas and Reebok websites. Work through them as efficiently as you can but don’t rush. These tasks should take approximately 60 minutes to complete in total. A moderator will be present with you during the session. Before beginning each task, please read/ listen to the instructions. Please think out loud as you work. Describe your thoughts about the websites you are using, why you are performing certain actions, how you make your decisions, why something is bothering you and so forth. If you feel you are stuck and would like to ask me a question, please do. I will however, ask you to try to find your way around the problem before specifically answering you, as this is how I discover the most interesting aspects of the websites you use. If you have any questions the moderator will be pleased to answer them for you.
Task 1 – Find a pair of running shoes that you can customise to your liking. Description
I’d like you to show me how you would find a pair of running shoes from the homepage. You should make sure the shoes you choose can be customised to your liking.
Purpose
To understand the journeys that users take to find a customizable product.
Post-task ratings Overall, this task was:
Very difficult ( )
Overall, I am satisfied with the amount of time it took to complete this task.
Very easy ( )
( )
( )
Strongly disagree
( )
( ) Strongly disagree
( )
( )
( )
( )
15
Task 2 – Customise the body, lace and sole (or the mid-sole) of the shoes, to Red, Blue and White colour scheme. Description
I’d like you to show me how you would customise the shoes to Red, Blue and White colour.
Completion criteria
Participant successfully completes customising the shoe.
Purpose
To understand the process of customising a product in order to evaluate any problems.
Post-task ratings Overall, this task was:
Very difficult ( )
Overall, I am satisfied with the amount of time it took to complete this task.
Very easy ( )
( )
( )
Strongly disagree
( )
( ) Strongly disagree
( )
( )
( )
( )
Task 3 - Inscribe the word “HAPPY” on your shoes. Description
I’d like you to show me how you would inscribe the word “HAPPY” on the shoes.
Completion criteria
Participants successfully complete personalising the shoes
Purpose
To understand and observe how users find the personalisation option and get feedback on overall experience.
Post-task ratings Overall, this task was:
Very difficult ( )
Overall, I am satisfied with the amount of time it took to complete this task.
Very easy ( )
( )
( )
Strongly disagree
( )
( ) Strongly disagree
( )
( )
( )
( )
16
FINDINGS:
Task difficulty
Task 1
Task 2
Task 3
Ave
Nike
4.3
4.5
4.5
4.4 (88%)
Adidas
4.8
4.3
4.6
4.5 (91%)
Rebook
4
4.3
4
4.1 (82%)
Satisfied with time taken to complete task
Task 1
Task 2
Task 3
Ave
Nike
4.5
5
4.5
4 (80%)
Adidas
5
5
4.6
4.3 (86%)
Rebook
4.3
4.6
4
4.6 (92%)
17
9. APPENDIX B: POST EVALUATION QUESTIONNAIRE
Thank you for completing the usability research. Please answer the following questions about your experience with the websites you used today. Privacy information: Please note that this questionnaire is completely confidential and
anonymous. The information gathered here in will only be used for the purposes of this study.
1.
Of all the websites that you used today, please rate each site out of 5 (5 is very positive and 1 is very negative) in the following category: Website
I found it easy to find what I was looking for
Rating out of 5
Nike Adidas Reebok
2.
Of all the websites that you used today, please rate each site out of 5 (5 is very positive and 1 is very negative) in the following category: Website
I found it easy to customise my product.
Rating out of 5
Nike Adidas Reebok
3.
Of all the websites that you used today, please rate each site out of 5 (5 is very positive and 1 is very negative) in the following category: Website
I think that most people would learn how to use the customising option very quickly
Rating out of 5
Nike Adidas Reebok
18
4.
Of all the websites that you used today, please rate each site out of 5 (5 is very positive and 1 is very negative) in the following category: Website
I would like to shop at this site again
Rating out of 5
Nike Adidas Reebok
5.
Please circle the website you found easiest to use overall? Nike Adidas Reebok
6.
What was it about this website that you liked over the other two?
7.
Please circle the website you found most difficult to use overall? Nike Adidas Reebok
8.
What did you like least about this site?
THANK YOU
19
NIKE
P1
P2
P3
P4
P5
P6
Ave
I found it easy to find what I was looking for.
4
5
3
4
5
5
4.3 (86%)
I found it easy to customise my product.
5
3
5
5
4
5
4.5 (90%)
I think that most people would learn how to use the customising option very quickly.
5
5
5
5
5
5
5 (100%)
I would like to shop at this site again.
5
5
3
5
5
5
4.6 (92%)
P1
P2
P3
P4
P5
P6
Ave
I found it easy to find what I was looking for.
5
5
4
5
5
5
4.8 (96%)
I found it easy to customise my product.
5
4
5
5
5
4
4.6 (92%)
I think that most people would learn how to use the customising option very quickly.
5
5
5
5
5
5
5 (100%)
I would like to shop at this site again.
5
5
4
5
5
5
4.8 (96%)
P1
P2
P3
P4
P5
P6
Ave
I found it easy to find what I was looking for.
3
5
3
4
4
5
4 (80%)
I found it easy to customise my product.
5
4
4
5
4
4
4.3 (86%)
I think that most people would learn how to use the customising option very quickly.
4
5
4
5
5
5
4.6 (92%)
I would like to shop at this site again.
5
5
3
4
5
5
4.5 (90%)
Rating out of 5
Adidas Rating out of 5
Reebok Rating out of 5
20