UX Case Study: Books Kinokuniya Usability Test Report

Kinokuniya Usability Test Report

PREPARED BY UX RESEARCHERS Yvonne Han Kua Si Liang Leow Hou Teng 17 January 2017

http://design.leowhouteng.com

Table of Contents

1. Introduction 2. Executive Summary 3. Goals 4. Evaluation Tasks/Scenarios 5. Methodology 6. Results 7. Recommendations & Implementation 8. Conclusion

http://design.leowhouteng.com

Introduction

Founded in Japan in 1927, Books Kinokuniya has been a presence in Singapore since 1983. It offers a diverse and eclectic collection of books in many languages, ranging from the latest Japanese manga to wide collections of books examining everything from philosophy, science to pop culture.

About this report This usability test report, compiled by Novel & Chord following a usability test, is the key document for Kinokuniya to g ​ ain a deeper understanding of their customer’s needs and frustrations​. The test is intended to determine the extent the proposed website redesign facilitates a user’s ability to complete routine tasks.

Usability Testing The tests were conducted face-to-face with a group of potential users (participants).

Participants were asked to complete a series of routine tasks. Sessions were recorded and analyzed to identify potential areas for improvement to the web site.

http://design.leowhouteng.com

Executive Summary

How the tests were conducted ●

Face-to-face usability test with a laptop.

●

Participants were assigned 3 tasks on the current website then repeated the same tasks (3 +1) for the proposed design.

●

Researcher took note of where the participants attempted to find a particular piece of information needed for the tasks.

●

Time was taken on how quickly the participants were able to complete the tasks.

●

Participants were asked to speak aloud what they were thinking.

At certain points during the test, participants were asked ●

What they understand about a page;

●

What they can infer from texts, labels, and buttons;

●

What they expect on the following page;

●

How easy or difficult they felt in performing a given task;

●

Their likelihood of using the proposed website.

The test identified only a few minor problems including: ●

Labeling of buttons to indicate what happens when it is clicked

●

Indicating steps in the on-boarding and checkout process

●

Instructions given with the use of popups.

This document contains the participant feedback, satisfactions ratings, task completion rates, ease or difficulty of completion ratings, time on task, errors, and recommendations for improvements.

http://design.leowhouteng.com

Goals

1. Assess the usability of the web interface design, 2. user flow, 3. and information architecture.

Evaluation Tasks/Scenarios

Tasks 1 – 3 were assigned to the participants on the current site. They repeated the tasks and completed tasks 1 – 4 on the new site.

1. Using the navigation, look for the storybook The Pigeon Tunnel

by J ​ ohn Le Carre's​, and check if the book is available at the Bugis outlet.

2. Using the search bar, look for the same book.

3. Add the book to the cart and place an order for the book.

4. From the home page, sign up for a membership.

http://design.leowhouteng.com

Methodology

Sessions ●

Participants were recruited from our pool of social contacts or random strangers we met.

●

Each session lasts for 30 mins – 1 hour.

●

Each participant were assigned​ 3 tasks on the current site​ then ​repeated the tasks and an additional task to complete on the new site​.

●

Participants were given the task scenarios and tried to find the information on the website.

●

Time taken for each tasks were recorded.

●

Questions may be asked during the tasks, which may affect the overall timing they took to complete the tasks.

●

After each task, participants rated on a scale of 1(very easy) – 5(very difficult) what they felt about the assigned tasks.

●

Additional questions were asked at the end of session to understand difficulties they faced which influenced their rating on the scale.

In addition, the researcher asked the participants the following overall website questions: ●

How likely will they use the new site.

●

What the participant liked most.

●

What the participant liked least.

●

Recommendations for improvement.

Participants ●

6 participants were randomly recruited from pool of friends and strangers.

●

Age range 20s to 30s, who are more likely to shop for books online.

http://design.leowhouteng.com

Results

Task Completion Success Rate ●

All participants successfully completed all assigned task e ​ xcept for Task 1 (find the book using the navigation) o ​ n the current site.

●

Participants were successful since the user flow is similar to any other e-commerce website.

●

It was impossible to find a book by browsing using the current navigation menu, given that there are 800+ pages in the book category (Fiction). However, the new design intentionally places the book on page 1 of the sub-category page.

Nevertheless, users were q ​ uicker in locating the sub-category page directly from the menu​ as compared to searching for it on a sidebar.

Current Site

Participants

Task 1

Task 2

Task 3

Task 1

Task 2

Task 3

Task 4

1

✘

✔

✔

✔

-

✔

✔

2

✘

✔

✔

✔

-

✔

✔

3

✘

✔

✔

✔

-

✔

✔

4

✘

✔

✔

✔

✔

✔

✔

5

-

-

-

✔

✔

✔

✔

6

-

-

-

✔

✔

✔

✔

Success

0

4

4

7

3

7

7

Completion Rates

0%

100%

100%

100%

100%

100%

100%

http://design.leowhouteng.com

New Site

Time taken to complete task Researchers recorded the time on task for each participant using a stopwatch. Some tasks were inherently more difficult to complete than others and is reflected by the average time on task. ●

Overall, there was an improvement in speed taken to complete all tasks.

●

For task 1 (find a book using the navigation menu), the average time taken improved from​ 2:20 on the current site, to 1:09 on the new site​. As none of the participants completed task 1 on the current site, the time taken to locate the correct sub-category was recorded in the table below.

●

For task 2 (find a book using the search bar), the average time taken​ improved from 1:03 on the current site, to 0:18 on the new site​. However, the search feature on the new site is simulated, hence users do not have to do an actual search.

●

As questions were asked during Task 3 on the new site, the time taken to complete the task was not recorded.

●

For Task 2 (find the book using the search bar), not all participants were able to test the feature since it was not included in the proposed website prototype.

Current Site

Participants

Task 1

Task 2

Task 3

Task 1

Task 2

1

1:52

1:18

1:20

0:56

0:10

2

1:42

0:20

1:20

0:57

-

3

2:36

1:55

1:48

1:11

-

4

3:09

0:39

1:00

1:17

0:35

5

-

-

-

1:34

0:07

6

-

-

-

1:00

0:19

Average Time Taken

2:20

1:03

1:22

1:09

0:18

http://design.leowhouteng.com

New Site

Rating given for each task After the completion of each task, participants rated the tasks on a scale of 1(very easy) – 5(very difficult) ●

Overall, there was a significant improvement in rating given to the new site as compared to the current site.

●

For t​ ask 1​ (find a book using the navigation menu), the average rating improved from 4 ​ .375 on the current site, to 2.0 on the new site​.

●

For t​ ask 2​ (find a book using the search bar), the average rating improved from 1.5 on the current site, to 1.25 on the new site​.

●

For t​ ask 3​ (complete the book purchasing process), the average rating improved from 1 ​ .75 on the current site, to 1 on the new site.

●

For t​ ask 4​ (register for an account), participants rated a ​1.5 on average​ for the new site.

Current Site

Participants

Task 1

Task 2

Task 3

Task 1

Task 2

Task 3

Task 4

1

4

1

2

1

2

1

1

2

5

2

3

2

-

1

1.5

3

3.5

2

1

2.5

-

1

2

4

5

1

1

3

1

1

2

5

-

-

-

2

1

1

1

6

-

-

-

2

1

1

-

Average Rating

4.375

1.5

1.75

2.0

1.25

1

1.5

http://design.leowhouteng.com

New Site

Summary of Data The table below displays a summary of the test data. Low completion rates and satisfaction ratings and time on tasks are highlighted in red.

Current Site

New Site

Task 1

Task 2

Task 3

Task 1

Task 2

Task 3

Task 4

Completion

0%

100%

100%

100%

100%

100%

100%

Time

2:20

1:03

1:22

1:09

0:18

-

-

Ratings

4.375

1.5

1.75

2.0

1.25

1

1.5

Likes, Dislikes, Participant Recommendations Upon completion of the tasks, participants provided feedback for what they liked most and least about the proposed website by comparing it with the current website. In addition they provided recommendations for improving the website, which was implemented to the current iteration.

Liked Most The following comments capture what the participants liked most: ●

The improved accessibility of the sub-categories/genres in the navigation menu

●

The addition of the membership sign up in the checkout process, since it offers discounts for their purchase.

●

Member privileges are presented clearly.

●

Product information is available and clear.

●

Customising book preferences recommendations.

●

Preview feature is useful when a user is unsure about buying a book.

●

Swipe for more books on mobile.

http://design.leowhouteng.com

Liked Least The following comments capture what the participants liked the least: ●

Participants felt that the inclusion of the membership in the sign up process was confusing since they have already signed up for an account.

●

Participants felt the inclusion of membership in the payment process was unexpected.

●

Customisation process was sudden and unexpected.

Recommendations for Improvement ●

Explain why the user should link the card to the account

●

Include ads in the website to promote the membership sign up so that the inclusion in the sign up and checkout process will not be as intrusive.

●

Redundant to verify purchases repeatedly since users are unlikely to modify purchase once they decide to checkout.

●

Indicate processes in the sign up onboarding so that users are clear what will happen next.

●

Label adjustments for buttons on what is happening next, such as ‘Add book preferences’, instead of ‘customise your account’.

●

Include tags in the book/product page.

Likelihood of using the site All participants expressed that they are ​likely to use the new proposed website​ if they want to buy a book. This is partly influenced by the brand recognition with Kinokuniya.

Although they do not usually purchase books from online, the new features (customised book recommendations and book preview) will allow them to browse new books and influence their decision to make a purchase.

http://design.leowhouteng.com

Recommendations & Implementation

The recommendations section provides recommended changes and justifications driven by the participant success rate, behaviors, and comments. Each recommendation includes a severity rating(1(least severe)–4(most severe)).

The following recommendations will improve the overall ease of use and address the areas where participants experienced problems or found the interface/information architecture unclear. M ​ ost of these recommendations has been applied to the latest iteration.

Task 1 – Find a book from the Navigation Menu & check the stock availability Change

Justification

Severity

Change ‘Literature & Fiction’ label to ‘Fiction & Literature’

Participants had trouble filtering through the list of 25 categories.

Storybooks are usually found under ‘Fiction’ or ‘Adult Fiction’ hence they had trouble finding it under ‘Literature and Fiction’

3

Add tags on book product page

Participants need to search for a product using keywords related to a book

1

http://design.leowhouteng.com

Task 3 – Add a book to cart and proceed with checkout Change

Justification

Severity

Hyperlink terms and conditions and privacy policy instead of displaying a lengthy text

Page was full of text

3

Membership fees to be shifted up

Participants were not able to draw the link 4 between the savings of 10% with the membership fees and will likely skip the step

Add 1 year membership fees at the top

Users can do their own mental calculation 3 if the fees are worth it

http://design.leowhouteng.com

Conclusion

Most of the participants found the proposed Kinokuniya website to be well-organized, comprehensive, clean and uncluttered, very useful, and easy to use. They felt the process was easy since it adopts a familiar e-commerce process.

The addition of customising book preferences and the book preview feature was a thoughtful inclusion on the new site.

The addition of the membership process in the signup and checkout process was acceptable and the participants were aware that they can skip the step.

http://design.leowhouteng.com