AD 611 Advanced Web Design
Usability testing methods Research and Evaluation of a Mobile Portfolio Sun young Ahn Department of Visual Communications Design, Purdue University Advisor: Petronio A. Bendito (pbendito@purude.edu)
1. Introduction Every day, people visit thousands of websites for various purposes, and each website tries to provide its information in an effective way. Even though websites present their contents in various ways, there are some principles that determine a good web design. Usability is one of the principles that provide a continuous and flawless web experience for the users. According to ISO 9241-111 usability is the degree to which a product can be used by specific users to reach specific goals with efficiency, effectiveness and satisfaction in a given use context 2. It means that usability should be understood as an interaction between a user and the product rather than a feature of the product alone. Even if a website has great contents, it will lose the opportunity to provide them for users if it has a low degree of usability. An inefficient and inconvenient web experience will make users leave the website and deter them from visiting the website again. So, we can say that usability is a fundamental factor to consider when designing a successful website2. 2. Background To measure the usability of a website and enhance the user experience, various types of evaluation methods were developed. Usability evaluation methods (UEMs) are testing tools used to evaluate the usability of a product design, and detect the problems. They can be classified in numerous ways. The most common way is to divide them into user testing methods and usability inspection methods according to user involvement 3. User testing involves representative users as participants. While user testing methods study actual users, usability inspection methods examine the product by following a guideline without user involvement. Among these two, user testing is considered a better way of finding problems related to users’ tasks and experience. User testing covers several methods such as, surveys, focus groups, laboratory testing, contextual inquiry, and remote testing4. A survey is a method to send out inquiries and collect data from a large population in a short period of time. A focus group is done by inviting a small group of end users in to talk about a product, a marketing campaign, or a new product concept. A laboratory-based usability test is a synonym for empirical usability testing. It is designed for evaluators to observe and record users’ interactions by asking a user to complete a given task in the lab setting. Generally, a user is asked to speak their thoughts aloud while the user is using a test system, and we call it ‘Think Aloud.’ Contextual inquiry is a combined form of observation and structured interview. The interviewer observes the user at work and asks questions about the work. Finally, remote testing is a method where the participant and facilitator are in two different physical locations.
Also a UEM can be classified as a formative evaluation or summative evaluation according to the time when it is executed. While a formative evaluation is done during development to improve a design, a summative evaluation is done after development to assess a design. In other words, the former is used to find usability problems to fix so that an interaction design can be improved, and the other is used to assess or compare the level of usability achieved in an interaction design5.
3. Selection of UEMs: Contextual Inquiry
Before selecting evaluation methods for a product, we need to think about the main goal because some methods are better than others depending on the situation. My product is a mobile portfolio and I would like to see how users react to the user interface that I designed. Because my goal for the evaluation is to observe the interaction between users and the product, and improve its design, I decided to utilize the method of contextual inquiry as a formative evaluation method. Contextual inquiry is a field method wherein usability evaluators go to the users’ workplaces, observe them at work, and ask questions regarding the work content, process, or product usage4. Unlike the traditional laboratory-based testing, an evaluator can communicate with a user in the contextual inquiry. Basically, laboratory testing is conducted in the room that has a one way observation mirror. To reduce the user’s stress of being observed, a user accomplishes given tasks without an evaluator in the same room. On the other hand, contextual inquiry allows a structured interview with a user. So, as a partner, the evaluator can find issues with a user. From the experience, the evaluator can learn the relationship between a user and a designer as well as a user and a product. Also, while lab-based testing needs a specialized lab setting, contextual inquiry doesn’t need professional equipment, and it has the merit that an evaluator can find problems in the user’s work context. However, the interview can interfere with the user’s intention or induce the user to answer as the interviewer wants. So, for a successful testing, an interview should keep the balance of power with the user.
4. Usability Test and Findings
To get various opinions on the product, I selected three users who had different levels of knowledge and experience on web design and usability. I observed the users, and asked them to accomplish prepared tasks. The tasks were mainly about finding certain information from a random page and figuring out the path to go back to the previous page. With the method of Think Aloud, users expressed their conscious thoughts and emotional impression of each webpage while they were exploring it. When they tried to do
certain actions, they were asked the intention of the action, and the expected scene after the action. Also, I played the role of a guide when they had difficulties in accomplishing a task. From the observation and interview, the results can be summarized into four directions.
4-1. Confusion between a Button and Text
The main problem of my website design was that it created confusion about buttons. When users faced each page, they tried to find the area where they needed to touch to move to the next page. To give a hint for users I put a small arrow beside the menu. However, users tended to touch every place where there was text with a bold font or a geometric shape on the screen. We can see the area that created confusion in Fig. 1. Because the big shape and bold types caught users’ eyes better than small arrows, users tried to touch those areas first. However, interestingly, users didn’t react to the text on the top of the screen. Based on their web experience, they considered it to be the title of the page rather than a button.
[Figure 1] Confusion of buttons
This problem also came from the lack of unity in the visual design. Even though users felt a sense of unity from the consistent use of the main colors, black, white, and pink, they felt that they could not find consistency in terms of the use of shapes and layout. While this diversity provides a creative and artistic impression, it hinders the users’ fluent navigation. As seen in Fig. 1, each page has different shapes of buttons, so users needed to take a little time to figure out which area functions as a button.
4-2. Inefficient Navigating System
One of the crucial principles of an efficient user interface design is the availability of free movement to the page where a user wants to go. To evaluate the efficiency of my portfolio, I asked users to navigate and locate certain information when starting from a random page. Overall, users intellectually understood where to go. However, even though users knew where certain information was located, they expressed the inconvenience of navigating. Because some web pages didn’t have a ‘Home button,’ they had to go back to the ‘Home’ page using the path they had passed. Users wanted a button that took them to the ‘Home’ page at once. One user even wanted a floating menu that she could open whenever she wanted. It would help users move to any place they wanted from any other place. For example, a user said, “Now, I’m reading your statement in the ‘Resume’ part, but I want to see your design now, is there any way that I can move to your design page?” Also, users said it was a little difficult to figure out which level or layer they were in. My portfolio consists of several layers, and each layer has alternating black or white backgrounds. Because black and white colors appeared repeatedly, users were confused how deep they had moved. A user said, “I feel like I get lost. I can’t remember how many layers I have passed. I think pushing a ‘Back button’ is the only way that I can come back to the ‘Home’ page!” The user suggested using different colors for each layer or each section. Web users figure out their location by visual cues such as color and layout as well as the typographic message. So, visual elements of the website should be able to function as a guide for navigation.
4-3. Misarrangement of Contents My portfolio has several menu pages such as ‘Home,’ ‘Works,’ and ‘Resume.’ These pages have three or four lists and they are vertically organized in rows. Even though the lists didn’t have numbers that indicate the order of the lists, users intuitively focused on the first list. So, to meet the users’ expectation, a designer should have arranged contents in order according to the importance starting from the first row. Because my product is a portfolio, users wanted to see my design works first. However, I put the ‘Resume’ tab first and placed the ‘Works’ tab after the resume, so I realized that my decision didn’t meet the users’ prediction. We can see this from the comment that “I see three lists here, and I think I should navigate each of them in a row from the first. However, I’m interested in your design rather than your personal information, so I would like to see the ‘Works’ tab first,” and “I think that some fancy stuff will happen after opening this first page!
Also, I made a similar mistake on the ‘Works’ page. I put various works of design in this tab, and I classified them into three, fashion design, fashion & graphics, and VCD projects. Because I worked as a fashion designer for a long time, I unconsciously put fashion design first, and VCD projects at the bottom. However, since the given situation was my job interview for a graphic designer, users wanted to see my graphic designs first. The order of the lists was contrary to not only the order of significance but also chronological order. From the three results, I realized that considering basic human perception is a fundamental step in designing an efficient user interface.
4-4. Feedback on Visual Concept
Even though several problems on the navigation system were found, I could see that users perceived the visual design as I intended. The main visual concept of my design is providing an experience of exploring layers by the action of cutting and folding. To get a raw impression of the portfolio, I asked users about the overall visual impression of it without letting them know the concept. There were comments such as, “I can see this portfolio has several layers and I feel like I’m moving around these layers,” “It seems that each page is covered with a black layer and I feel like I pull the next layer,” “Some pages look like papers randomly cut by a scissors,” and “The overall design gives a visual impression of sharpness because of the cutting-like edges.” From these comments, I saw that users could figure out my design intention. I also got words such as ‘feminine,’ ‘playful,’ ‘creative’ and ‘fresh’. Users thought that the portfolio had a little different design concept and protocol from a normal website, and gave new feeling. However, it provided an unfamiliar feeling as well. Generally, people need a little time to learn new methods, so an unfamiliar interface and protocol can cause an inefficient and inconvenient web experience.
5. Design Revision
To reduce the possibility of miscommunication, the website was revised based on the result of the test. First of all, the text and shapes that were not used for buttons were removed from the menu pages. Instead, the title of each page was placed on the top of the screen. Because the text on the head part is not perceived as a button, it will decrease users’ mistakes to touch a wrong area. Also it contributes to an efficient navigation experience. In every page, users can figure out where they are with the title on the top. Second, I put a ‘Home button’ on each page to help users easily come back to the first place where they started to navigate. Finally, I changed the order of lists on the menu by putting ‘Design’ and ‘VCD
projects’ on the first line. It will help viewers focus more on my design works rather than personal information, and gives an impression of a graphic designer rather than a fashion designer. In addition to these main changes, other problems are also amended in terms of contents, layout, and so on. You can see the revision in Fig. 2 below.
[Figure 2] Design Revision
6. Conclusion
As mentioned at the beginning, usability is not simply a characteristic of the product, but involves human interaction with a product in a context of use. In other words, usability is formed by complex interactions with not only the product itself, but also the physical and social environment in which the product is used. A high degree of usability means a successful communication between a user and a designer. To minimize the possibility of misunderstanding, a designer needs to have a user-centric perspective. In this light, understanding people’s perceptions and reading their psychology are crucial to enhance user experience. Also, there always exists a tension between a desire for creativity and continuing the conventional way of communication. An attempt to create a new type of user interface design provides fresh and dynamic web experiences for users. However, if the design is not based on enough consideration for usability from the perspective of users, it only creates confusion and inconvenience. On the other hand, overemphasis on efficiency makes people get bored. Because users want both aesthetic values and functional values at the same time, finding a balance between the two is one of the most important roles of a designer.
Participants
User1: Male/ Age: 35/ Student/ Major: Mechanical Engineering User2: Male/ Age: 31/ Computer programmer/ Major: Computer Science User3: Female/ Age: 34/ Housewife/ Major: Business Administration
References 1. Smith, W., “A Guide to Implementation and Compliance.” (Eds.) ISO and ANSI- Ergonomic Standard for Computer Products, Prentice Hall Saddle, New Jersey, 1996 2. Maria E. Alva O., Ana B. Martínez P., Juan M. Cueva L.,T. Hernán Sagástegui Ch., and Benjamín López P., “Comparison of Methods and Existing Tools for the Measurement of Usability in the Web”, J.M. Cueva Lovelle et al. (Eds.): ICWE 2003, LNCS 2722, pp. 386–389, 2003. 3. Sirpa Riihiaho, "Experiences with Usability Evaluation Methods", Licentiate's thesis, 2000, pp. 7 4. Peishan Tsai, "A Survey of Empirical Usability Evaluation Methods". GSLIS Independent Study 5. H. Rex Hartson , Terence S. Andre & Robert C. Williges, "Criteria For Evaluating Usability Evaluation Methods", International Journal of Human-Computer Interaction, 15:1, pp. 145-181