Psychology 301(Beachboard) Website Heuristic Evaluation by Meghann

Psychology 301(Beachboard) Website Heuristic Evaluation

Heuristic Evaluation The following section contains a list of violations of the 10 usability heuristics developed by Jakob Nielsen (Nielson, 2000) as they apply to the Psychology 301 course website, hosted by the BeachBoard software at California State University, Long Beach (https://www.BeachBoard.csulb.edu). A team of 4 investigators independently evaluated the current version of the Psychology 301 by performing a number of probable tasks that students and faculty would perform. These tasks were developed from information regarding “course goals” taken directly from the Psychology 301 website. The website was evaluated in terms of overall flow, ease of use, and navigation. * The following is an ordered list of heuristic violations found and ranked by degree of severity.

BeachBoard Violations Heuristic violations listed in this category are directly related to the design and software development of BeachBoard. The following list acts as a summary to assist the web and software developers of BeachBoard with the most crucial heuristic violations. (For a full description of these summaries see the section below: Heuristics in Detail, page 9)

Top 10 Heuristic Violations (Most Important) BeachBoard 1.) The system does not inform user of saving on quiz/test page when the “save” button at the bottom is selected. However, it does inform user when individual questions are “saved”) 2.) Bread crumbs are not used consistently (they use inconsistent names with the locations they relate to) 3.) No control over the presentation of announcements, instructions, and/ or assignments. (Unable to checkmark complete or “read” sections) 4.) No visual indication that a link has been previously selected 5.) High Level Navigation buttons don not indicate the users current position 6.) instructions are given from CSULB not BeachBoard

7.) No search bar exists for BeachBorad 8.) Undo/Redo function is not supported but Back/Forward is supported 9.) Breadcrumb bar disappears when scrolling takes place 10.) Bread crumb actions are not consistent with description. (Bread crumb links take the user to different pages than assumed or in some cases where they have not been previously)

Psychology 301 Related Heuristic violations listed in this category are related to the development and design of the Psychology 301 website on BeachBoard. Thus, the following list is aimed to help the faculty that developed and currently update the Psychology 301 website. (For a full description of these summaries see the section below: Heuristics in Detail, page 9)

Top 10 Heuristic Violations (Most Important) Psychology 301 Related 1.) Links are not available when referenced 2.) Current order of High Level Navigation (HLN) buttons do not follow user mental models 3.) Inconsistent format and depth of information in the site. (Some sections are shallow and other are deep) 4.) “Assignment schedule” contains too many colors and does not effectively inform users the meaning of the colors. 5.) Confusing use of the color Red to symbolize a topic heading and clickable link) 6.) Heading titles are too long 7.) Help function available only for BeachBoard not PSY 301 8.) HLN button titled “Instruction” does not effectively describe the information within it 9.) Instruction on using BeachBoard for Psy301 is not easy to find (FAQ exist in “Instructions” button) 10.) Schedule button is redundant

Table of Contents Introduction…………………………………………………………………………… 5 Heuristics in Detail………………………………………………………………….... 5 Nielson Heuristic 1. Systems Status and Update……..………………………………. 5 Beach Board Related……………………………………………………….. 5 Psychology 301 Related……………………………………………………. 6 Nielson Heuristic 2. Match Between System and the Real World…………………….7 Psychology 301………………………………………..…………………….7 Nielson Heuristic 3. User Control and Freedom……………………………………. 8 Beach Board Related………………………………………………………..8 Psychology 301 Related………………………………………………….... 9 Nielson Heuristic 4. Consistency and Standards …………………………………....9 Beach Board Related………………………………………………………..9 Psychology 301 Related………………………………………………..…..11 Nielson Heuristic 5. Error Prevention ……………………………………………....12 Psychology 301 Related……………………………………………………12 Nielson Heuristic 6. Recognition Rather Than Recall……………………………….13

Psychology 301 Related…………………………………………………....13 Nielson Heuristic 7. Flexibility and Efficiency of Use……………………………… 13 NONE Found………………………………………………………….13 Nielson Heuristic 8. Aesthetic and Minimalist Design…………………………13 Psychology 301 Related…………………………………………….…14 Nielson Heuristic 9. Help users recognize, diagnose, and recover from errors....15 None Found…………………………………………………………....15 Nielson Heuristic 10. Help and Documentation………………………………...15 Beach Board Related…………………………………………………..15 Psychology 301 Related…………..…………………………………...16 References………………………..……………..……………………………….17 Table 1 ~ Performance Data for the Amount of Time (in seconds) it Took Each User to Finish each Individual………… 18 Table 2. ~ Performance Data for the Number of Clicks it Took Each User to Finish each Individual Task……………….… 19 Table 3. ~ Symbol Recognition Data (Percentages calculated out of four users)………………………………………………………. 20 Appendix A ~ Symbols with blanks to allow users to fill in meaning…………………………………………….…22 Appendix B ~ Matching of symbol with meaning that is used on Beachboard…………………………………24 Appendix C ~User generated categories and the categories used on Beachboard…………………………………………26

Introduction Heuristics in Detail This section contains an elaborated description of Jakob Nielsen’s 10 usability heuristics and how they are in direct violation of these standards on the Psychology 301 website. Violations of each heuristic are divided into two sections: BeachBoard (BB) related and Psychology 301 related to help facilitate corrections. Each violation has been given a title, a description of the violation, the location of where it was found, and a recommendation to correct the violation.

Nielson Heuristic 1. Systems Status and Update “The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.” Beach Board Related 1. High Level Navigation buttons should indicate the users’ current page position Location of Problem: All Pages Description of Problem: The high level navigation (HLN) buttons (the list of links located on the left side of each page) do not visually change (ie. Color) when the user selects them to help inform the user which webpage is currently being viewd. For example, when the user navigates to the “Grades” page, the HLN button entitled “Grades” do not change color or visual appearance to show the user were exactly in the website hierarchy they currently are. Recommendation to fix Problem: As the user selects a HLN button, have that button change color to indicate to the user what page they are currently on. For example, if the user selects “Grades” from the HLN options, this “Grades” button should change to a color visually different from the remaining HLN options. The “grades” button should remain in the new color while the user stays on the “grades” content page and only change back to the original color when the user selects a new HLN button. Each HLN option should behave this way.

2. No visual indication that a link has been previously selected Location of Problem: Anyway a link exists

COURSES > PSY 301 PSY-DISCIPLINE & PROFESSION 3861 > SCHEDULE

Description of Problem: BB does not keep the user informed that a link has already been visited by changing to a different color (which is standard in many web applications). This may disorient the user as he/she navigates through the site causing him/ her to accidentally re-visit already viewed links, especially when a page (i.e., Instruction and Assignment pages) contains a large number of links. Recommendation to fix Problem: After a link has been viewed by the user the color of the link should change to a different color. This new color should remain the same until it expires after a pre-set number of days (usually determined by the user). 3. Breadcrumb bar disappears when scrolling takes place Location of Problem: All Pages Description of Problem: Breadcrumbs (that indicate the user’s most current position in a site) should remain persistent, and always visible at the top of the page. In BB, the breadcrumbs are associated with the page frame and thus as the user scrolls down the breadcrumbs disappear from view and become useless as navigational aids. Recommendation to fix Problem: Breadcrumbs should remain static on the top of the webpage and not disappear when the user scrolls down the page. The titles of each breadcrumb should have the exact same title and be in the same order as the web pages viewed by the user, thus, actively reflecting the users position. Psychology 301 Related 1. The extensive use of colors on the “Texts, Readings,” “Assignments” and “Assignment Schedule” web pages distracts and confuses users. Location of Problem: Syllabus section Description of Problem: (Color and information overload)

The third HLN link brings the user to a page with prerequisites and assignments. Although the designer of the website did a great job categorizing the students by last name, the next page bombards the user to too much confusing information. This violates the “Visibility of System Status” heuristic as well as the “Aesthetic and minimalist design” heuristic because there is too much visual information jumbled into one section, which might confuse the student. Recommendation to fix Problem: Color coding is typically a useful way of providing visual structure to complex information. However, too much color just adds to the problem. Color-coding should only re-enforce an existing logical order. When the use of color to convey meaning is employed, only 3-5 colors should be used. In the current example, the “Assignment Schedule” webpage should use only two alternating colors to help visually separate rows of assignments. Instead of relying on color to help students associate same assignments across multiple pages, hyperlinks should be use to transport the user directly.

Nielson Heuristic 2. Match Between System and the Real World “The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.” Psychology 301 Related 1. Current order of HLN buttons does not follow user mental models. Location of Problem: All Pages Description of Problem: The high level navigation (HLN) buttons (in red on each page) do not reflect any noticeable order for their placement. Currently 10 navigational options exist. Some of the titles of these HLN options are redundant or ill-defined. This may confuse the user and tax their memory as to which button contains which information. Recommendation to fix Problem: (See Figure 2, page 30) 1. As mentioned above (in Aesthetic & Minimalist Design”) remove the “Schedule” button and move its information into the “Assignments” button 2. Separate the existing buttons into two groups in separate colors 3. The groups shall be related to each other as follows

a. Vital Class info i. Announcements ii. Syllabus iii. Assignments iv. Text, Readings v. Quizzes/ Surveys vi. Instructions (change to “ BeachBoard FAQs”) vii. External websites 2. HLN button titled “Instruction” does not effectively describe the information within it. Location of Problem: All pages (High Level Navigation) Description of Problem: The high level navigation button (HLN) titled “Instructions” does not effectively describe the information it contains which may cause users to accidentally select it, causing a navigational error. For example, although this button contains “instructions” on how to use the BB site, users may interpret the term “instructions” to mean specific class assignments… which is not the case. Recommendation to fix Problem: Change the button title from “Instructions” to “BeachBoard FAQ’s.” This title more accurately describes the type of information contained within this option. Nielson Heuristic 3. User Control and Freedom “Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.” BeachBoard Related 1. The users have no control over the presentation of announcements, instructions, and/ or assignments. (Unable to checkmark completed or “read” sections) Location of Problem: Announcement Page COURSES > PSY 301 PSY-DISCIPLINE & PROFESSION 3861... > ANNOUNCEMENTS

Description of Problem: Users are not able to visually indicate what announcement they have already read. Each time the user logs-in they are bombarded with an often long list of announcements that they have to pause and read to make sure nothing new has slipped in. This overloads the user, making BB a daunting task. Recommendation to fix Problem: Allow the user to either “check box” or change the background color for any announcement, instruction, or assignment. This will allow the user to quickly see what they have already read and not be FORCED into re-reading old notes. Also, once the item has been selected, the list order could be re-shuffled and the checked item could be sent to the bottom of its section. Or the section can be collapsed into its main heading. This will reduce the visual clutter of each page.

Nielson Heuristic 4. Consistency and Standards “Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.” Beach Board Related 1. The system does not inform user of saving on quiz/ test page when the “save” button at the bottom is selected. (It does inform user when individual questions are “saved”) Location of Problem: COURSES > PSY 301 PSY-DISCIPLINE & PROFESSION 3861 > QUIZZES/SURVEYS > ACCESS TO QUIZZES > TAKE ASSESSMENT 24 ABOUT ACADEMIA

Description of Problem: When taking a quiz on Psy 301, the users are able to save their answers in two different ways. First, the users may select the “save” button at the very bottom of the page to save all the answers on the page. Secondly, the users may select the “save” button adjacent to each question on the quiz. This option only saves that one answer and not the entire page. The problem exists in that the only the “save” button at the bottom of the page provides a pop-up window telling the students that the answers have in fact been saved. The “save” button adjacent to each question does NOT inform the users that their answer has been saved. This is a problem for two reasons. First, the action of the two “save” button options act inconsistently from each other. Secondly, since the “save” button adjacent to the questions does not inform the user that they have in fact saved their answer, it fails the heuristic to keep the user informed. Recommendation to fix Problem

Add a feature that will keep the user informed about the system status (e.g., a short message that tells the user what questions have been saved) such as the pop-up that double confirms your intent to submit responses.

2. Breadcrumb titles are inconsistently labeled Location of Problem: Anywhere in the site where there is a “Courses” breadcrumb Description of Problem: The breadcrumb titles do not use terms that accurately reflect where their link takes the user. For example, the breadcrumb titled “Courses” takes the user to different page than the tab courses. Recommendation to fix Problem The title of each breadcrumb should be exactly the same as the page title it refers to. Whenever a breadcrumb item is selected, the users should be taken to the webpage that accurately reflects the title. 4. Psy 301 utilizes a number of confusing symbols unique only to Psy 301. This ambiguity may confuse or mislead users. Location of Problem: Grades COURSES > PSY 301 PSY-DISCIPLINE & PROFESSION 3861... > TOOLS

Everywhere Description of Problem: A global symbol key that explains the meaning of each symbol to the user does not exist. Rather, the user is expected to innately understand the symbols’ meanings. The ability to make a global symbol key is possible because the “Grade” page currently has a grade related symbol key at the bottom. Recommendation to fix Problem: Users should be kept informed as to the meaning of symbols on the Psy 301 website. A number of options exist to correct this problem. First, using “tool tips” (which provide a brief description of the symbol via a pop-up window when the mouse hovers over a symbol) is a dynamic option that effectively solves this problem. Another option would be to create and post a system-wide key, or legend, that would inform the user about the 11

meaning behind each symbol. A third option would be to conduct a usability study with the primary users to determine how quickly they learn the symbols and/ or make suggestions for possible symbols to use. Psychology 301 Related 1. Currently Psy 301 uses the color red to symbolize a topic heading and clickable link. This format is misleading to the user and will cause navigational error. Location of Problem: COURSES > PSY 301 PSY-DISCIPLINE & PROFESSION 3861 > SCHEDULE

Description of Problem: BB instructs its users that when a topic is in red (on the instructions page) it is a clickable link with additional information within it. However, this is not standard throughout BB. In many cases the color red is used only to represent a topic heading and not a link, while blue links as scattered throughout other sections. Sections with links already contain a visual icon (a folder) to the left of it indicating that more information is available within. The color red may suggest an ERROR and disrupt the users flow. Recommendation to fix Problem: Since BB already contains the use of visual pictures (folders) to represent that extra information is available, BB doesn’t need to change the color of internal links to red. Instead use blue (the universal standard) to symbolize that a link exists.

2. Psy 301 uses inconsistent formatting throughout the website when presenting the user with additional information. Location of Problem: Menu tabs Description of Problem: The web pages are not always set up the same. Some have folders (which force the user to look inside) while other use only text to list all the assignments. Being inconsistent slows the users’ navigation through the site because he/she must remember the individual format for each page. Recommendation to fix Problem Description of the information within a folder could “pop up” when the mouse hovers over a tab option or “dropdown” when the user selects it. This would maintain a similar look and feel for all the pages and improve the scanability of the content provided.

3. Ambiguous icons used in the review section after the user submits a quiz. Location of Problem: COURSES > PSY 301 PSY-DISCIPLINE & PROFESSION 3861 > QUIZZES/SURVEYS > ACCESS TO QUIZZES > REVIEW ASSESSMENT: 24 - ABOUT ACADEMIA

Description of Problem: The review section, after a quiz, shows ambiguous icons that one must hover over to get an indication of what they may mean. Recommendation to fix Problem Show a key that will explain what the system icons represent.

Nielson Heuristic 5. Error Prevention “Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.” Psychology 301 Related 1. BB doesn’t inform/ warn the user that they must complete the entire quiz (can’t stop mid way) Location of Problem: Quizzes > 18a GRE

Description of Problem: The user can open the quiz w/out a warning that they must be able to take the quiz all the way through Recommendation to fix Problem: Before the user is allowed to continue, they should get an message that pops up and informs and asks them if they are sure they want to proceed because they will not be able to go back (which is found in the GRE quiz that can not be retaken)

Nielson Heuristic 6. Recognition Rather Than Recall “Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.” Psychology 301 Related 1. Options are not available when referenced Location of Problem: Syllabus section, Assignments section Description of Problem: The second link under the syllabus tab, references the Psychology Department website, however, it does not provide a link to get to that website. This violates the “Recognition not recall” heuristic because it does not provide the student with the option of transferring to the Psychology website to view critical information. Recommendation to Fix Problem: Instead of indicating to the student that this option is available, a link should be provided to the student that will redirect them to the Psychology website so that they can view the office hours.

Nielson Heuristic 7. Flexibility and Efficiency of Use “Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.” NONE Found

Nielson Heuristic 8. Aesthetic and Minimalist Design “Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.”

Psychology 301 Related 1. Schedule button is redundant Location of Problem: Schedule / Assignments pages COURSES > PSY 301 PSY-DISCIPLINE & PROFESSION 3861 > ASSIGNMENTS

Description of Problem: BB currently holds two High Level Navigational (HLN) buttons that contain the same information. This unnecessary clutter adds more visual information that the user must interpret to proceed. The Schedule and Assignment buttons contain the same information. Recommendation to fix Problem: (See Figure 2) Remove the HLN button titled “Schedule” and move the small amount of information under this button to the button titled “Assignments.” This will allow the user to locate necessary information, since now, all assignment relate information will be located in the same place. Plus, this will minimize the site “busy” appearance. 2. Heading titles are too long Location of Problem: Assignments page COURSES > PSY 301 PSY-DISCIPLINE & PROFESSION 3861 > SCHEDULE

Description of Problem: Under the “Assignments” button the sub-heading titles are too long. For example, “Instructions for Preparation for all required assignments.” Long titles disrupt the “scanability” of each section, forcing the user to waste valuable time reading each section. Recommendation to fix Problem: Keep the sub-heading title short and descriptive. For the example listed above, change “Instructions for Preparation for all required assignments” to “Assignment Instructions.” When the user hovers the mouse over this option a “tool tip” could pop-up with a more detailed description of the sub-heading. Keeping the sub-heading short allows the user to scan the page without being bogged down with unnecessary information.

Nielson Heuristic 9. Help users recognize, diagnose, and recover from errors “Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution. None Found

Nielson Heuristic 10. Help and Documentation “Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.” Beach Board Related 1. Give instructions from BeachBoard not CSULB Location of Problem: Everywhere Description of Problem: The BlackBoard service does not provide help documentation for their own site. Recommendation to fix Problem: Create an actual help section that is hosted on the BlackBoard server with instructions on how to use the website, rather than having the school create a help section. 2. No search textbox exists for BB Location: Throughout the entire application Description of Problem: No search textbox exists for users to directly use key terms to search the Psychology 301 site. Recommendation to fix Problem: Search textboxes are standards across many internet applications that allow users a more direct way to search for items in a site. Psychology 301 should employ this technique to aid in searching, especially since Psychology 301 contains a wealth of information that not easy to search through using Look See and Decide searching patterns.

Psychology 301 Related 1. A “Help” function is available only for beach board not PSYCHOLOGY 301. Location of Problem: Psy301 main page > Instructions > General Instructions – Click here for important information (This tag is not helpful)

Description of Problem: Help function available only for BeachBoard not PSYCHOLOGY 301. Help documents are not easy to find, they are under “instructions” where you’d think that you’d find actually assignment instructions. Instructions are more of a “how to” and addressing common problems. FAQs only address explanations to course rules. Recommendation to fix Problem A number of recommendation exist to address this problem: 1. Add a “Help” function for the course 2. Add walkthrough examples or pictures or dummy examples such as a fake quiz 3. Create a “Help” Chat box that allows the user to ask/ post questions. 4. Provide “Step Guides” that walk the user through the most common problems. 2. Instruction on using BB for Psychology301 is not easy to find (FAQ exist in “Instructions” button) Location of Problem: Throughout the application Description of Problem: Instructions for Psychology 301 users to use BB are difficult to find. A section that contains FAQ is used to satisfy this category. However, the “help” icon at the top of the page takes the user to a BB technical Help page with instructions for using BB. Recommendation to fix Problem: Psychology 301 should include instructions that allow users to effectively understand how to use the Psychology 301 site through BB. For example, the “instructions” should contain a link to the BB technical “help” page. The technical page should contain “howto-videos” similar to the videos found on the faculty technical help page.

References Lamantia, J. (2003). Analyzing card sorting results with a spreadsheet template. Retrieved May 1, 2008, from http://www.boxesandarrows.com/view/analyzing_card_sort_results_with_a_spreadshee t template Nielson, J. (2000). Designing Web Usability. Indianapolis, IN: New Ritters. Nielson, J. (2005). Heuristics for user interface design. Retrieved April 30, 2008 from http://www.useit.com/papers/heuristic/heuristic_list.html

Table 1 Performance Data for the Amount of Time (in seconds) it Took Each User to Finish each Individual Task.

User 1

User 2

User 3

User 4

Task Number

Difficulty of Task

Time (seconds)

Task 1

Medium

Task 2

Easy

Task 3

Hard

151

Task 4

Medium

Task Number

Difficulty of Task

Time (seconds)

Task 1

Medium

120

Task 2

Easy

120

Task 3

Hard

156

Task 4

Medium

Task Number

Difficulty of Task

Time (seconds)

Task 1

Medium

232

Task 2

Easy

151

Task 3

Hard

240

Task 4

Medium

251

Task Number

Difficulty of Task

Time (seconds)

Task 1

Medium

305

Task 2

Easy

340

Task 3

Hard

642

Task 4

Medium

453

Table 2.

Performance Data for the Number of Clicks it Took Each User to Finish each Individual Task.

User 1

User 2

User 3

User 4

Task Number

Difficulty of Task

Number of Clicks

Task 1

Medium

Task 2

Easy

Task 3

Hard

Task 4

Medium

Task Number

Difficulty of Task

Number of Clicks

Task 1

Medium

Task 2

Easy

Task 3

Hard

Task 4

Medium

Task Number

Difficulty of Task

Number of Clicks

Task 1

Medium

Task 2

Easy

Task 3

Hard

Task 4

Medium

Task Number

Difficulty of Task

Number of Clicks

Task 1

Medium

Task 2

Easy

10.5

Task 3

Hard

Task 4

Medium

Table 3. Symbol Recognition Data (Percentages calculated out of four users)

Symbol

Percent Correct

100 %

50 %

100 %

0% 75 % 50 %

100 %

75 %

100 %

50 % 25 %

75 %

100 %

Appendix A For the following SYMBOLS: 1. Describe what you think the symbols in the Left column MEAN to YOU. 2. You encountered the following symbols while navigating through BeachBoard. 3. If you are not sure of an items meaning, please write your best guess. Symbol

Meaning

In the Quiz section:

Throughout the website:

Appendix B For the following SYMBOLS: 1. Compare the symbols on the Left column with one another. 2. Chose the BEST meaning for the corresponding symbol. 3. Choose only ONE answer for each symbol. Symbol

Your Answer for the Symbol

Meaning

(Write the Letter)

A. Incorrect B. Clipboard

C. Document D. Page E. Calendar

F. Personal Information G. Announcement H. Partial Credit

I. Assessment J. Test K. Correct

L. Folder

M. Paste N. Grades

O. Edit Document P. External Link

Q. Tasks R. Open Book

S. Portfolio

T. Digital Dropbox

Appendix C

User Created Categories

BeachBoard Menu

Announcements

Syllabus

Instructions for Assignments

Instructions

Schedule & Assignment Information

Schedule

Assignments

N/A

Faculty/Staff

Reading Assignments

Texts, Readings

Quizzes/Surveys

Grades & Completed Work

Gradebook

External Websites

Additional Information

N/A

FAQ/Help

N/A

Incomplete Assignments

N/A