Milestone 3

Steelbeam Software

Boltri Eversman Hovanes Ray

Milestone 3 Contents Introduction

Introduction ........................................... 1 Widget Prototype ...................................... 2 - 6 Initial Evaluation Plan ................................ 7 - 9

After reviewing our research and analysis of each of our three designs, we have chosen to prototype the widget design and to offer this design as a solution to the FMCSA as a way to promote safety on our nation’s highways and aboard the 39,324 U.S. motorcoaches.1

Sources .................................................. 10

S t e e l b e a m

S o f t w a r e ,

I n c .

-2Widget Prototype

Figure 1: Mock-up of safety widget as it would appear on a motorcoach carrierʼs website (widget outlined in red www.greyhound.com A Note on Implementation Our widget prototype was implemented as an applet using the Java Swing library. Additionally, our design includes a simple user ratings site that was actualized using a Model-View-Controller (MVC) architecture in the Java programing language. This site also required the use of a database to maintain persistent ratings data. We used Xeround’s free MySQL database to store this data. 2

S t e e l b e a m

S o f t w a r e ,

I n c .

-3Our applet includes several features designed to increase the effectiveness with which the information is displayed:

Finally, at the bottom, the user can click the “More Information” button to be directed to the FMCSA Carrier Overview page for the carrier that they were viewing (see Figure 4).

Figure 2: DOT Score Panel

DOT Score Panel (Figure 2) The DOT Score panel has several components. At the topcenter is the title panel, which displays the name of the carrier and the title “DOT Safety Score”, to make it clear to the user what data is being displayed.

The overall score panel is center-left and displays the score as a large number. This size of the number is designed to catch the user’s attention and draw their view to this number. In addition, the background of this number is colored in pastel hue. The hue changes based on the score value and uses cultural norms to communicate quality. A score of 0 has a red pastel, while a score of 10 has a green pastel, with other scores being a gradient between red and green.

The other panel features individual scores for each of the BASIC categories. These scores are calculated based on the FMCSA percentile ranking using the formula (100 - percentile)/ 10. A value of “no violations” equates to a 10, and incomplete data is displayed as “n/a” and not used in calculation of overall score. For each category, the user can mouse over the label and see a description of that category (see Figure 3).

S t e e l b e a m

S o f t w a r e ,

I n c .

-4-

Figure 3: Within the widget, a user can mouse-over any of the five BASICS to see a description of the category. This allows the user to quickly understand the data we display in the DOT Score Panel.

Figure 4: The “More Information� button at the bottom of the widget links to the FMCSA Safety Measurement System site for the carrier that the user is currently viewing. We decided to incorporate the SMS site based on our user-centered design evaluation performed during Milestone 1 - we think that the site presents the information clearly. The only drawback, is that the FMCSA uses a different (and unintuitive) scoring scale.

Note- Unfortunately, the FMCSA API does not provide developers with this url. Thus, we have decided to store the url for each carrier in our Xeround database.

Obtaining the url is a challenge, however - the url is found only through a search on the site.

S t e e l b e a m

S o f t w a r e ,

I n c .

-5Figure 5: User Reviews Panel

User Reviews Panel (Figure 5) The User Reviews panel’s main feature is a gauge. This gauge uses a scale from 0 to 10 and is based on an average of user reviews; the gauge’s needle will point to the appropriate value to match this average.

Below the gauge is the numerical equivalent for the data displayed by the gauge. This number display is surrounded by glow effect, where the color depends on the score (similar to the DOT Panel’s overall score background).

Another feature of the gauge is that when the score drops below 3.0, a red indicator light will flash. This is intended to draw the user’s attention to the review panel because the score indicates that the carrier is potentially dangerous. The buttons on this panel link to a website where users can see a list of reviews for the carrier (see Figure 6) and leave a review (see Figure 7).

One other feature present in both panels is the use of a background image. These images, which depict the exterior and interior of a motorcoach, use the Gestalt principle of Figure and Ground to add variety to the background while still making clear what the actual content of the applet is.

S t e e l b e a m

S o f t w a r e ,

I n c .

-6-

Figure 7: User Review View Ratings Site The “See All Reviews” button links to the view-ratings page for the appropriate carrier.

Figure 6: User Review Rating Site The “Leave a Review” button links to the ratings page for the appropriate carrier.

7 people found this review to be helpful !

Did you?

Did you find this review helpful?

S t e e l b e a m

S o f t w a r e ,

I n c .

-7-

Initial Evaluation Plan The initial usability evaluation plan must analyze a set of user-interactions with our widget and ratings system. We will ask potential users to complete a series of benchmark tasks (listed below) to accurately demonstrate and evaluate the robust nature of our program. After completing the list of tasks, each user will submit a subjective questionnaire (containing the questions raised below) in order to provide an effective metric by which to judge our system. In addition to this questionnaire, an interview process will be completed both prior to use of our widget as well as after its use in order to find out more about our users and further evaluate the effectiveness of the widget. Also listed below are various other evaluation techniques that could be used to implement a thorough evaluation/testing plan.

Benchmark Tasks •

•

•

Find official FMCSA data for motor coach carrier - View the overall safety score (based on the BASIC scores) of a carrier - Determine each individual BASIC score for a carrier and determine whether each score is good or bad - View more information for that specific carrier (visiting the FMCSA Safety Measurement System page for that carrier) Create a review for a motor coach carrier - Set an overall review score for the carrier - Write comments about the carrier - Leave a name with their review Read other reviews for a motor coach carrier - Determine average overall score that the users have given a motor coach carrier - Determine which user reviews are more reliable than others - Vote a review to be helpful or unhelpful

S t e e l b e a m

S o f t w a r e ,

I n c .

-8Subjective Questionnaires • • • • • • • •

How How How How How How How How

aesthetically pleasing was the application? easy or difficult was it to navigate through the information? slowly or quickly did the application function? easy or difficult was it to determine the FMCSA safety ratings of the motor coach carrier? simple or complex was it to write your own review of the motor coach carrier? easy or difficult was it to use the gas meter to understand user review scores? easy or difficult was it to tell whether or not a user-written review was reliable? helpful did you find the information in the application?

Interview Questions •

Before completing benchmark tasks -

•

Have you ever looked up FMCSA motor coach safety ratings before?

• If yes, how easy or difficult to understand did you find the information?

Do you read reviews for products online before buying them? How much do they affect your purchasing decision?

Do you ever write reviews online for products that you have purchased?

Have you booked a charter bus trip in the past?

• If yes, tell us about your selection criteria. How did you choose a carrier?

Have you ridden on a charter bus before?

• If yes, tell us more about your experience.

After completing benchmark tasks -

Do you have a better understanding of motor coach safety after using the application overall? What about for the specific carrier?

How easy or difficult was the safety data to understand?

Do you think you would write your own review using this application, or would you most likely just read other reviews?

Would you rather use this rating system by itself, the official FMCSA data by itself, or both in combination? Of the two types of data, which was most helpful in determining how safe or unsafe a carrier is?

Was the application aesthetically pleasing? What would you change about the way it looked?

S t e e l b e a m

S o f t w a r e ,

I n c .

-9Other Evaluation Techniques •

We deemed the following techniques to be appropriate techniques for our purposes -

-

•

Interview • Great way to understand more about an individual who will use the application • Allows the individual to explain the issues of the application after using it Questionnaire • Able to gain a greater amount of information from a much larger sampling of people than an interview • If there is a statistical agreement on certain topics, the product can be modified to match those

We consider the following techniques to be less appropriate for our purposes -

-

Focus Group • Poses the risk of the group being dominated by an individual - an interview is a better way to talk to individuals using the product • Difficult to plan a successful focus group - especially in a college town where residents are busy and have odd schedules • Difficult to obtain participants without the ability to offer some kind of incentive • Difficult to get a different types of users in this area (Athens), therefore a majority would consist of students Research • We’ve already determined who out target users are, so further research isn’t a priority

We have developed a robust system that makes the most use out of existing resources. By embedding our widget into a motorcoach carrier’s website, we can ensure that the widget will receive the appropriate attention given it’s important safety data. We are excited to present our solution to our target users and to use their feedback to improve on out initial widget design.

S t e e l b e a m

S o f t w a r e ,

I n c .

-10-

Sources 1

" 2

Motorcoach Census 2011, American Bus Association, page 5, http://www.buses.org/files/Foundation/Final_Motorcoach_Census_2011_7-3-2012.pdf Xeround: The Cloud Database, http://xeround.com/

Steelbeam Software, Inc.

Mark Boltri

Dillon Eversman

Melissa Hovanes

Nathan Ray

S t e e l b e a m

S o f t w a r e ,

I n c .