USER STUDY REPORT
TIMOTHY AU STELLA GUTIERREZ OBI VATTANAWONG MELISSA WONG
TABLE OF CONTENTS OVERVIEW.........................................................01 USER TEST GOALS............................................01 USER TEST LOGISTICS A. BRIEF OVERVIEW..............................02 B. TARGET USER GROUP.........................02 C. BACKGROUND INFO ON USERS.............02 D. NUMBER OF PARTICIPANTS...............02 E. INFO SOUGHT FROM USERS...............03 F. RECORDING DATA..............................03 INTERPRETING THE RESULTS...........................03 RECOMMENDATIONS & REVISIONS......................04 APPENDIX A: USER TEST FORMS
OVERVIEW
The Spring 2012 User Test brought a different focus for our project. While staying within the same scope of the intentions of our project, the physical prototype differed greatly from the test ran in Fall 2011. Circo is a limited interaction table-like surface meant to be installed in a large conference setting in which participants can choose to connect and compare with other individuals their skill sets at any time. The physical prototype consisted of a round table about 4 inches in diameter with RFID readers placed in four specific spots, equally spaced around the circumference of the table. The table was powered by an Arduino which communicated the values received from the RFID reads to a database and displayed visual representations of participant data via a ceiling mounted projector. Processing was the program used to generate the visual graphics.
USER TEST GOALS
Due to the substantial modification our project had gone through since the first semester, we had many new learning goals and uncertainties, and that by running this user test we hoped would answer and refine our project. These learnings would help us shape and form our final prototype with user specific feedback in terms of how the table functioned and whether or not there could more suitable alternatives. In terms of the physical table, we needed to give a form that could best be intuitive and afford and maximize interactivity. The use of the RFID technology needed to be obvious in terms of how it would facilitate the whole table. However, it must remain a secondary priority to users as the technical elements are not the intended focus of this project. The second goal specifically addressed the graphics representation the project was able to produce. It was known that the graphics needed to be simplistic so users would understand their representation immediately while maintaining a depth of data, in which they would be able to compare and contrast it to others. The third goal addressed the strength of interaction between the users and the physical table. What engages the users with our table? What will keep them interested? What will cause them to leave? These are all questions that may not largely effect the form but how the table was able to facilitate its own dialogue.
01
USER TEST LOGISTICS
A. BRIEF OVERVIEW The test was delivered in the Green Screen room at the SFU Surrey campus. Each test session was 15 minutes long which was sectioned into three different parts—all of which were critical to our evaluation of the prototype. The three sessions were the Identity Creation Station (the survey and data collection), the interaction and use of the table and lastly, a short one-on-one interview with a member from the project team. Users were asked to fill out a survey sheet (Appendix A, Image 1.1) with the guidance of instructions displayed on a computer screen (Appendix A, Image 1.2) which was then inputted into an excel file by the team. Upon completion of inputting the data, participants were given RFID tags and allowed into the space where Circo was situated. Participants were not given any instructions or explanations with the exception that they may leave the test when they felt they understood what the project was about and did not have much more to learn from it. Once all the participants had exited the space, they were asked to participate in an interview session. A 16 post-test questionnaire was constructed beforehand to maintain a consistency when conducting the post-test interview.
B. TARGET USER GROUP Due to the wide range of possibilities of skill sets available to be inputted into the system, we felt that covering all types of people would be too large for our user study. Hence we decided to focus our attention on collaboration, self-learning and team building for creatives in Vancouver. This allowed us to limit the possible skill sets to better facilitate engaging and meaningful experiences for our target user group. Our target user group were university students, and those with professions or experiences related to arts, design or technology with a curiosity in collaboration. During the test, we also received a few participants who were not familiar with design and technology, giving us a nice contrast in how they saw the project.
C. BACKGROUND INFORMATION ON TESTED USERS In order to fulfill the requirements of our test, users were given a page of questions asking for their previous experience in the areas of art, design, technology, as well as their goal as a creative. We asked them to rate their familiarity with technologies, their previous conference experiences, and team building experiences while all remaining anonymous. All personal information was to be destroyed upon completion of the project.
D. NUMBER OF PARTICIPANTS Since the goal of this user test was to understand the types of interactions users engage in with the system and with others, we only had a limited amount of RFID tags that our base system could handle. Tests were repeated once the system was in place, therefore we conducted two very similar tests with different numbers of participants each time having a different focus for each. Tests consisted of four to five users at a time in three different rounds.
02
E. INFORMATION SOUGHT FROM USERS First we tested the initial process of setting up an identification tag and inputting participant information into the system to generate their corresponding graphic that was to be projected onto the display surface. We believed four participants per test was sufficient in terms of helping us identify issues with the timeline of events and placements of elements, and helping us identify their joys and frustrations if any, and understanding of what is being done. For the second test, we wished to focus on the input aspect of our system where participants were encouraged to give subjective ratings and detail words to describe themselves. We wanted to see how well they felt their identity wasrepresented from the simple rating system, and the shape computationally created. Finally, the third test was to identify which table shape was ideal for our purposes. There were a few shapes explored, and we decided it would be between a square or circle. For our working prototype in the user test, we decided a circular table would be more appropriate as there were no constraining sides, thus not restricting the amount of people that can naturally gather around the table. With a circular surface, we wanted to see how it would impact the participants’ interactions with each other.
F. RECORDING DATA Initially, a questionnaire was given as a screening process for our users and to gain an understanding of their experiences and thoughts prior to the test. During the test sessions, one team member was a designated facilitator for the participants, while the others observed participants as individuals, the interaction between each other as a group and the group with the table. Two cameras were set up; one top down view of the table, and another side view on the group, for post-test analysis. There was no need for a speak-out-loud protocol as the project was not testing the interface, but more their personal reflection, reaction and outside understanding they gain after their experience. Also it was more natural for the users to experience the project without disruptions of the testers involvement, hence the post interview that was conducted with each participant. The semi-structured interview consisted of close and open ended questions as we wanted reflections that were more qualitative than quantitative.
INTERPRETING THE RESULTS
As part of the post-test analysis, there were a few key findings that helped us shape revisions for the final step of this project. Drawing from the post interview answers given by the participants, many stated that the motion of placing their RFID card on the table was memorable and that they enjoyed the intimate environment that was created. The visual graphs were found to be interesting and gave the impression that they as participants had to really market themselves. While the graphics became a representation of that person, there was concern for the standards in how participants rated themselves. Additionally, the graphics did not necessarily reveal a person’s personality. There were a few concerns over how the information gathered was arbitrary and some had wished there would be more numerical-based questions. Overall, participants enjoyed using the table a communication tool with others and that they felt like it helped facilitate conversation with those who they had never met.
03
RECOMMENDATIONS & REVISIONS
Based on our findings from the user test, we decided to implement the following revisions in the final iteration of the physical prototype. All of the revisions address and reflect necessary changes to better suit the needs of the users while maintaining the intentions of our project. In the final prototype of our project, we would change the visual graphics so that they speak more about the person beyond their skills such as their personality. The idea is that we would have a blank space underneath each attribute where they could write words related to the attribute specific to them. Some ways to execute this would be to have words fade in and out around random points on the corresponding axis. A second method is to have a dot traveling through the lines in the graph. Once the dot hits the intersecting line on the graph, a word (randomly selected out of the words the person listed) would fade in and then fade out when the dot travels again. We believe that this would allow the graphic to become more of a true representation of the individual and help facilitate on-going conversations - something that we cannot specifically design, but can design affordances to help facilitate this. To standardize how people would rate themselves, they would now be restricted to 'points' that they can allot to each attribute. For example, they would have 100 points to distribute over 6 attributes. Allowing users to distribute a specific number of points towards themselves, they can properly gauge a more accurate projection of themselves as a whole instead of individual attributes. Additionally, we do not feel like we need to address how we gather participant information as it would not be arbitrary in its intended setting (a conference setting). The information would be gathered well ahead and would be inputted into the system prior to the conference. It is important to us that the initial interactions with the table are not guessed, thus we want to make an obvious connection between sensors and tags. The users would then know how to use the table and the cards together as a unified object. The idea is to have a logo in the sensor areas, and repeat them on sleeves for the tags, or if in the case of a cup, the mark or logo will appear on the sleeve of the cup.
04
APPENDIX A
FORMS USED IN USER TEST
IMAGE 1.1 NAME: _________________________________ TITLE OR BRAND: _______________________ ie. Least ___________________________ Most 1. 2. 3. 4. 5. 6.
Least ___________________________ Least ___________________________ Least ___________________________ Least ___________________________ Least ___________________________ Least ___________________________
Most Most Most Most Most Most
________________ ________________ ________________ ________________
Color#______________
________________
PARTICIPANT#______________
Tag#______________
IMAGE 1.2 Please Fill in your First & Last Name and give yourself a Title or 1-2 words that describe you the best.
attribute in relation to the design context.
1. Creativity 2. Communication 3. Experience 4. Teamwork 5. Problem Solving 6. Logic On the chart below, please choose 5 attributes that best describe yourself and write them in spaces provided
Precise Accurate Concern for Quality Critical Listener Non-Verbal Communicator Attention to Detail Creative Slow Start/Fast Finish Indecisive Temperamental Competitive
Confrontational Direct Results-Oriented Sense of Urgency Change Agent Process-Oriented Quick to Change Independent Optimistic High Trust Level
Not Fearful of Change Contractability Rather Talk than Listen Verbal Skills Good Supporter Team Player Persistent Cooperative Sensitive to Others’ Feelings
Accomodating Dislikes Confrontation Persistent Controlls Emotion Adaptable Good Listener Product-Oriented Slow to Change Self-Disciplined Pessimistic