DESIGN METHODS from analysis to evaluation
KATIE WILLIS
ANALYSIS
CASE STUDY — BACKGROUND
methods 1—4
In the Fall of 2013, the Visual Communications Senior class at Herron School of Art and Design had the task of choosing a place to observe and conduct research at throughout the course of 3 months. Each week, research would be conducted in one of three phases; analysis, synthesis, or evaluation. This research followed strict guidelines of a method chosen by the design team in charge of conducting the research. In total, there were 8 weeks worth of data collected — meaning 8 methods chosen and used.
SYNTHESIS methods 5—7
EVALUATION method 8
This book of methods is based on a case study of The Monon Trail in central Indiana. The Monon Trail is an exercise and leisure trail that spans from the heart of the state, Indianapolis, to a few suburbs North. The team of visual designers who worked on this case observed problems between city drivers and trail users and then found several opportunities to improve upon existing trail features and interactions. They eventually narrowed down on one. From analysis to evaluation, this methods book covers a set of design research methods that can be adapted / implemented in to your own design research.
user observation database method 1 Organizing and sharing observational data and insights from projects
This method should be chosen because of the vast amount of information that can be derived from using it in the correct context. In spaces that are larger, observation can be a great tool to use when needing to identify a lot of information at one time. The details of an observed place can be captured as well as behavioral patterns. When trying to identify initial problems within a certain area, observing will help serve as a great tool. Observing a place can help you see a number of problems after documenting them. If this is done correctly, observations can give you an endless amount of information that will pertain to an opportunity space.
TIPS
INSIGHTS — HOW TO
+ Be unobtrusive + Become familiar with the setting before collecting the observed data + Pay attention to details + Make the shift from a wide to narrow perspective + Listen to conversations around you carefully + Be tolerant of ambiguity + Reflect on findings
Although observation can be a great tool for facilitating the absorption of a lot of information, it can only achieve that goal if it is carried out correctly. Before going out in to the field, make sure you have everything that you need in the way of tools and information needed. Doing initial research before going out in to the field will help you observe more efficiently and effectively.
TOOLS + Photography + Video camera + Field notes
Pay close attention to the big and the small pictures when you’re observing. It may be easy to overlook some things, so step back for a minute or two while trying to look at the observed space from a distance. This will help you in the determination of where to focus your efforts next.
WHAT IT DOES Through the user observation database process, different kinds of information will be gathered and will then need to be analyzed and synthesized. These two processes are a very integral part to making sure the method of observation is both successful and meaningful. With conducting observation, we’re able to see the strong, comparable, relationships that are formed between the sociocultural behavior one observes and also the physical environment. Through the synthesis phase, we can determine what is important about that relationship and what is worthy of looking in to further.
ethnographic interview method 2 Having conversations with people about their daily lives and contexts
This method should be chosen because ethnographic interviews help to build empathy within the interviewer. In addition to you building empathy these interviews will help to focus on a certain experience with little distraction. This method also promotes you learning in context by interviewing people in the setting that a certain activity is taking place at. When conducting an ethnographic interview, it should take place in the same location where the activity being discussed took place. This allows for less bias when interviewing the subject and also allows the conversation to be more direct and less abstract. Along with these ideas, conducting the interview at the location of activity helps aid peoples’ memory.
TIPS
INSIGHTS — HOW TO
+ Ask open ended questions + Become familiar with the setting beforehand + Pay attention to details + Establish positive relationships + Be empathetic + Reflect on findings
Before going out to conduct your ethnographic interview, make sure that you’ve put together an interview protocol. This protocol is a detailed plan of whom you will visit, whom you will talk with, how long you will be there, what you plan to ask, and how you will function on your own or with a partner. After this protocol is put together, it is important to assemble a kit that you’ll take in to the field with you. This assembled field kit should include everything that you anticipate needing such as cameras, note taking materials, audio recorders, etc.
TOOLS
When you get to the site that you will be conducting your interview at, it is important to establish positive relationships with those you’ll interact with. You’ll need to make your participants feel comfortable and also explain everything to them about the interview process. Lastly, capture everything that you gather. You should be writing down what your participants say and do. Be sure to keep this data for analysis later on.
+ Photography + Video camera + Field notes + Tool kit
WHAT IT DOES Ethnographic interviews are concerned with understanding the peoples’ activities and experiences from their own perspectives. It allows researchers to learn about people in their own words. Observations are told from the participants’ point of view.
behavioral mapping method 3 Used to systematically document location-based observations of human activity, using annotated maps, plans, video, or time-lapse photography
This method should be chosen because behavioral mapping is a traditional and exploratory way to collect qualitative observational data. As a tool, behavioral maps are used to document observable characteristics, movements, and activities of people at fixed locations or in transit to another location. Behavioral maps also allow you to document approximate ages and genders, and the details of environmental contexts.
TIPS
INSIGHTS — HOW TO
+ Be descriptive in note taking + Become familiar with the setting beforehand + Reflect on findings + Be patient + Remain unobtrusive
There are two kinds of behavioral mapping that you can use in order to conduct your observations. The first, is place—centered mapping that is based on the observations of people in certain site—specific locations. Researchers will use their own designed diagram to help understand the surroundings. Designers will need to include signage and any fixed or portable items that may affect peoples’ behaviors or interactions. The key point of these maps is to determine key points of interaction and improve the space design or the service flow.
TOOLS + Annotated map + Time-lapse photography + Video
Individual—centered mapping is the second of two types of behavioral mapping that can be used. This mapping follows the activities and actions of a specific person. The purpose of this mapping is to learn about people instead of learning about the use of a particular space.
WHAT IT DOES Behavioral mapping allows designers to learn about people and places that they inhabit. This can either be done separately or in combination with one another. This method is a traditional qualitative way to collect exploratory observational data.
think—aloud protocol method 4 Requires participants to verbalize what they are doing and thinking as they complete a task, revealing aspects of an interface that delight, confuse, and frustrate
This method should be chosen because the think—aloud protocol is a great way to get both quantitative and qualitative data in an evaluative, self reporting, and observational way. This protocol is used often because of how straightforward it is. Having participants think out loud is key to getting them to articulate what they are thinking, feeling, or doing. As designers, we ask our participants to complete a set of tasks that align with their daily goals.
TIPS
INSIGHTS — HOW TO
+ Ask questions + Encourage conversation + Pay attention to details + Establish positive relationships + Reflect on findings
Before conducting the think—aloud protocol, it is important to have a set list of questions that you will ask the participants in the field of research. If conducting this method with more than one participant, having a set list of questions ready beforehand will streamline the entire process and make your data easier to compare. In addition to having set questions ready beforehand, the entire time spent with your participant should be planned out. You may have to repeat certain steps based on the participants, but if you have a set schedule of your activity before, it will make comparisons that much easier afterward.
TOOLS + Photography + Video camera + Field notes
Depending on your participants, you may have to remind them multiple times to verbalize their thoughts, feelings, or actions. There will be instances where people won’t feel natural talking out loud about what they’re doing, so come prepared with prompted cues to initiate verbalization. Participants, in general, shouldn’t have issues with speaking about and completing a task at the same time.
WHAT IT DOES The think—aloud session allows participants to verbalize what they are doing and thinking while completing tasks. Knowing what they are doing and thinking can help you, as the designer, to know where they have issues with certain tasks and situations.
activity network method 5 Structuring activities of stakeholders and showing how they relate to each other
This method should be chosen because of it’s ability to group research based on relationships and comparisons. An activity network method allows us to relate the activities of stakeholders in the context they are studying and group them together. After the clustering of activities is complete, that data can be turned in to a network diagram that shows all of the activities of the stakeholders together. This, in turn, shows their overall inter—relationships.
TIPS
INSIGHTS — HOW TO
+ Be comprehensive + Reflect on findings + Pay attention to patterns + Don’t ignore any connections
In order to begin your activity network, you first must list peoples’ activities that you want to compare. These activities can come from all of the stakeholders in the study. After you’ve listed activities to compare, you’ll need to determine the similarities between these activities. You can do this by determining a scoring scale that can be used to measure relationships amongst these stakeholders. This scale can have a range from no relationship to a maximum relationship with medium ranges in between. After this scoring scale is in place, create a symmetric matrix; one where activities can be entered in both row and column headings. These cells in the matrix will represent relationships between activities.
TOOLS + Large wall space + Markers + Paper or white board
Once the activities are entered in cells, you will use your scoring scale to score the relations. If you can, score these relations as a team so as to reduce bias. After the scoring is complete, you and your team will need to sort the completed matrix so that similar scores are next to each other. It is important to keep similar scores next to each other so that you can identify clusters in these next steps. When identifying clusters, see how many you can visually identify based on differing colors. Define and label those clusters once they’ve been identified. Ask questions and define these as a group. After this is complete, you will need to create a network diagram by making nodes that will represent each cluster and drawing lines between them to identify relations. In the end, make sure to capture your insights and discuss, as a group, your findings.
WHAT IT DOES Activity Networks allow you, as a designer, to see connections amongst entities and find the importance amongst those newly identified relationships.
ideation session method 6 Engaging people in contextual activities, observing them, and later interviewing them
This method should be chosen because ideation sessions allow for you to understand how people will respond to an actual situation by having them ideate and become engaged with the solution making process Ideation Sessions allow for designers to identify a specific user group and research participants. These sessions allow designers to test early assumptions and find out more about user needs.
TIPS
INSIGHTS — HOW TO
+ Ask open ended questions + Pay attention to details + Establish positive relationships + Be empathetic + Reflect on findings + Set up beforehand + Be hospitable
Before starting with your Ideation Session, you will need to identify your user group and research participants. You will also need to know what sort of specific user behaviors you will want to study during the session.
TOOLS
When participants take part in these activities, you will need to observe them engaging with them. Decide how you will capture these observations and make sure it is unobtrusive. However, you will need to make sure everyone knows beforehand that they are being observed and documented for the purpose of research.
+ Photography + Video camera + Tool kit
Once you’ve gathered your participants, establish what activities you will want them to partake in. Engage users in activities that will help you look at your desired experience more closely and aid in getting results you’ll need.
After participants are done with the activities, conduct interviews to learn about their opinions and responses to what you conducted in the activity. Have some set questions prepared in case they’re not inclined to talk freely. Document this in an appropriate way. Lastly, let your participants know what your next steps are and ask them if they would be willing to help you in the future. You never know when you may need them again!
WHAT IT DOES Ideation Sessions allow for designers to identify user needs and have users help with ideating on solutions for a set problem. These sessions focus on experience and help promote learning amongst the design team as well as participants.
insights sorting method 7
This method should be chosen because insights sorting is a very cohesive way to gather the insights generated from research and sorting them into agreed—upon clustering logic.
Manually sorting insights from research to find clusters and hierarchies
Insights Sorting can reveal clustering patterns and allow us to analyze these clustered patterns to better understand the topic and provide a strong foundation for generating design concepts.
TIPS
INSIGHTS — HOW TO
+ Ask open ended questions + Become familiar with the setting beforehand + Pay attention to details + Establish positive relationships + Be empathetic + Reflect on findings
In order to begin, you will start by gathering insight statements that you have generated from prior research. Insights are simply interpretations of what you have observed in your previous research about people and the contexts they were in. Write these insights out on sticky notes if possible. In a team, begin to cluster these insights based on a set sort of logic that you’ve come up with. Discuss this logic with your team. One sort of logic that you may use is similarity.
TOOLS
After you and your team have agreed upon your set logic, cluster and re-cluster your insight statements until you have reached a set clustering pattern that you’re happy with.
+ Photography + Video camera + Field notes + Tool kit
Lastly, you and your team will need to define these set clusters and recognize why your insights are grouped in the way that they are. Discuss your next steps as a team.
WHAT IT DOES Insights Sorting allows you and your design team to identify insights gathered through research and cluster them based on relevancy and commonality.
solution enactment method 8 Acting out solutions to demonstrate how they work and create value
This method should be chosen when needing to present design solutions to an audience in order to demonstrate how they work and create value for the stakeholders. Solution Enactment is a great tool for communicating abstract ideas into terms that audiences can understand and relate to. This method works well in demonstrating details of a solution and the solution as a whole.
TIPS
INSIGHTS — HOW TO
+ Ask open ended questions + Establish positive relationships + Reflect on findings + Have multiple solutions
Before getting started on enacting your solution, try to envision your user’s journey. Imagine a situation in which your designed solution exists. Try to imagine how your user interacts with this solution and how that translates to how they feel while interacting with it — sketch this out.
TOOLS + Photography + Video camera + Tool kit
After sketching how you believe your users will interact with your solution, explore a range of possible encounters the user will take with it. Explore a wide range of interaction points and experiences that could arise. Once you’ve explored a wide range of encounters and interaction points, be sure to rehearse what you will do in front of your users. Anticipate questions and have prepared dialogue ready. After you have practiced, you’ll be ready to present your solution enactment to your stakeholders. Be sure to document key points during the enactment. In addition to capturing key points that you make, be sure to capture the feedback from the stakeholders and discuss your next steps with them.
WHAT IT DOES Solution Enactment allows designers to demonstrate to the stakeholders how their solutions work and get feedback on what they could do differently.