User Research Lecture

Research!

Brad Soroka Design Manager, Product UX/UI at Digital Telepathy

dtelepathy.com/workshops

Writing a Research Plan.

Recruiting and Screening Participants.

Asking the Right Questions.

Conducting Interviews.

Synthesizing Interview Data.

Content.

Cheat sheet.

Questions.

#1: Writing a Research Plan.

“Usability testing isn’t just for experimental psychologists. What can I do to make it easier for people on agile teams?”

Dr. David Travis | userfocus

The Lean Research Plan.

Title.

Author.

Stakeholders.

Last updated.

Background.

Goals.

Methodology.

Participants.

Schedule.

Script.

Cheat sheet.

Tips for Writing a Research Plan Goals: 5-7 questions you are trying to answer.

1.

2.

3.

Always phrase as a question.

Get one level more specific.

Dig deep into a feature.

Participants: Traits /attributes of ideal candidates.

1.

2.

3.

Behavioral attributes work best.

Age & gender are less important.

Leave a link for the screener.

Background: 5 sentences on project history.

1.

2.

3.

3rd-parties understand.

Keep it to 3-5 sentences.

No jargon.

Methodology: How to go about your research.

1.

2.

3.

Quantity of interviewees.

Topics of the interview.

Examples of tasks for users.

Schedule: When events will happen.

1.

2.

3.

Leverage this workshops series.

Start with the Research Plan End with delivery of results

Questions?

#2: Recruiting & Screening Participants.

Recruiting agency.

Have your client do it.

Do it yourself.

Let’s DIY! How would you recruit participants?

Personal and social networks.

Recruiting sites.

Craigslist.

Screener.

10 questions or less to filter out participants and find the right ones.

Exclude early.

Do you, or anyone you know, work for Dropbox?

A) B)

Yes No

Beware of the hacks.

Beware of the hacks.

Screener questions.

Screen for behavior.

Open-ended.

Multiple-choice.

Select-all-that-apply.

Open-ended.

For how many years have you been using Dropbox?

MUST BE BETWEEN 3 AND 5 YEARS

Open-ended.

On an average day, for what activities do you use Dropbox?

LONG ANSWERS > SHORT ANSWERS

Multiple-choice.

How frequently do you upload files to Dropbox?

A) B) C) D) E) F)

More than once per week Once a week Once every 2-3 weeks Once every month Less than once every month Other

NO NO NO YES YES NO

Select-all-that-apply.

Which of the following activities do you perform on Dropbox: (check all that apply) [ [ [ [ [ [

] ] ] ] ] ]

Store documents Edit documents Comment on documents Share files Store photos Edit photos

MUST HAVE

Scheduling.

Date & time.

Does Friday, April 1 at 8:30am PT still work for our interview?

Location.

Does Friday, April 1 at 8:30am PT still work for our interview?

We’ll meet on GoToMeeting here. You will need to install a plugin beforehand.

Permission to record.

Does Friday, April 1 at 8:30am PT still work for our interview?

We’ll meet on GoToMeeting here. You will need to install a plugin beforehand.

With your written permission, we’d like to record this interview for research purposes. Is this OK?

Cheat sheet.

Hire a recruiting agency.

A good recruiting company can save you a lot of time and effort, often while providing you with the best participants. This is the most expensive option, but it removes much of the heavy lifting.

If your team is short on time and has the budget of $100-$150 per participant (in addition to participant incentives), a recruitment agency can handle the initial calls, screening, and appointment scheduling with participants. Using an outside recruiter does mean that you give up some control over the process and some insight may be lost without an ongoing relationship with the respondents. â—?

â—?

Panel agencies: Sites like UserZoom.com and Entrust provide you with access to thousands of recipients with huge databases with extensive demographic information.

Panel agencies are useful for large surveys and usability testing projects.

Market research recruiters: You may want to use a professional recruitment firm when there are very specific participant requirements or skill sets for in-person testing.

Have your client do it.

Sometimes it’s easier for your client to recruit participants, especially if they can use their own employees or their customers. Your client already has all the information they need to select the best users for the interview, and employees or customers may be more inclined to participate if the invitation comes from a contact.

Recruiting through your client gives away some of your control over the process and is reliant on client resources and willingness to be involved. To set your client up for success here remember to: ● ● ●

Provide: Do your best to clearly define a participant profile and session schedule.

Create: Write a script detailing what information to give participants about the study.

Monitor: Check frequently with recruited participants to ensure that the appropriate criteria are met, and the interviews are properly scheduled.

Do-it-yourself.

The least-expensive approach is recruiting research candidates yourself. It may take more of your time, but you can often complete the whole process faster yourself than a vendor can. In addition to client employee or user directories there are various inexpensive online services that can help source participants. ●

● ●

Direct: Adding a “Interested in participating in a survey?” link to your website, product, or email signatures,are low-friction ways to recruit. It might take longer to build a database of users this way but will set you up for future testing.

Recruiting sites: Your best bets are survey-specific website services such as User Testing and Ethn.io. However, recruiting over social networks can work as well.

Personal network: For general purpose websites, coworkers or friends who are unfamiliar with the application you are testing can work well for broader interaction testing. For more specific user types try reaching out to appropriate professional associations, or community / student groups.

Open-ended

Multiple-choice

Select-all-that-apply

Open-ended questions are a good way to help you understand how the participants articulate themselves, or to not give away answer.

These are great for understanding a specific attribute of a user. It’s a quick way to pick a single criteria from a range.

We’re providing the user with several options, of which they can select multiple. A specific selection must be included for a participant to advance.

Use with “Exclude early”

Use with “Define Time Ranges Clearly”

Use with “Screen for Behavior”

Examples:

Example:

Example:

For how many years have you been using Dropbox?

How frequently do you access files located on Dropbox?

Which of the following activities do you perform on Dropbox: (check all that apply)

On an average day, for what activities do you use Dropbox?

A) B) C) D) E) F)

More than once per week Once a week Once every 2-3 weeks Once every month Less than once every month Other

[] [] [] [] [] []

Store documents Edit documents Comment on documents Share files Store photos Edit photos

Questions?

#3: Asking the Right Questions.

Avoid leading questions.

How easy was it to _________?

Does waiting in long lines at the DMV make you frustrated?

How easy was it to _________?

How easy or difficult was it to _________?

Does waiting in long lines at the DMV make you frustrated?

How easy was it to _________?

How easy or difficult was it to _________?

Does waiting in long lines at the DMV make you frustrated?

Tell me about the last time you waited in line at the DMV.

Avoid the word “would.”

Discuss past behaviors, not future ones.

If we built this product would you use it?

How much would you pay for this?

What would you do in this situation?

If we built this product would you use it?

Tell me about the products you use?

How much would you pay for this?

What would you do in this situation?

If we built this product would you use it?

Tell me about the products you use?

How much would you pay for this?

How much do you spend on these products?

What would you do in this situation?

If we built this product would you use it?

Tell me about the products you use?

How much would you pay for this?

How much do you spend on these products?

What would you do in this situation?

Tell me about the last time you were in this situation.

Have them tell you a story.

What is the last movie you saw?

What is the last movie you saw?

Tell me about the last time you saw a movie. ...you mentioned you rarely go out to movies, can you talk more about that?

Ask the “5 Why’s.”

Do you use a subscription service?

“Yes, spotify”

Do you use a subscription service?

“Yes, spotify” Why do you use spotify over other services?

“I like their playlists.”

Do you use a subscription service?

“Yes, spotify” Why do you use spotify over other services?

“I like their playlists.”

Why do you like their playlists?

“Because they have a huge variety”

Do you use a subscription service?

“Yes, spotify” Why do you use spotify over other services?

“I like their playlists.”

Why do you like their playlists?

“Because they have a huge variety” Why is variety important to you?

“It allows me to discover new music”

Pop quiz!

Can you spot the issue?

Do you use a subscription service?

“Yes, spotify” Why do you use spotify over other services?

“I like their playlists.”

Why do you like their playlists?

“Because they have a huge variety” Why is variety important to you?

“It allows me to discover new music”

Can you spot the issue?

Do you use a subscription service?

“Yes, spotify” Why do you use spotify over other services?

“I like their playlists.”

Why do you like their playlists?

“Because they have a huge variety” Why is variety important to you?

“It allows me to discover new music”

Cheat sheet.

Question bank.

What is the hardest part about __________?

Tell me about the last time that __________?

What have you tried to solve this problem? Tell me more about that.

Describe a typical __________?

You mentioned __________, tell me more.

How many times do you __________?

If you had to teach someone __________, what would you say?

Has __________ happened to you? What did you do?

Questions?

#4: Conducting Interviews.

Before the interview.

Test your setup.

Primary & backup.

Wifi.

Audio & video.

Screen capture tools.

Determine team roles.

Writes

Speaks

Does not write

Does not speak

Writes

Speaks

Does not write

Does not speak

Facilitator.

Writes

Scribe.

Speaks

Does not write

Does not speak

Writes

Observer.

Speaks

Does not write

Does not speak

During the interview.

Describe format and length Explain the purpose.

Ask for raw, unfiltered responses.

Focus on objectives over questions.

Ask follow up questions as they arise.

Use “tell me more.”

Use body language.

Smile! :) Active listening.

Stay open.

Build rapport but be careful when talking about yourself or relating to them.

Don’t be the expert.

Make them feel like the expert.

Play dumb.

Ask questions you know the answer to.

Use trailing questions. Wha…..?

After the interview.

Is there anything we didn’t ask?

Do you have any questions for us?

Can I follow up with you?

Note notation.

Note notation. “ ”

Verbatim quote.

“I love Custom Reports...I use them 2-3 times per week”

Note notation. “ ” ★

Verbatim quote.

Key moment.

★ Custom Reports are great, but it’s cumbersome to run them every 2 days.

Note notation. “ ” ★ #

Verbatim quote.

Key moment.

Theme.

#repetition User talked about running reports multiple times for multiple questions.

Note notation. “ ” ★ # //

Verbatim quote.

Key moment.

Theme.

Comments.

Reports are awesome but tucked away. // I wonder if they’re valuable enough to elevate to the top-level?

Post-interview recap.

Identify Top 3 things.

Convert to digital notes. ‘Thank you’ notes and gift cards.

Cheat sheet.

During & after the interview Body language.

Here are 3 pointers to make sure that you are demonstrating to the participant that you are actively engaged: ●

●

●

Smile: You want to appear friendly and welcoming from the get-go.

Active listening: Say “mmhmm” at interesting points, maintain eye contact, and nod when the participant is speaking.

Stay open: An open posture with arms open, and body square, can indicate engagement.

Note notation.

Below are 4 handwriting notations to help speed up notetaking, and effectively communicate insights in your notes. ● ● ● ●

“ ” : Indicates a verbatim quote from the participant. ★ : Indicates key moments, insights or quotes. # : Indicates a recurring conversation theme. // : Indicates a comment, usually after a statement from a participant.

Post-interview recap.

After completing the interview, there are a few steps left to ensure that your team retains all of the information that you uncovered: ●

●

●

Top 3 things: Meet as a group to determine the top 3 insights from the interview.

Digital notes: Type your notes.

Add a link to the recording, screenshots, and Top 3 things.

Send a ‘Thank you’: Send a handwritten ‘Thank you’ note to your participants. Have all team members sign it.

Questions?

#5: Synthesizing Interview Data

How do we know when we are ready to start analyzing data?

When we are hearing the same things over and over.

Read transcripts line by line.

Write Observations.

Research goals in hand.

Observations on the wall.

Arrange into groups.

Name your Themes.

Brainstorm opportunities & insights.

Tedious to send custom reports to manager

“I wonder if this means…”

“It would be game changing if…”

Brainstorm opportunities & insights.

Tedious to send custom reports to manager

I wonder if this means‌

Managers have a recurring need?

Final results.

Questions?

Wooooohooooo!

Recap.

Writing a Research Plan.

Recruiting and Screening Participants.

Asking the Right Questions.

Conducting Interviews.

Synthesizing Interview Data.

dtelepathy.com/workshops Product Workshop, Friday 10/14 25% Off Promo: SDSU25 Brad Soroka | brad@digital-telepathy.com