A blueprint for User Experience Research (UXR)

UXR is a term which labels a systematic approach to better understand your users: who they are, their needs & problems, and their desires & wishes. Doing UXR well let’s you effectively identify potential solution spaces which you can start find functional requirements for. Those requirements can be used to start solution finding, and build great products around. In some ways, UXR is one of the most foundational, critical, and impactful parts of Product Management.

So why aren’t we all doing UXR all the time? Well, most of us do. PMs are spending a ridiculous amount of time talking to their users. However, good UXR is expensive and hard to staff, especially if you are a young company. It can also be effectively outsourced, either to other roles (have the PM pick it up), or to external companies (such as this one). But all of this doesn’t mean that it is easy to do, or results will be of a useful and high quality.

If you are a smaller shop, and you’ll like to do UXR well, here is an interview-based UXR blueprint which I found super helpful. It can be implemented with almost any x-functional team quickly. It provides real results in a systematic way and allows you to persist structured information for months. It enables you and your team to augment critical product decisions with real user data. This blueprint is deeply informed by my 10+ years at Google as a PM, and combines best practices I have seen while working on products such as Google Search, Google Analytics, and Google Ads. Ready? Let’s get started!

Step 1: The research brief

In your first step, you need a research brief. It is a document capturing the core hypotheses (the thing you believe to be true), and the associated questions which help you uncover whether you can accept (meaning your hypotheses is indeed true), or reject (meaning the hypotheses is false) your statement.

What makes good hypotheses is that they are strong statements. A strong statement known to be true or false brings your thinking forward. You could, for example, hypothesize that users are willing to pay for the service you are offering. Or that companies will choose your solution over their existing solution which they have in place. These are yes / no type hypotheses, which would cut off whole branches from your potential tree of solutions. If you find potential customers arent willing to pay for your service, you don’t have to consider all the things needed for billing an invoicing them, for example.

Writing questions to help understand your users behavior are key. So what makes good questions? Well, firstly, they need to be short and easy, open ended, and non-threatening. The last part is key, as you might have users who need a bit to warm up, before you can actually ask them more serious questions, such as questions about their spending habits or what they are willing to pay money for.

Asking open ended questions is also key. It is the only way to have users open up about how they really feel. As an example, you could say: Tell me more about your usage of feature X. This way you make limited assumptions (that they are using feature X), but avoid tainting the answer with a suggestive preconception (i.e. which part they might use, or which function they find more / less appealing). Another good question is a variant of ‘What comes to mind (about feature X)?’. You can also direct toward likes and dislikes with ‘What do you find best / worst about feature X?’.

Asking users for how to improve a feature or product is an often used path to have them help you see something you might be missing. ‘In your mind, how could feature X be improved?’ is a great question to ask. Variants can be ‘… how could feature X be more helpful, more elegant, better meet your needs, etc.’ Be mindful that ‘better meets your needs’ relies on a good understanding of the needs. Without it, you lack the reference point. Remember to build up questions slowly, in small increments, to avoid getting answers which are hard to compare.

Here is an example research brief template for you to use.

Step 2: Showing artefacts

Whether or not to show artefacts during a UXR interview will change the day amici of the interview, and the way you can interpret your results. It is important to be intentional about this. If you plan to show mocks, remember that participants will be biased in the direction of accepting your feature as useful.

Participants have a tendency to acknowledge the work you put into the mocks. They probably look shiny and nice. They probably are well though out. Showing mocks might help you understand users response if they decided to use the feature in the first place. But that is a big if. Not knowing where and how users would discover your feature, and if they would be i clines to use it in the first place, is a dangerous assumption to make without verification. So do make sure to verify first principles, and then dive into the usage patterns and feature details using mocks after.

When you decide to show artefacts such as mocks, make sure you aren’t too prescriptive about it. If you like to understand whether or not a participant can locate a symbol or call to action, present the screen in the most realistic form (including surrounding elements), as to avoid unintentionally directing the participant toward your element of interest. Let them figure it out, and it will take time. If you present a symbol to signal something is wrong, ask them simply: ‘Can you describe what you see here?’ It might feel tough to let the participant ‘hang in there’ for a bit, but you will see they will feel the urge to chat. If you want more info, simply say ‘Anything else?’

Step 3: Involving the team

Your team is a key resource in helping getting the interviews done, capturing feedback, and observe things collectively that maybe an individual could have missed. While conducting the interviews there are four key roles: the participant, the interviewer, the note taker, and the observer. Each of them have a critical role to play in maximizing the value of the interview.

The participant is the main person of interest. They are recruited based on characteristics which make them a good and representative individual. They represent a specific use segment or persona the team needs to gather more info around. Their role is to answer the questions of the interviewer to best of their knowledge.

The interviewer is the main host of the show, and the person directly interacting with the participant. The interviewer makes critical decisions when to ask what questions Heather to adjust the flow of the interview, or whether to ask follow up questions on a specific topic. They are often times the ones who need to decide quickly and based on live observations of the participant. It is advised to make the interviewer someone who is comfortable with theses tasks. The interviewer also needs to monitor comments from the observers (typically via chat), and take their input into account as they conduct the interview.

The note taker is the person with the hot hands. They need to jot down notes and capture what has been said quickly and completely. It is best practice to take notes full-text as spoken, and avoid aggregating or inferring at this point. This avoids biasing the data which has been captured. It is better to do interpretation and aggregation in a separate step, once all data has been collected. If time allows, the note take is encouraged to capture actual quotes from the users and mark them if they are key quotes for the hypotheses at hand. I.e. a user might say something like: ‘We have tried many many solutions to fix this problem and nothing worked yet. We are dying to get this addressed and all our leads are fully aware this is a pressing need.’ Statements like these make a great call-out in a product deck or pitch, as they are very strong pointers to a need for your feature.

The observers (typically more than 1) are observing the interview from a place that isn’t directly visible to the participant. Have them silently join the video call, or provide a real time live stream of the interview to them if you can. The participant should be made aware that more observers are watching. Observers have the main responsibility to capture things in the interview and communicate ares where they might want the interviewer to either ask a follow up, or go deeper in. The observers should communicate with the interviewer via chat. The interviewer is the final moderator, so they will either accept or reject the request depending on what they think is best for the interview.

Step 4: Conducting the interviews

When doing the actual interviews the interviewer needs a couple of things close by: the interview script, the notes sheet (check out this template), and a way to communicate with the participant (outside of the provided video call link). There is a high chance your participant is a no show. So be prepared to call them and confirm they are still able to participate. The interviewer also wants to have relevant tabs open to show artefacts if the interview requires it, and have the chat window open to read updates from the observers. Ideally a live stream link has been shared with observers upfront, or they join the video chat with their cameras turned off and mics on mute.

Once the interview is ready to commence, don’t forget to welcome the participant, thank them for participating, and introducing yourself. Tell them they can stop and exit the interview at any time if they like, and of they need a break they can just holler. Least thing you want is a rushed response because they think they can’t ask for a bathroom break. Let them know there are observers on the line. Confirm they are ok for you to take notes.

During the interview don’t forget to take breaks, speak slowly, and don’t rush it. Monitor the chat frequently for input from observers. Don’t worry about bote taking, this is done by the note taker. If you have to adjust and deviate from the script, explicitly state that so that others can follow along. Make sure the note taker knows where you are in the script.

During the interview the note taker will follow along using the study sheet (example sheet). It contains rows for each question (grouped into study sections), and a column for each participant. The notes are taken into each cell. If the interviewer jumps around the note taker needs to follow along. New rows can be added of new questions are added live by the interviewer or on request from the observers.

Note takers should mark areas they haven’t understood clearly if time allows. Maybe the interviewers or observers can backfill the term or comment later from memory. The goal is to have an as complete set of notes as possible.

Once the interview comes to an end, don’t forget to thank the participant once more for their time and contributions. Give Thema gift card or token of appreciation (not book, coffee mug, etc) as a sign of how much you value them.

Step 5: The debrief

Right after the interview is done allow 20-30 minutes for a quick team debrief. During the debrief have folks backfill gaps, identify relevant comments, and express their most memorable things from the interview. Capture those separately from the interview notes (i.e. a separate row in the sheet). The debrief is mainly to ensure any potential open questions are addressed, and a mutual understanding of the key takeaways is developed. Have all team members share their thoughts in a round table like fashion.

Step 6: The research report

The key deliverable of your study is the research report. It can be done in many ways, but i found a simple short deck to be the most effective (see template here). It allows to capture both the core hypotheses and show the main insights. Decks force you to be brief, as there can be a lot of text from the notes which can be hard to parse for decision makers.

Our research team has done an amazing job putting decks together for us. These decks summarize the goal of the research and the key hypotheses. They highlight the study type and execution, alongside how many folks were surveyed (the ‘n’ of the study). They have a summary section which shows high-level results and recommendations. It also has a section with details where data is captured which backs up the recommendations.

One thing I liked about the results was that they tied back to the research brief. For example, if we asked participants whether they would bay for the service we offered, the results deck would highlight how many people would give a strong and decisive answer. I.e. it would say ‘Almost all participants were willing to pay for the service.’ ‘Almost all’ was a technical term, meaning 80% or more of participants. We encoded the percentages I to natural language terms to make results easier to parse for non-technical teams.

Overall the research report needs to provide guidance what to do next. There are three potential avenues:

1. The hypotheses can be accepted. Follow the path laid out, and develop a set of requirements which can be used to start solution finding.
2. The hypotheses can be rejected. Going down this path isn’t valuable. Develop a new hypotheses to open up other problem spaces for your business to work on.
3. The results are inconclusive. More research might be needed, if practical and not too costly.

The research report is the foundation for the stakeholder presentations which will conclude the next steps. Individual slides are often copies into product review decks, and the detailed deck is referenced back for context.

Summary

In summary I want to encourage a systematic use of UXR, the persistence of the information which was gathered, and the ability to have your organization benefit from this knowledge over time. Once you have implemented above blueprint, be open to adjusting it during the initial iterations. Not everything will work at once, some things might better be skipped, and maybe other added.

If you ever have any questions, feel free to book time with me for a quick PM consultation session.

Materials / templates

All materials are hosted on my Google Drive with view-only access. Make a copy or download in the format which fits you most. Note: the most value is gained if many folks can view the sheet in real-time during the sessions, so some form of cloud-based editing is preferred over local, static editing.

• Example research brief
• Study sheet to take notes during interview sessions.
• Example results presentation