Doing UX Research Well

I work as a Google Analytics Product Manager in The Bay area. This article summarizes 5 best practices I found worked well when executing User Experience Research (UXR) sessions.

1. Invest significant time into your research brief
2. Be open to various execution models for UXR
3. Work closely with the UXR team
4. On taking notes …
5. Allow communication between UXR lead and session observers

---

Strong investments in UX and UI have revolutionized the experience of users of Web and mobile apps over the last decade. Gone are the times where even enterprise users are willing to put up with ugly, clunky, or too technical user interfaces. Most parts of a software UI these days – especially for customer products, but also more and more for enterprise software – is being evaluated whether it is intuitive, easy to use, and optimized to support the user action it wants to support. In my day-to-day job as a PM for a b2b analytics software, I often leverage User Experience Research to determine which direction to take from a product perspective when building product screens.

1 – Invest significant time into your research brief

• Above everything else: have one!
• Iterate on it, and ensure you calibrate
• Put special focus on questions and flow

The research brief is like your product requirement doc or product spec for what you want to achieve through research. It contains details around recruiting, the questions you plan to ask, and the flow you will guide your participants through. First priority is to have one (you would be surprised!), and then spend significant time on it. Write it, give it some time, re-read it, correct it, iterate on it. I typically need a solid week once a good draft has been put forward by the UXR team. This also includes resolving issues around recruiting participants (recruiting more broadly is faster, but might not yield the precision in insights you need, recruiting narrower is more precise, but slower), and the mode in which you try to execute the research (in-person, or remotely).

2 – Be open to various execution models for UXR

• Explore faster (remote) and more detailed (in-person) setup models
• Be aware where in the product development lifecycle you are (early stage vs product iteration)
• Ensure the session is done representative, and systematic (reproducible)

UXR can be done in various ways. An informal chat with a colleague can be considered UXR. But the main characteristic of good UXR is that it is representing your users, and that is is systematic (it can be re-produced to achieve similar results). Multiple avenues are possible to achieve good UCR. I have done in-person interviews, lab-setups, prototype and mock sessions, as well as remote UXR sessions and sessions via phone successfully with great results. For remote sessions, we could recruit fast, but we often had to deal with tech setup issues and a higher no-show rate, wasting precious UXR resources and (session) time. For in-person setups it was often insightful, but limited in terms of customer footprint. If it needs to be faster, I lean toward a more remote setup, and accept the haircut on no-show for the increased speed in recruiting, and the benefit of having a bigger customer representation (you can go nationwide as opposed to just a single location). When we are deeper into the product development cycle, I prefer in-lab sessions as they give me a better handle on understanding where the user is, how much pre-knowledge they have, and show prototypes and mocks in a more reliable setup.

3 – Work closely with the UXR team

• UXR is a deeply involved process, especially for the leading PM
• Reiterate and clarify product requirements & vision
• Enable others to do their job well

UXR isn’t the ‘throw over the fence’ type service. Sometimes it is attractive to think it could be. Models that expedite research sessions, and make it feel like a PM can just ‘order’ some UXR off the shelf and get it delivered haven’t worked as well in my experience. The devil is in the details, and for the results to be useful for making product decisions, you’ll have to be involved early and deeply. This doesn’t mean to do other peoples job. This simply means to be there, offer advice and clarity on the product requirements, and reiterate the guiding vision to help your UXR team come up with the best research flow possible. In one of my previous sessions, just starting by asking questions to the UXR team really helped get us off on the right track. We did repeating meetings to get alignment as a team across UI, UX, and PM. Later I started offering some advice, and the team ended up delivering on-time, with great insights.

4 – On taking notes …

• Assign note-taker upfront from the core team which worked on the brief
• Prepare the notes doc to include questions
• Always do a de-brief after each session

Taking notes is critical for the value of the whole exercise. Taking notes is the key deliverable during research sessions. The better the notes, the better the insights. I like to prepare a doc or spreadsheet for note taking. It includes the participant, all the questions, and a predefined area for typing answers. Note-takers can be anyone from the team who has worked on the brief. Ideally include engineers also. Take notes as the participant speaks, don’t paraphrase or interpret, just type it all down. Always do a de-brief where you verify things you couldn’t catch during the session, and discuss things that stood out to everyone who was in the session. Capture memorable notes – they come in handy for presentations to make learning more personal.

5 – Allow communication between UXR lead and session observers

• The session leader is in control
• He / she can accept or reject real-time pings from observers
• Real-time pings offer another layer of value beyond the predefined (default) script

This one turned out to be a new one for me: the ability to calibrate the flow in real-time through chatting with the person leading the research session. I loved the ability to post questions in real time when I felt the participant was on to something, or he / she mentioned something extra insightful. It is also helpful to clarify. Make sure it is clear that the session leader can accept or reject these pings. It can be overwhelming for the session leader, especially if things don’t go as planned (i.e. technical issues arise, or the participants get frustrated with something). The default that worked for me was always: the session leader is in control. The default mode is to run through the flow from the brief. Real-time pings can be taken into considerations at the discretion of the session leader.

---

There is tons more to consider for excellent UXR. I have been lucky to work with some of the best in the industry, and I deeply respect the value the UXR teams everywhere bring to the table. The product teams are lucky if they have the luxury of fully staffed UXR teams available. Hopefully these tips are insightful, let me know your best practices in the comments.