Qualitative research in product management
Not many companies use qual research because these methods do not fit with their culture. There are many reasons for this:
- With qual research, you won’t be able to cover a large number of users (not that you need this large number)
- Qual research is usually performed by user researchers or user experience designers and you don’t work with a researcher or your designers are swamped with design work
- Usually it is not very clear what result you need from this type of research
- You don’t know how to get started, what type of questions to ask and what not to ask
- It is, usually, easier to just launch a feature in production and measure its performance than spend 1–2 weeks discussing with users.
Here are some benefits of performing qual research:
- You won’t find statistically sound solutions with qual, but you get help with formulating hypotheses.
- You will have the chance to ask a few “why-s” to understand the reason behind anything that the user says.
- You should look for what the customer does, not what she says. Hence, depending on the goal of your research, you might get the chance to watch what she is actually doing on a well prepared mock-up or even on production code.
- You might uncover motivations (think about JTBD type of interviews), problems or frustrations with a product or service.
- You might have some “aha” moments that otherwise you won’t have the opportunity to understand them.
So what type of qual research is out there?
As a product manager you won’t go too deep into performing or understanding all the types of qual research. There are many methods to perform it (see some of these on the NN Group blog) and if you discuss with a user researcher, you will probably uncover even more.
For me, as a product manager, the following are the most important types of qual research methods that I have been performing for a while and I am recommending them to everybody.
- JTBD interviews to uncover user motivations
- Exploratory interviews to understand possible problems within a domain, with a product, a service. You can also uncover what users think about a website, product, location or event. You can find a very thorough guide on these types of interviews on the NN group
- Usability tests in which you will uncover usability flaws and if your mockups are easy to understand
- Hallway testing of early mock-ups in which you will uncover early signals if your solution makes sense or not.
How to perform user interviews?
There’s a lot of literature on this topic. The most important thing for me as a product manager is to ask the questions that will uncover insights AND to try not to bias the user. There are exceptions to biasing the user and, when you are experienced, you might use them to uncover even more insights. Here’s a great article aimed at beginners.
How to organise user interviews?
To avoid confusion, try to build a list of 5–10 prospects first and then try to organise interviews with them o a daily basis until you exhaust the problem that you are trying to solve.
Make sure to have someone who will help with note taking and try to have this person stay a silent observer during the interview. It is crucial that your note taker doesn’t interfere with your interview, because you want the interviewee to focus on you and your questions.
Ask your colleague to give you feedback post interview, so that you can improve your skills and to add more questions to your list, should you have forgotten to ask a specific one during the interview.
What to do post a user interview?
Debrief your notes and the notes of your colleague, discuss conclusions and possible outcomes. The most important step at this stage is to document thoroughly the interaction, so that you can return later and remember what you discussed with your customer. So, don’t forget to document it.
Some FAQs regarding user interviews
Question: How many interviews do you need to get some statistical confidence
Answer: This question is wrong from so many points of view. We’ve discussed earlier that by doing user interviews you form hypotheses and understand more thoroughly the reasons behind user behaviours. You can also uncover usability issues with your products. Regarding usability, NN Group claims that 5 interviews should be enough to cover most of the usability issues that your product has. You can try out even more: 6–8, but you’ll probably end up with marginal improvements in your findings. Regarding the other user interviews, start with 5–6, form some hypotheses and try to build solutions or validate solutions with quant methods (A/B testing, launching in production or surveys).
Question: When to perform them?
Answer: Try to perform max 2/day. They are time consuming and you’ll also need to debrief and document them. Also, try to participate in user interviews whenever you don’t have a full operational way (back to back meetings). You will need to think thoroughly about the user interviews that you are performing.
Question: Can you outsource them?
Answer: Yes, you can work with a user researcher or a team of user researchers, but the best way in the beginning is to try to do them yourself. If you can’t, make sure to participate in all the interviews as a listener or note taker. This way you will gain more empathy and you will feel closer to the user than reading a report that someone sent.
