This is Part 2/18 in the series “How to Build an Innovative New Product or Company” on the topic of how to conduct buyer and user interviews to help shape product definition, innovation, and development
 |
We don’t give lip service to consumer understanding. We dig deep. We immerse ourselves in people’s day-to-day lives. We work hard to find the tensions that we can resolve. From those tensions come insights that lead to big ideas.
— former P&G CEO Bob McDonald |
There are two diametrically opposing goals of user interviews
You have to know in advance what your goal in an interview is or you risk confusing the process. You can use interviews to either …
- grow the potential solution/ideation space by more deeply understanding users’ needs or
- narrow down the space by zeroing-in on a specific direction
When I was strategy consultant, I supported a colleague who was doing a few expert interviews to inform the direction of a strategy he was developing for a client. As consultants, we did lots of interviews. We’d rely heavily on talking to experts. Not because they could do the hard work of strategy creation for us, but rather because they are such a direct way of validating or disproving hypotheses that informed the direction we were taking.
In this particular instance, the consultant did three expert interviews. Two of them confirmed his hypothesis and the third was challenging: it raised nuances about why it might not be true. The consultant colleague of mine ended the phone call and said “well that guy was a waste of time.” And, I suppose, there was nothing inherently wrong with him saying that. He was on a tight timeline and had all the evidence he needed that he was directionally correct. He didn’t have the time to get into nuances. Sometimes the key to strategic thinking is to rise above details and make bold moves, knowing that when any innovative idea is let loose into the real world, it will encounter some degree of friction. His goal at that time was to gather enough evidence to defend zeroing-in on a specific direction.
What matters is that you know your goal. Having the time to deeply understand your buyers and users is tremendously valuable. In that scenario, your goal is not to get simple answers; it is to develop real relationships and to get a complex appreciation of users’ lives and motivations. You want to grow your set of potential options and challenges (e.g. through open-ended, empathetic user interviews). Only later should you prune them back (e.g. through more decision-oriented expert interviews) when you need to align on a specific and well-informed path.
IDEO has written several great books about this empathetic, deep-immersion approach to innovation. In “Creative Confidence”, IDEO partners David and Tom Kelley share the classic story of how the project lead of GE’s MRI machines, basking in the glow of a new products’ technological perfection, immediately saw a young patient crying because he was scared to go near it. He took a journey that led him to redesign GE’s pediatric MRIs visually and to create storylines around them, such as the patient sailing inside a pirate ship. It’s a perfect case study where the product was initially designed for the buyer and the technician with no insight into (or empathy for) the user.
Conduct customer interviews to develop empathy for your users and to grow new ideas
First: whom do you want to talk to? Who are your users? The answer may be clear—or you may need to consider what mix of age, gender, career level, geographic location, or other factors you need.
Come to the interview with an interview guide: a set of questions that you step through and document answers in. Questions are ideally open-ended when you’re in exploratory mode, not yes-and-no. You can allow variation from the guide as needed, but it will keep you focused and make sure that results from multiple interviews can be aggregated into coherent insights. Your interview guide should evolve as interviews progress: certain questions will be answered easily and new areas to explore will be discovered. Have an interviewer and a separate notetaker; write down what your interviewee says, with real quotes, not just what you interpreted their answer to mean.
How to find interviewees
I’ve used GLG and similar firms in the past to help me find and schedule experts to interview. If you’re looking for CxOs in Fortune 1000 companies and are willing and able to pay for such access, they are a great option. For a startup, you may need to look at other avenues.
LinkedIn has been useful. Most business leaders are on it and it’s a great way to find Director- and VP-level employees who are easier to reach and more willing to talk (and sometimes better informed). I usually offer an Amazon gift card in exchange for a one-hour expert interview. LinkedIn has a premium plan that lets you send InMails on their platform. Yield on such cold outreach is low though: a 5% or 10% response rate.
Tom Chi, formerly of Google X, shares a great insight in the video linked below: if you’re building a B2C product, go to the mall twice a week to get in front of potential customers. They can be a receptive audience because some people go to the mall when they’re bored so they are interested in seeing what you’re up to. And also, perhaps most important, the mall does the demographic segmentation for you: there are stores that focus on different groups such as skateboarding teenagers. Find the store that fits your user demographic and stand near it. Then have a couple of engineers sitting in the food court who are able to update the prototype in real time as you get feedback from users. Of course, this works if you have a relatively simple B2C product and are looking for initial feedback. It doesn’t apply as well to the challenges that I have found most interesting—such as deep usage questions (which require days of usage to fully understand), usage that requires being part of a users’ daily workflow, or B2B users (whom you’re not likely to be able to find in a mall, at least not in the right mindset).
You can’t ask users to evaluate or rate behaviors that they don’t do today
Find the leading users who have already found a way to accomplish the behavior your product is trying to enable and interview them. Why is it hard? How can we make it easier? For other users, ask them “Do you do this today?” “Do you try to do it today but have trouble? If so, where do you get stuck and why?”
Be aware of the “consolation prize” interview feedback
If you’re sharing an idea with a potential buyer/user and they don’t like it, don’t follow up with “Well about if we did this?” or “What about if we changed it this way?”. They’ll likely say, “That would be better!” because they don’t want to disappoint you. Ask them to disappoint you. Start new ideas carefully with a clean slate and ask for objective ratings. If you take the consolation prize answer, you’ll end up with a bad product and both of you will be disappointed later.
Be aware the pitfalls of the “Pepsi Challenge” in interview feedback
In the 1980s, Coke and Pepsi ramped up their marketing to win customers away from each other. Pepsi set up tasting tables where Coke drinkers would drink a small sip of Coke and Pepsi and apparently say “I like Pepsi better”. This was because Pepsi is sweeter. So, in small doses, they may prefer Pepsi. But ask them to drink an entire can every day and most likely their feedback would be “Pepsi is too sweet”. I’ve found the same thing when looking over UI designs. In small doses, people like bold and colorful. But when they have a product fully built out and are using it every day, they want less attention-grabbing graphics, less text, and simpler iconography.
IDEO puts special emphasis on “extreme users” (i.e. power users)
Who are the potential users of your product who are early adopters, tinkerers, and innovators? Who has already cobbled together a solution to what you’re building by themselves, using Excel and other available off-the-shelf tools? Who are the users for whom your value proposition is to simply help them save time and do more efficiently what they are already doing (as opposed to helping them do something they hadn’t even considered doing)?
Those are your “extreme users”. By observing and engaging with them, you will be able to fast-forward through a lot of product design decisions. Other “extreme users” are those who have specific needs that make existing products a poor fit—such as tall people driving small cars or elderly people with arthritis trying to open medication bottles.
IDEO, for example, interviewed people without teeth during the ideation for a toothbrush that was later marketed under the slogan “unless you have a flip-top head…” A senior partner at the strategy consulting firm I worked at a few years ago shared this story with me as inspiration for a series of interviews I was about to embark on in the healthcare analytics space. I went back to him later, “I just spent weeks with these toothless people down by the bus station and really got nothing usable from it…”
How to manage a 1-1 user feedback interview
Remote research (by phone or web conference) has several positives: people are easier to recruit, you can connect with people all over the world, it’s easier to have other members of your team join or listen in and it’s easier to keep your interviewees in their natural environment (e.g. their office) where they may be using the product you’re building.
Remember to ask them to be at their computer in a quiet room when you schedule the interview—otherwise some people will schedule you during their commute.
At the start of the interview, check that they still have an hour to spend with you; if not, ask to reschedule. You’re paying them. Don’t accept an abbreviated or distracted time slot. Ask them if it’s okay to record the session, which web conference tools like Zoom make it easy to do. Remind them not to share any confidential information about their employer during the interview.
Jake Knapp’s book “Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days” has some great guidance from him and his colleagues who run the User Experience Research Group and Google Ventures. This is a topic they know something about.
Sitting side-by-side with a user as they are testing out a new user interface can be beneficial, but that’s only one type of interview. My colleagues and I do lots of interviews all the way through the product development process—from understanding needs to asking how a basic problem solution might fit into their day to seeing how they interact and use a prototype. Many of them can be done as well over a web conference—which will allow you to talk to a greater range of potential users more efficiently.
Jake explains five stages of an interview:
- Warm intro. Smile. Be positive and enthusiastic. Put the interviewee at ease. Tell them how you’re looking forward to getting their feedback and reactions. Let them know you’re still working on the product and so you know it’s not going to be perfect. Ask if it’s okay to record the interview for your internal notetaking later on.
- Ask a couple of open-ended questions to better understand their context. Jake recommends starting with small talk (“how long have you been doing this role?”) to zeroing in on a topic related to your product (“do you use any tools or software systems on a daily basis in your job?”). Open-ended questions are ones that can go in many potential directions.
- Bring out the idea, prototype, or product. Let them know “I didn’t design this, so you can’t flatter or offend me—please share honest feedback.” In a startup, you may well have been doing all the design and interviewing but any opportunity to put some distance between yourself and the product is helpful to keep the interviewee from coloring their reactions.
- Ask them to try to accomplish something with the product. Yes, you can tell them step-by-step what to click on, but what would you learn from that? Your product needs to be able to be useable with no training. You can ask “if you were seeing this for the first time, how would you decide if you wanted to spend more time with it?”, “What would you want to look for first? Why?”, and “What are you thinking when you see this?”
- Debrief. Away from the product again, “What did you like? Dislike?” and “if you could change or add something, what would it be?” Ask open-ended questions throughout, as opposed to, “would you likely use this on a daily basis or a weekly basis?”, which assumes one of those two options is viable for them. If you feel the need to ask a yes/no question (e.g. “did that graph look wrong to you?”), try an open one instead (e.g. “I see you’re pausing on the graph…”). Or show them two mockups and ask, “do you prefer one over the other? If so, why?”
Remember: the interview is about them, not you
This is a trap I’ve fallen into in the past. I’ll say things like “You’d never use this feature, right? Because …” or simply, “I agree”. Your opinions don’t matter. You’re there to learn and document. They’re the expert. If you want to hear your own opinions, set aside another time to talk to yourself. Saying less and reinforcing your view that they are the only expert in the room increases the invitation for them to speak and share.
Your job is to be the advocate for your interviewee, not for you or your product.
You can have two people on your side join the interview, but you should have one person be designated as the lead interviewer and another as the notetaker. If the notetaker wants to ask a question, they should hand it off to the lead interviewer who should time it and frame it in the right way.
After asking an open-ended question, allow some silence after their answer. Silence is an invitation for them to speak. If you’re always running up against them trying to speak yourself, you are reducing the power of that invitation.