The early user research playbook for founders — Jeanette Mellinger’s expert advice for validating your idea with high-quality interviews

 Yeah, it's my pleasure. Thanks for having me.

 Uh, to kick off the conversation, maybe we could start really broadly on the topic of user research. And I think it's kind of one of those things, that everyone knows about and knows they should be doing it, but often either don't do it or don't do it well.

And I was hoping to spend a lot of our time together exploring kind of the nuances of what Excellent. Customer research looks like and sounds like. Um, and so maybe with that context, we could start with some of the things that you see as common mistakes or traps that founders tend to fall into when they're doing this important work of talking to customer.

 Of course. Yeah. And of course it's hard to do something well when we even have a hard time defining it ourselves Sometimes as user researchers, do you wanna call my job user research or UX research is this customer feedback? Um, and I think that's, that speaks to the, the, the nuance and a lot of the different ways that one could do this.

But I'd say that actually one of the first mistakes is that we don't call it user research, um, especially if I'm thinking. Let's see, an early stage founder, um, they've gotten the memo, which is great, that they should be getting to know their customers and integrating their needs into their initial product and making sure they're solving a real problem.

That's, that's a huge leap for us to have really adopted that mindset. But then as founders go out and get to know their customers, if they don't recognize that what they're doing is user research, whichever formal title you want to give in and, and recognize that there's some rigor that comes with it, you might not actually end up getting out of it as much as you really, really can.

So I think in a way, the first mistake is not recognizing that there's a bit of, um, a skill and an expertise to that particular part of your initial job, um, which is understandable. You're, when you're starting a company, there's a million hats that you're wearing and you're not expected to be an expert at all of them.

Um, and yeah, I'll dive into that in, in a moment in terms of some of the main skills I think are worth adopting from the practice of, of UX research. But the first one is to say, yep, I'm gonna wear this as a, as a skill hat. And I, it's a really different thing than just. Poking a friend in chatting. Right?

What, what's the, the leap between those two things, the chatting and the, experienced user research that you could be doing? So there, there's that first mindset piece. I wanna first ground you in a story that really stood out to me when, uh, I first started working with First Round as a user research expert.

Uh, the first few founders that I spoke with, one after the other said, I do care deeply about my customers. I've spoken with dozens of them, in one case, even hundreds of them. But wow, I'm so excited to work with you because now I want to add a little bit of rigor to this. I didn't get that much out of those conversations.

I sort of have a hint of what I might need to do, but it feels like it's just my intuition now is the time for rigor. And it's that moment of saying, wow, you've put in this much time, this much effort, and not gotten out of it very much. This is part of why I'm excited to talk about these tips, because I don't want people to get, stuck in that moment.

It's natural that your first. Few conversations might be a little bit more loose and you do wanna add rigor, but stop before you've gone to dozens or hundreds. Um, and so the main three things that I think, um, I've noticed is patterns across working with now dozens of early stage companies through first round and outside of it is that there's, um, three main steps.

One, um, when you're planning research, I find that teams don't tend to be nearly focused enough in it. I can dive into that a bit more. Um, two, when you're in the learning phase, you're asking questions. It's just so easy to inadvertently bring in bias or honestly not, not dig deep enough and find something that's unexpected.

Um, something that's actual discovery. Uh, and, and three, when you're trying to figure out what to do with your learnings, um, there's very little analysis done, uh, and let alone done in, in a depth that can get you to some real unexpected insights. But it's nice that it ends up mapping to I think, the main very high level steps of research.

One, you're planning, research. Two, you're conducting it, you're learning three, you're acting on it. And so it's that. I think I'm a big fan of frameworks as a researcher, and the one I I often tell teams is one, focus, two better questions. Three, analyze.

 So with that kind of high level architecture, why don't we take them in turn and maybe start with the first topic, which is planning research. What does that looks like when it's done well?

 Yeah. it probably takes more time than you'd like, but saves you a lot of time later. done well, I like to think of asking teams to, uh, chunk their research into many research sprints. So let's go back to that example of a, let's say, a few dozen conversations you wanna have. It's not to say that you shouldn't have these few dozen conversations, but instead chunk them into, five to eight very focused conversations, one after the other.

Um, and so then in terms of structuring those and saying, how do I think about this first chunk, um, the next framework for you that it, uh, makes up what I, I consider to be a bit of a mini research plan, more of an informal research plan is, one, asking yourself what decision you wanna make. Next. Two, defining one to two learning goals, and then three, figuring out whom you're going to speak with.

 the first piece, um, that decision to make this is often the one that we skip, um, and actually is a lot harder to define than teams expect.

Um, you can also rephrase this for yourself as desired impact, but you wanna ask yourself, what is the next decision that I wanna make or action that I wanna take in order to take the next step, um, with what I'm building? And, you know, that I hope I can have customer feedback inform and that decision.

Some examples here might be, um, of course we can start with saying, I don't know, I wanna define what I'm supposed to build. Period. Um, Let's make it smaller than that though. It's good. It's good to think about the higher level, things that matter to you most, uh, and that you need to define over the next reps.

But it could be something like, I want to, um, refine what is my target customer. definition I want to, um, figure out which is the most important first feature to build for them. Think about the meeting you're gonna be in in a couple of weeks where you want this research to have informed it and think about what am I going to be talking about in that meeting that I wanna be able to do?

And then that gets you to question two for learning goals and saying, okay, what do I need to know that I don't already know, that will help me make that decision? And it's so interesting when I walk teams through this exercise, a lot of times when I'll ask them about the decision they wanna make, they'll end up jumping to learning goals without meaning too.

 but I think it's cuz we naturally move into learning mode. Let me say, oh, I need to decide and understand what the, the deep problems are of my customers, let's say. Um, so then if you're having trouble, that's still a learning goal.

That's still something you want to learn about is what is their in-depth problem. So then you ask yourself, okay, in order to do what, in order to frame some part of my value prop in order to decide on the first feature to work on et cetera, um, I suggest that teams toggle back and forth between those, but clarify that decision to.

And then the next piece of focus is around those learning goals. there's gonna be a million things that you wanna learn. What are one or two things that you need to learn first that you know the least about in order to inform that decision? Um, this is, I think, one of the biggest team mistakes that teams can make.

And, I've worked with companies a lot of different stages, and this is true for much more, um, mature companies as well. It could be so tempting to wanna ask about 10 things at once. But, uh, it's unfortunately going to, um, get you some pretty shallow, uh, responses. This is your, um, opportunity to, to go deep.

And also sometimes when you try to learn about too many things at once, you can have some, uh, variables that start to conflict with one another. So really say, what are the one or two things I must learn now? And then three, you wanna focus on a particular target customer,

 one mistake and one of the, um, the things that can again, lead to that moment of saying, wow, I've spoken with dozens of people. What do I get from this? Is if we talk with a lot of different types of people, even about the same questions, we're going to probably get some conflicting feedback.

And that's not necessarily that there isn't a clear way forward, it's that we might be talking to people who have such disparate needs that unfortunately some of them might be the one, the people that we wanna serve and some not as much. And then it can, create a lot of noise for us. So this is where I'd suggest really getting clear on who do I believe is most important, either one to serve as a business or two that is most related to this decision I might want to make and 

If you speak with five, six, maybe eight people, um, of a decently narrowed target customer type about a couple of main learning goals, uh, you can get enough qualitative data in those deep conversations to really start to see some patterns and to have a lot more conviction than even 50 more loose conversations that aren't as focused with a lot of different types of questions and a lot of different types of people.

And we get nervous sometimes when we hear, oh my gosh, I only spoke with five people. But really if you bring that focus into it, you can get something that is, much deeper than you'd expect. it's kinda the magic number in qualitative research that if you're well focused, you can move forward after five I.

 With that bit of context, let's say someone is a chef in a kitchen and has this insight that they think there's an opportunity to create a new kind of pan, maybe that heats faster, has some set of properties that are different and valuable to other chefs around the world. And so they've been thinking about this and they're gonna go do customer research for this pan.

 how does this map to sort of the set of ideas that you just shared, what, what would that look like if this person was doing it well?

 Yeah, that's great. So again, they first say, all right, I wanna build a mini research plan. Let's first think about the decision I want to make. And at this point it might be the larger decision is probably, do I want to actually build this pan? Um, but then they might say, actually, I want to further first define who is the right target chef, um, and what is the main value prop for that particular chef, let's say.

Um, and so then the next thing that they want to learn is what are the, top cooking related issues for this hypothesized, top chef profile? And. What do they think of my faster heating pan idea? And then as they start to define the target customer, it's not necessarily a chef anywhere. And here's where I can get a little bit dorky with the target customer piece, although don't be, um, I wouldn't, I hope that people aren't intimidated by bringing in this much rigor.

But you wanna think about what we, uh, call psychographics and demographics. Ideally, psychographics is how people behave or think, and demographics are some of those often easier to pinpoint details, age. Um, in this case, maybe it's, um, the geography of the chef. It's number of years of experience. And I often suggest that people start with psychographics.

And here you're gonna think about, wow, okay, if someone wants a pan that's going to heat faster, that probably means that they're a chef that's in a really fast moving kitchen. Um, maybe they have too much demand in their, in their kitchen, or they're trying to innovate the ways that they cook. I'm gonna hypothesize that it's about chefs that are in overloaded kitchens.

Uh, and I'll say, okay, if that's, uh, if that's probably the person, then maybe the demographic is, it is in a large city. It's in New York City with a restaurant that serves this many people. Um, and it's a chef of, you wouldn't even necessarily have to go to, let's say, years of experience or, um, a type of cuisine.

But you think about which of these demographics might relate to this particular type of chef that I think that a faster heating pan would. And then go from there. And, and for instance, if you think that wow, there's a split between chefs that might just be overloaded in a kitchen and need a heat faster versus a chef that's approaching it in an innovative way, those might be different chef profiles.

So then try five interviews with the first type and see how that goes. And or do five interviews with similar questions for group B, the innovation group.

 And so if you don't, you know, and you're sort of getting at this, but if you don't know who this is gonna most resonate with, is your perspective that you kind of create three to five of these cohorts of five to seven people who are in a similar, psychographic or demographic makeup.

 Yes. But with a little bit, um, you can do a little bit more legwork so that you don't need to go into five interviews with each of those subgroups. Um, this is where, you know, some of the great discovery research to be done is even just, um, secondary research or general observation before you get in front of new customers.

So in this case, the chef has a wealth of experience on their own and they can think about, okay, what do I and my peers, what are we interested in? What types of chefs are we in? What scenario do we mostly want this faster heating pan? You can also go to the chef forums, um, or go to a chef convention, find out what people.

Are talking about and what, what problems they're facing and see if there's a particular type of chef group that is, um, most facing that And then just develop a hypothesis for yourself of what do I think are a couple of these segments, these groups, um, that I most wanna serve.

And then just talk with the first one or two groups, five people a piece will say, that you think most fit it and go from there. If it's not really gaining traction, then you can revisit the other groups as well. But I wouldn't tie yourself out with, with doing that. I, I would just go do some immersion in the space to have more of an educated guess here.

 Before we move on to the second phase, is there anything else founders or product folks should keep in mind in phase one as they're thinking about the who and the goals I should say.

 I think it's just remember to keep it focused. Uh, we're gonna discuss a lot of tips today. It's funny, as I was preparing for this interview, I was looking back at top takeaways that, um, teams have had whenever we've talked through these different phases and it's all across the board. And that's because there's so many different steps here, um, for people to toggle, in order to get more out of their own research and just which one resonates?

Depends on. Who you are and probably where you are in your practice of talking to customers. And, but the, the simple thing that I would really wanna just drive home here is focus. if you're wanting to try to learn about 10 things, put some of them on the back burner and say, what are the one or two that I need to dig into now in order to make, make this decision now?

But target customers say, yes, of course you wanna get to know about all of them. What are the one or two that you most need to get to know now and focus on them. And then as you see what comes from research, then you can absolutely re-up and go learn more. Cuz you're gonna learn some things you didn't expect and you can have a chance to revisit those other goals.

So just don't forget to really focus and chunk those. Um, and this is true really at, at any stage of, of research, um, any stage of maturity in, in the questions that you're asking. Um, but it's particularly true in this discovery mode when it's otherwise so easy to get overwhelmed by the number of things that, that you want to cover.

 Right. So that's sort of phase one. Let's dive into phase two and maybe talk about it at the philosophical or framework level, and then maybe we can re revisit our fictitious potential pan company. So this is all about the learning phase. Um, and one of the things you said a few minutes ago was, it's easy to inadvertently bring in bias.

I think kind of the number one thing that I've seen is confirmation bias. I want this pan. I think this is a good idea. No matter what happens in these conversations, I'm going to leave thinking the other person thinks it's a good idea as well. Um, or you do something such that the person doesn't wanna offend you, so that, that they actually don't share what they think of this pan or problem or whatever.

So let's dive into this learning phase and, and how to make the most of it.

 yeah, of course. Um, I'll also break this into chunks cuz I think there's a lot that, that we can cover. And I'll start with this bias thing first, but, um, at a high level, when I think about asking better questions, there's two camps. The first one is, uh, getting better data, getting better qualitative data.

This is, um, when I'm asking for their feedback on the pan, will they, to the extent that they can truthfully tell me how they feel and, um, not that I'm implying that anyone is trying to del dilute you, it's because of these forms of inadvertent bias that we bring in as interviewers or that we might kick up in our interviewees.

Um, that we might unfortunately sometimes get false signal. You know, someone said, oh yes, I would absolutely do that, and then they actually won't. But now you have enthusiastically decided to build something that perhaps people don't want as much as they had indicated, right? So, number one, getting better data.

That's one I'll dive into first and, but number two that I wanna just flag will be coming next is uncovering. Insights that we didn't expect. Um, kind of like a le level two of up-leveling your question asking game. Um, that first level is already hard enough to do, to say, I wanna get a good read on the questions I know to ask.

But really number two, that level two is the real opportunity of an in-depth, interview and especially at this early discovery stage, is when you can find out things you didn't even know to ask about and maybe get signal on some of your blind spots, the problems that you didn't actually realize were there are that might get in the way.

Um, they might tell you, yes, of course I want a faster pan, but you realize they're bigger problems are actually related to something entirely outside of their cooking implements or is a totally different cooking implement and they just wouldn't have thought to bring it up. so those are the two things that I wanna make sure that we can, can talk about.

Um, but I'll first touch on the bias piece. And there are three, I think three main human tendencies that are good to be aware of. I think we're all, we all know that there's, there are a million forms of bias that we fight. You've just named one of them. Confirmation bias is, absolutely the one to keep in mind for, for ourselves and, and to recognize that.

Um, and I think the chef example here is great, that in a way the more that you are the user and you've built the product, the more that you have at stake and the more that you want to hear what you want to hear. And, um, one of the founders, uh, who's, a part of our first round portfolio who I asked for, um, his reflections on, on our time working together.

He said that one of his biggest tip for himself is to, to not have happy ears, to not be listening for something That sounds good. Um, we can talk about a few ways to combat this in terms of types of questions to ask. We can get into that in a moment. But also, this is around our analysis. It's, making sure that we're asking better questions, but also that we're not just listening for the things we wanna hear.

The second form of bias, um, that you alluded to is acquiescence bias. And this is a sometimes lovely, but not all that helpful, uh, inclination that we have as humans to be nice to one another, to make, to try to make one another feel good or to hope that people will feel good about us. And, uh, this is, I think, a nice cousin of confirmation bias, but one that I wanna call out on its own as interviewers.

What this can look like is that we will show that we're excited about our idea. We won't even mean to we, but it could be as early as your recruitment email, um, saying, will you please speak with me? I have this amazing idea that means that someone. Inadvertently feel like they need to tell you that your idea is good.

They don't wanna make you feel bad about it. So again, we'll talk about ways to get around that. Um, and then as the interviewee, sometimes again, you might want to be a little bit aspirational in your answers without even meaning to. The most classic example of this is that the doctor, they ask you how much water you drink.

Oh, I definitely have eight cups of water a day, um, when it's really two, but you figure tomorrow you'll start drinking eight. So it's not really all that off that, that the more that we might signal the ways in which, we want someone to answer, they might inadvertently feel they need to answer that way without, again, intending to dilute us.

Um, another way that this can show up is that if people can tell we care about something, they will probably want to spend time talking about it. So they know that we're interested in pans and they can tell you the 50 things that don't work about their pan. Um, even if it doesn't actually keep them up at night, they can find things to talk about because they want to be helpful to you.

And so, um, the more that you don't necessarily show your hand in terms of what it is that you want them to be telling you, the better. Uh, and the final human tendency to be aware of, I think is really. Uh, applicable to this group is that we are not very good at predicting the future.

Try as we all might, and it can be so tempting for us to say, Ooh, will you use my product? Um, what will make you switch to my product? How much will you pay for my product? People can guess, but we can't bank on it. And, um, really the best predictor of future behavior is our past behavior, our existing behavior.

So the final human tendency to call out there is that we also don't have great memories. So asking someone about what, how they were doing it five years ago, um, probably won't be that reliable, but a recent specific example is the best way to get at what they might be doing next. Um, I can, if you'd like, I, um, can dive into some more of the specific tips around how to combat each of those forms of bias, 

 Yeah, I think that would be a great scaffolding. So maybe let's go back to the top and talk about confirmation bias, and given that human tendency, what should a founder do?

 Well this is, this is the main tip and the least surprising tip, but I'm going to maybe put a slightly different spin on it, which is that you just have to avoid, um, asking leading questions. A leading question is one in which we, um, in our minds probably already have an idea of. What the answer might be, and then we wanna validate it, right?

So you might say, um, do you think this is easy? Do you struggle with this? Uh, in that case, um, we're implying that we do think that they struggle with it or that it should be easy. And, um, that may or may not be true, but they might feel compelled to say yes or no. So the, the nuance to how to notice if you're asking a leading question is, I have two tips here.

One, just notice if the question could be answered with yes or no. It often starts with, do you X, Y, Z? Um, and if so, bing, you have a leading question. And also, the second problem with only eliciting yes or no answers is that they just might, you might miss a lot of the richness. There are very few people you talk to who will just say Yes, period.

They might elaborate a little bit, but it's a lot less rich than saying, asking, tell me about this experience. what do you think of x, y, Z pan. And we, we can give some more pan examples in a moment there. The other thing to look for with leading questions, um, is notice if you have an adjective in there, an implied opinion, um, how e even if, let's say you've taken away the yes or no piece, and you say, how easy was this for?

That implies again, still that it was probably easy. So the way that I would rephrase this is first, tell me about this experience. Um, and then if you must ask about something a little bit more pointed, like say easy. Then another way to get around it is to always train yourself to ask about the counter here.

So the positive could be, how easy was this? Or to what extent was this easy, if at all? I love that extra little piece that you can put in there as well. That's the, if at all, gives them an out so they can say, eh, wasn't easy at all. So to what extent was this easy, if at all? And then always ask the counter to what extent was this difficult?

What did you find difficult about this? And there, I wouldn't even necessarily give them the out, make them tell you something that is, um, negative. That's the main tip under confirmation bias. But there's a few other pieces here. The first one is mindset. Remember, you're coming in to try to learn things that are scary in order to not spend all of your time developing your product and finding out that no one wants it or that you missed a key detail that would really have helped adoption.

Um, so first, remember, it is an opportunity and it's critical to be able to hear the scary things. Um, and then you also want to, um, make sure that as you are in the conversation, let's just say that you start to hear something that feels really good. Almost that's your alarm right there. The moment you hear, oh yeah, this product is great, that's a sign to say I should ask the counter.

Um, what is something that I can push on here? Also, as you, um, run those five interviews, we'll probably start to notice a pattern after three or four of them even. Um, which by the way is a sign that you're onto something good. Once you've really hit so many patterns and you feel like you're hearing the same thing over and over again, great.

That, that probably means you're done with this many research sprint. But that's also the moment to say, how can I push on what I'm now beginning to assume is true? Can I ask a new question that, um, even pushes against the patterns that I think I'm hearing here? Um, and then finally, an analysis as you even just do, I, I recommend that every team does 15 minutes of a debrief, right after every conversation.

One of the questions you ask yourself in that is, what did I hear that related to my hypothesis? Um, that perhaps supports it? And then also, what challenges. Again, always ask yourself the counter, um, and just be kind yourself. Know that it's not that fun to hear things that are tough, but you'd rather hear it now.

Um, and so again, it starts from your mindset beforehand. It's how you ask questions during, and it's also the pieces of information that you pick up on in your analysis after.

All right, great. Let's move on. Acquiescence bias.

 with this one, it, it starts as early as the, um, the subject line in your email to recruit people to speak with, but, um, the main tip here is to generally frame things a bit more neutrally and don't. Hold off on talking about your product, um, is the, the simpler way of putting it. And this will allow you to get a better read on what is actually most important to the person talking to you before they inadvertently want to jump into the mode of being really helpful to you.

So what this can look like with the, um, chef example, let's say that we're talking about, finding other chefs to speak with in your, as early as that email, you want to be sharing at least a little bit of information about who you are and whom you wanna speak with, but minimal information when you can about what your product is.

So you can say, I'd like to better speak with, um, chefs who are working in a busy kitchen. Um, tap your expertise, and then make it really easy. This is just a tactical thing, but make it really easy, uh, for them to schedule with you, incentivize them for their time, um, particularly if this is not someone in your network.

Um, but as early as that, you don't need to say, I'm trying to build the fastest heating pan, because then they will feel like they need to speak to that. So can you go a level up and just say, I wanna speak with, chefs who are under a lot of strain. Want to learn how to maximize their time in the kitchen more some, something like that.

Um, even that I'd say, I actually would correct myself and say that I don't even want to allude to too much of which problem I think I'm solving for them. But I think that you can tease out this balance between what's a little bit of information about the space I'm, interested in, and who is the right person for me to speak with without giving everything away.

This also then continues as soon as you start the interview. It can be tempting to say, um, I would like to, talk to you today about pants. And then people will feel like that's the first thing that they need to jump into. Instead, I would start a bit more open-ended. Most of the talking that I do is at the beginning of an interview, a lot less later on, and at the beginning you say, Hey, I'm such and such.

I'm wanting to learn more about X, Y, z. Gives them a little bit of grounding in the conversation. I'm going to eventually show you an early idea, but for now, I want to better understand you and your world. So anything that I ask you, um, You know, feel free to elaborate on what's most true for you.

And you don't have to also answer any questions you're not comfortable with. When I do that at the beginning and let them know what's coming, then they realize that answering my questions before they know what the idea is, is useful to me. Cause otherwise they might just feel as though they're waiting for you to get to that.

So, be very clear. These questions are, are what's useful to me. And this also gets at, um, preventing acquiescence bias by letting them know from the beginning that they're the expert, right? That you want to hear from, from them and hear what's true for them, not just what, what you are interested in. Um, throughout, again, you'll, you'll want to recognize that they are probably going to be, try to be very helpful to you.

So once they figure out you're interested in pans, they're going to want to talk to you about, about pans. Even if they're complaining about pans, um, they're like, oh, we know that this is what's most interesting to Brett. Um, but what if there's something else that was really, concerning to them, uh, outside of pans?

Then you have maybe missed that opportunity if you dive into that right away. So we're, we'll talk about this in my sort of level two, how to get unexpected insights piece. But I have a framework that I use for structuring all conversations, which is to zoom out first start with your bigger context, then you zoom in to that thing you know, you wanna talk about and your idea, and then you zoom out with some final.

Um, higher level question, uh, that helps wrap up the conversation. And those zoom out sections really help with acquiescence bias because they help you see what people really care about when they're not just, um, digging into trying to help you with your, your specific idea and topic. Um, and then, uh, I'll quickly move on to the, uh, future facing i, uh, hiccups that we have and that the, there's a very simple tactical point here, which is to ask about specific recent examples and behavior, um, as we'll talk about in the sort of zoom out in and out framework.

643f0100ebf3abb113931b79

User research expert Jeanette Mellinger shares her three-step process to high-quality user research, with tailored advice for founders in the early days of validating a startup idea.