Don’t have a UX research team? Jane Davis’ tips from Zoom, Zapier & Dropbox to get you started

 When I start hearing a theme, I write it down partially because writing it down makes me more cognizant of the fact that it is appearing in my head. And then the next thing I ask myself is what would need to be true of the next several interviews for me to feel like this wasn't a theme and I will oftentimes seek out information that goes against what I've heard to pressure test it.

So if I think I'm hearing a theme around recruiting, being the most difficult part of a research project, I will deliberately go in the opposite direction where I will say, tell me what the easiest part of recruiting is, or like what's the most difficult part of your research process so that I'm giving people the opportunity to give me an answer that I'm not expecting.

 Welcome to in depth, a new show that surfaces tactical advice, founders and startup leaders need to grow their teams, their companies, and themselves. I'm Brett Berson, a partner at first round, and we're a venture capital firm that helps startups like notion, Roblox, Uber, and square tackle company building firsts. Through over 400 interviews on the review.

We've shared standout company, building advice, the kind that comes from those willing to skip the talking points and go deeper into not just what to do, but how to do it with our new podcast. In-depth you can listen into these deeper conversations every single week. Learn more and subscribe today at firstround.com

today's episode. I'm really excited to be joined by Jane Davis. Jane has worked at an incredible stack of startups. And as tons of enterprise experience at different company stages these days, you'll find her at zoom where she's the director of UX research and UX writing. These days, you'll find her at zoom where she's the director of UX research and UX writing.

Previously, she headed up UX research and content design at Zapier and manage the growth and research team at Dropbox. I'm going to start off by admitting that I really nerded out in this conversation and used it as an opportunity to ask her all of the tricky questions I've always had about customer development in Jane's responses are spot on giving a great window into the day-to-day work she does, and best practices she leans on.

Throughout our conversation. Jane walks us through the end to end research process from clarifying your goals and asking the right questions to selecting participants and synthesizing insights that can inform product building. We also cover the different phases of building. We start by going through how she applies her playbook in the early stage startup context.

When you're shipping the first version of your product and you don't yet have the resources to invest in a full-time research team. We also dig into challenges such as deeply understanding the problem you're solving, taking on a competitive or Greenfield market and figuring out willingness to pay. We then get into best practices for prototyping and iterating as well as some of the common roadblocks startup leaders face later on, including how to build for multiple users and what to do when people aren't excited about your product or using it frequently.

What I loved about our conversation is that even though Jane gets into some big concepts like confirmation bias or stated and revealed preferences, she also does a spectacular job of diving into the brass tacks of how to approach this work. She shares tons of tactical pointers from the specific go-to questions.

She always asks customers to why you should always pay for transcripts of your user interviews, whether you're talking to potential customers before you start a company, or are looking to get better feedback from your current users. There's tons of insights in here for founders, for product builders and design folks alike.

I hope you enjoy this episode. And now my conversation. Thank you so much for joining us. Oh, 

 thanks. It's great to talk to you today 

 so I thought we could start by maybe going in reverse company size order and start with some of the things you've learned in the applicable ideas and frameworks that fit really early stage.

And we can grow through company maturity. And one of the reasons I'm excited to talk about these topics with you as you've operated it at so many different scales. I 

 wind up all over the place as it turns out. 

 So on the, I thought a really interesting place to start might be when you're spending time with founders and CEOs of five person or 10 person or 15 person companies, and maybe they're about to ship their first version of the product, or they have their first 50 customers.

And they're not going to build a user research function at nine people, but they want to infuse a lot of the things that you do oftentimes at scale into a really early stage company, in terms of talking to customers, taking that information, organizing it in a way that can inform where they go and order that information as well.

I was interested to talk about like, if you're teaching has really small company about how to take some of these ideas and implement them, what are the types of things that you would be explaining to them? 

 Sure. So I think a lot of the time when I'm working with companies that are very early stage, I really want them to focus on a deep understanding of a problem.

One of the things that can make that difficult with startups is that you oftentimes are starting from a solution. You're like, I have a great idea for solving a problem. And that can be true, but the thing that's most interesting to me there is. Always that you've identified a problem in need of a solution that is the much harder part to me.

And the part that offers a lot of really rich opportunity. And so stepping back from discussing concepts, from putting solution ideas in front of potential customers, and really just deeply understanding the actual problem you're trying to solve and the people for whom you're trying to solve it, it can be really difficult when you've got what feels like a really strong idea in your head to step back and say, let's actually move away from what has felt like progress in the form of coming up with this idea and actually spend a lot of our time.

Really just deeply understanding the space we're trying to operate in and understanding not just the specific problem that we think we're solving, but everything that goes into the rest of it as well. So a great example here is thinking about the workflows that surround a single solution. I have a friend who was putting together a FinTech tool where they were really looking at this one specific part of a workflow and they had what they felt like was a really strong solution for it.

They were really having a lot of trouble getting traction with customers. And after talking to him, the thing that kept coming up was that people weren't looking for just that one part of the workflow to be solved, what they really wanted was a much more end to end solution so that they weren't constantly transferring from tool to tool to tool.

And so really understanding what it is people are actually trying to accomplish and the problems that you're trying to solve for them rather than saying, Hey, we think we've got the solution right here. I like that approach one because it helps you avoid confirmation bias, but two, it actually sets you up for later growth and innovation in the same space where if you're really deeply understanding the problem and you do have a strong solution for it, then.

You can also understand what the next opportunities are in the space. So you can say, okay, we're going to build this one solution first, but we know that there are opportunities in these three other places that are related to the solution we're building. And we have these built-in growth opportunities for our company and these built-in product expansions or.

Other lines of business that we could be pursuing if we're successful with this initial one or an opportunity to pivot. And so I think of this kind of research as what we call digging the well, it's very much about creating this repository of insights that you can draw from for years and years and years beyond just that initial round of customer visits or user interviews, where you can say to yourself, if we really deeply understand the problem space, then we don't have to invest in understanding another small piece of it later.

That's the number one thing I really try to get early stage companies to focus. Can 

 you share the how behind that? So you talked to a founder and they did this classic thing where they jump into the solution and they tell you how excited they are about it. And you explain to them, and you're like, you need to be deeply rooted in the problem and the customer.

And they're like, great, totally agree. And then you give them your playbook to the how behind that. Can you explain what that looks like? 

 Yeah, absolutely. So the first thing is that it never ever looks like showing a solution to a potential customer. So even if you do want to do solution validation, I never do it as part of this particular round of research.

So the playbook I try and get founders to run is very much start from absolute zero. Start from assuming, you know, nothing about how this person is doing their job or about this person's life. And really just try to get the most detailed picture, you can of how they get things done. And so for very, very early stage startups and founders, I think one of the most fruitful things can be to just try to understand all of the workflows that go into someone's job and how they get their work done.

Or if you're moving into the consumer space, how you, you spend your time outside of work because you never know what kind of insights are going to uncover? And so leaving it broad, gives you the opportunity to find, find out things that are relevant to you that you wouldn't have thought to ask. Um, so as much as possible, I. advise founders to keep it incredibly broad and then dig in. And so my go-to question and my playbook is always tell me what you did yesterday. And I will have people just walk me through their day from start to finish and we can go for an hour longer just digging into that. And so they'll be like, I picked up my phone and just glanced through my email.

And then we can start asking about what did you do when you were glancing through your email? Did you respond to anything? Is that something you do every single day? And then just digging in to all of those different moments. And I always have a set of things that I want to walk out of the interview, knowing I usually have three to five bullet points, and those are the things that I want to understand.

So it could be something as specific as have you ever paid for custom content on any of the platforms you're on like Facebook stickers or discord, nitro pack or things like that, or it could be as broad as do you ever share files with people outside of your organization and digging into how they do that?

And. Ideally from there, you can form a relationship so that you can say, this was so helpful, we'd love to come back. When we've got some more ideas about how we might solve some of these problems for you and show them to you so that you aren't having to re-recruit over and over again, to talk to people.

I really do focus on these. What can feel like meandering or indirect path interviews, because they give you the chance to get the full picture of the problems someone is trying to solve during the course of their day. And it also lets the participant lead. So rather than saying like, oh, tell me about all of the problems you have related to this topic.

It's just, they might not even think of the thing you're trying to do as a problem and that's quite telling as well is if somebody isn't bringing that up as, oh, this is a real pain point for me, then it's time to examine whether you're even looking at the right problem space. Will someone pay for something?

If it doesn't feel like a pain point is always an important question. 

 Are there other questions that you find particularly illuminating or is it all about the follow-up, so somebody says, you know, I log into Salesforce and then you say, well, why did you log into Salesforce? Is that sort of the primary way?

Or are there other types of questions or ways you might use that time to generate the most kind of useful. 

 It really depends on where the company is, but a few questions that I particularly enjoy asking to uncover opportunity. One is, tell me about the last time you thought there's gotta be a better way to do this.

I really like that question because it gets people thinking about what is the last time that my technology let me down, or that I encountered a problem in my life that I thought should be solved. And wasn't, those are really interesting opportunities. So I always like to say, when is the last time you thought something should be possible?

It wasn't. I think the other generic ones that I like are tell me the last time you had a really good experience. And I don't say with technology, I literally just say, tell me the last time you had a really good experience, because it will tell you about the things that they value. And that's another important one to know is what types of things do people value in their experiences and their interactions, because that can inform a lot of the design decisions that you wind up making and where you wind up investing your resources in terms of what you build and how you build things.

So those are two that I kind of find myself going back to over and over again. Also, when I'm trying to understand more specific workflows with people, I'm a big fan of the magic wand question, which is if you could wave a magic wand and change one thing about how you're doing this today, what would it be?

And that's another one that gets you at more targeted opportunities, as well as the things that people most care about solving in the way they get work done. 

 If we were going to focus on more specific questions, let's take a fictitious example. Like you and me are going to go create a new piece of software to help user researchers do their job better.

I don't know, something broad like that. And you're starting to do this very early customer understanding work. How would you set that time up? Are there specific areas that you would go outside of these core questions that you outlined? 

 I always base my research on what are the decisions we need to make based on what we find out.

And that's the Genesis of those. What are the three bullet points we're going to walk out of here with? And so for building a new user research tool, the top things I would want to know are how are the people getting research done today? What parts of that are working well? And what parts of that. Are not working well.

And so I would start by saying, all right, walk me through the last research project you did. And then I would just say, oh, well, I set it up. I started by pulling an email list of our participants and they'd say, oh, all right, well, what tools did you use for that? Tell me a little bit about how that went.

Were there any problems did you encounter any stumbling blocks? And so with that one, I really like to make them either workflow or project based because you can start digging in, in that way. So that it's a little bit more targeted, but it's still a storytelling experience. And so every step of the way we, like what I'm hearing you say is you normally start by pulling an email list and putting it into an Excel file.

Tell me what you do next, slowly, creating the story and feeling out the way they feel about each point in it. Every time by the end of that, ideally I should have a full picture of. Their research project workflow. So from the very beginning, like how do you get research projects? Where do the ideas come from?

How do you know you need to kick one off all the way through, what do you do with your findings? Once you've finished the research project, what happens with them then? And then oftentimes I will even have sketched out that narrative arc as we're talking. And I'll say, so if you had to circle the part in this project, that was the most difficult.

Tell me which part that is and why I do the same thing with which part of this was easiest, which part of this required the least effort. Why was that? And then I will get at the wish questions as I think of them, which are always like, what do you wish you could do right now that you can't, or tell me about your ideal experience.

And that's not because I'm trying to get their solution ideas, but because again, I'm trying to get at what they're trying to accomplish and what they value most and what is most difficult. I have a general rule that I don't listen to any solution ideas that come out a few X research interviews, which is probably a controversial thing to admit on a podcast like this.

But really I try to. Instead ask why they are coming up with that solution idea. So I always get it. Oh, well, what makes you say you would want a tool that does that? What is it about that idea that would be so helpful to you and really digging in any time someone is working on that level of solution to get at the underlying problem?

 Why do you discard the solutions? Is it the belief that people just are good at identifying their own problems, but not the right solution? 

 No, it's because I don't want to anchor to any given solution when I am in that exploration phase. And so when I say I discard them, what I mean, isn't I forget that I ever heard it it's that I specifically set it aside until I am done fully understanding the problem.

I don't include them in research write-ups I don't really include those things in my readouts or anything like that. It's just, I set them in a parking lot where it's like, here are some of the ways in which people have suggested that these problems get solved. But right now, what we're really trying to do is understand the problems to be solved themselves.

And at that point, solutions can act as a distraction. 

 Do you often have to sort through when people are unintentionally lying to themselves or to you, is that part of the way that you kind of root around in these conversations? So like an example that comes to mind for me is you're talking to someone and you're asking them about their Netflix consumption.

Then you ask them, what are the types of things that you like to watch? And they're, you know, telling you history, documentaries. And then if you actually look at what they watch, it's too hot to handle or, or something else like the classic revealed preference idea, I would just assume there are so many biases, not even a malicious desire to lie to you, but that you kind of have to sort through in some way.

 Yeah, absolutely. And that's where I think really the absolute best possible relationship to have in an early stage company is analytics and qualitative research. That is my one true pairing. And that's because, because you can understand both those things and I try never to discount the revealed preference versus the stated preference, because the stated preference tells you a lot about a person as well.

And so understanding how a person wants to view themselves is actually incredibly valuable because it tells you a lot about how you want to present them back to themselves and your product. I really value those moments where people are telling you that best version of themselves, because in many ways, a lot of opportunity

as we are designing and building products is to help people feel like that version of themselves. And so when someone is saying, this is the kind of person I want to be seen as even if you want to make sure that you still give them the opportunity to, and the ability to watch too hot, to handle presenting them with historical documentaries as part of their front page on Netflix makes good sense because it fits with how they want to project themselves onto the world.

And if you can mirror that feeling back to them with the product while still satisfying their actual preferences, what an incredible opportunity. I 

 also don't want people to think that all I've been doing is watching the last two seasons of "Too hot to handle" 

 no judgment if you have, it's been a long year, it's 

So if you are looking for something on the corners of the internet, that'll really suck you in to both seasons are pretty interesting. You can also learn a lot about people, it's sort of inactive anthropology. Anyway, one of the areas we didn't touch on in this process is actually selecting the people that you're going to talk to or study.

And I'm curious to get your thoughts on how to approach that. And maybe we could go back to our user research, fictitious product. Obviously there's something very different between talking to an individual contributor, user researcher. There's different talking to someone who has a team. A team of teams, an early stage company, a late stage company, someone that's earlier in their user research career, late people that aren't user researchers, but do some of that work in their job, maybe a product manager.

And so how do you think about that process of having divergent conversations with divergent ideal customer profiles and how that might evolve over 

 time? Yeah, that's a great one. And I think one of the things that I see a lot of startups struggle with is the tension between casting a wide net for product market fit versus selecting your market first.

And obviously there are advantages to both of them. So when we're thinking about our fictitious user research startup, I think the question there is, there are a few initial questions that. The company could ask itself, we could say to ourselves, do we believe there's an opportunity in helping career researchers inside companies do their job?

Do we believe there's an opportunity in helping non researchers conduct research and sort of segmenting our opportunities out that way? I'm a big believer in secondary research and lit review and market research as much as possible and kind of leveraging work that's already been done. And so if you can find materials that help you size each of those different opportunities super broadly, you're like, oh, there are a lot of other startups that are already operating in this one market, but this other market is under-penetrated.

And there's no tool that people are saying is really satisfying. This need. So starting from that kind of secondary research and seeing if you can narrow down your potential markets at all, saves you a lot of time because. Ideally once you've narrowed that down, you want to talk to quite a few people from each of those different potential markets and those different segments.

And so if you can narrow it down, at least even with just some best guesses and say, okay, we're going to prioritize our markets this way. We're going to prioritize non researchers. And then we're going to prioritize freelancers or people who run consulting firms. And then we're going to prioritize in-house researchers at this size businesses, because we think the opportunity is greatest in those three.

And they're all very different. So the more you can make some difficult decisions upfront, you at least are. Scoping your research effectively and making it easier for yourself to go out and deeply understand something for all that. I talked about how important it is to really broadly understand the problem space.

The thing that makes that digestible and useful is when you've already identified a group of people with something in common that you want to work with from there, I'd probably say, all right, let's talk to six or seven different folks who are non researchers that conduct research, and let's understand how they're doing it today.

What tools exist? What problems do they have? Is this enough of a problem that we think they might be willing to pay for a tool to do it. We could go into a complete tangent on monetization strategies and trying to identify monetization opportunities early stage. But for the purposes of this, let's just assume we want people to subscribe.

Or otherwise pay for our service directly. And so if we can create those segments and then deeply understand each of those segments, different needs and opportunities that lets us say, oh wow, okay. We thought that there was an opportunity in this market, but it turns out they're doing fine with the tools they already have.

Or when we started looking at this market that we thought was our third best opportunity, that's actually where things are turning out to be most interesting. 

 So once you have these conversations and ideally you've done them in a high quality way, what is the process to synthesize that into insights that can actually drive at maybe a macro level, what you're building, but maybe you're doing work on a new potential feature with an existing.

 Now that I am further along in my career, I play extremely fast and loose with synthesis and it mostly takes place in my head. That is not an endorsement of that way of doing things by any means. It's just by way of saying, there's no one process that will work for every single person, but the way that I mentor people in doing it, and the way I always recommend doing it is get transcripts.

Whatever else you do to save money in your early stages. So it's a start-up please, please, please record and transcribe your interviews with users and customers and people so that you can go back and identify and pull out these common threads. It just makes everything so much easier. And so when I am personally doing a more rigorous synthesis, I will have the key things that I wanted to learn from each of the interviews.

And I'll usually set up a document that just has headings. And then I will pull out all of the information from the transcript and just copy and paste it over into those sections. And then once I've sectioned, everything like that, I'll go through each of the sections and look for common themes. So in recruiting, it might be a pain point is.

Waiting on someone else to pull an email list or in post project, it might be figuring out where to put the research findings so that someone else can find them six months later. And then I will just put all of the things from the transcript that I heard into those different themes or stages and look for common threads.

And if I'm not finding any common threads, that's actually a really interesting thing to note as well. It means that either I haven't segmented my market as effectively as I thought. So I'm not finding people with anything in common or it means. That we're not turning up any major opportunities because nobody has the same pain points.

Humans are so good at finding patterns. Like it's one of my favorite things about people is that we are really, really, really good at finding patterns often to our own detriment. Like we find patterns where we shouldn't necessarily, but I think if you've gone through three interviews and you start thinking to yourself, I think I'm starting to hear a theme around this topic.

What I will oftentimes do is create an ongoing. Synthesis document where I start putting those, like might be a theme, but I haven't validated it yet kind of things. And so I will write all of those down and just be keeping an eye out for them. When I start doing my more formal synthesis, for example, I've been doing some research on custom content and how, and when people use custom emojis and things like that.

And I've only done two interviews, but I've already heard a few things that I think I'm probably going to wind up hearing as ongoing themes. I actually keep an erasable whiteboard on my desk and I just have them written down on there so that I can go back and include those in my synthesis process. 

 Is there anything you do to make sure that you don't create some level of confirmation bias where you start rooting around and hoping and pushing other people to say similar.

 much as possible when I start hearing themes, I don't chase them. I don't change my interview process at all. I ask the same questions and keep the same structure, even as information is coming in. And this is really the tactic I would use for these very early stage, very broad interviews. Obviously there are situations where you would want to use what people are telling you from interviews.

Like when you're doing rapid iterative, evaluative testing and updating prototypes in between sessions and things like that. So the first thing I do is I don't change my interview approach at all, even as I hear themes starting to come out. The second thing I do is when I start hearing a theme, I write, write it down partially because writing it down makes me more cognizant of the fact that it is appearing in my head.

And then the next thing I ask myself is what would need to be true of the next several interviews for me to feel like this wasn't a theme. And I will oftentimes seek out information that goes against what I've heard to pressure test it. So if I think I'm hearing a theme around recruiting, being the most difficult part of a research project, I will deliberately go in the opposite direction where I will say, tell me what the easiest part of recruiting is, or what's the most difficult part of your research process so that I'm giving people the opportunity to give me an answer that I'm not expecting.

 So once you start to collect these themes, what does that process look like then to translate it into product? 

 That's my favorite part. And it is one of the reasons that I love working in-house with companies, because the best possible way to do this is to have your product and design and engineering folks with you throughout the course of the research so that all of you are working together to identify these themes and come up with ideas for what they mean for product.

Now that is not always feasible or even desirable, depending on how quickly your company is trying to move. So the number one thing that I think helps with this is starting by deeply understanding. A team's roadmap and what they're trying to accomplish and what decisions they're trying to make. And so the first thing I ask teams, when they come to me with research ideas, or when I'm trying to help them create a research roadmap is really what decisions do you want to be able to make?

Jane Davis shares the playbooks she’s honed leading user research teams at Zoom, Zapier and Dropbox, tackling everything from specific questions to ask in customer interviews, to frameworks for getting feedback as you launch and prototype products.