I’d argue that references are the single most valuable tool in hiring, but I find that most people don’t agree with me. The problem isn’t references themselves, but how we run them.
The standard practice for executing a reference call is generally horrendous in both tech and venture, and it has been for many years. The typical format is a lazy back-and-forth between a handful of generic questions and overly glowing answers from someone the candidate handpicked, all in a sub-10-minute call. At best, it adds no value and, at worst, it does active harm by just confirming the direction you were already leaning.
Yet many of the critiques you hear against references overlook the fact that traditional interviews are themselves quite flawed. And how could they not be? What we’re asking of them is an extremely tall order. Over a handful of hours in interviews, we’re trying to understand who someone is at their core, size up their current capabilities, project out their future skills and get a sense of the conditions that would allow them to do extraordinary work.
Hiring is like trying to get married after a second date. It’s a bit of a miracle when the match actually turns out to be excellent. Anyone who has done a lot of hiring (and marrying, I suppose?) knows that you’ll have a pretty good sense of how things are going within the first two weeks. And of course, with an even longer time horizon (say, if you could remake the decision in six months), you’d be radically more accurate.
But unlike dating, you often can’t run months-long trials when hiring. Faced with those constraints, we assemble proxies to try and speed things up, like work projects, Zoom and in-person interviews, and reference calls. We’re all trying to patch up the gaps in our understanding and “buy” more hours together.
References are one of the more underrated ways to do that. Here’s a bit more on the process I’ve kept refining over the years and come to call “composite referencing”:
Reset on what references are for
The way I think about taking references is that they go better if you treat them like a police sketch. You’re the artist tasked with drawing someone you’ve never laid eyes on before, working from what witnesses describe. There will be conflicting accounts and outright mistakes, and each person will remember different details. But question by question, you’ll be able to slowly fill it in until the face starts to resemble the real person.
It’s not going to be photorealistic, but neither is the picture you’ll get from interviews alone. (That’s a strange mix of overly flattering self-portraits from the candidates and your own attempts to draw their face from memory after a few hours together.)
Very little of the reference process involves outright trying to figure out if a candidate is “good” or “bad.” I don’t really believe there are good and bad employees, only those who excel or struggle in the context of specific roles and company cultures.I tend to find references to be especially helpful for assessing someone’s base wiring. Are they competitive or not? Are they detail-oriented or not? While there may be room for improvement on these kinds of dimensions, these traits are largely hardwired by the time you’re meeting a candidate. But it can be hard to get an accurate read on these qualities in an interview alone.Note: I’ve been most inspired by Graham Duncan’s classic post here and I highly recommend giving that a read (or a re-read).
Up your numbers
If you had unlimited time, you would do 12-15 references per candidate, but that’s admittedly very difficult to pull off and likely unwise to do across the board.
Hiring is an exercise of working within constraints where you’re trying to figure out how to maximize what you learn per unit of effort. You’re constantly trading time for accuracy, trying to get the balance between efficient inputs and quality outcomes just right.
For each role, there’s an optimal place to operate on that curve. The question to ask is: How consequential and how painful is it to get the hiring decision wrong? If you’re hiring a CRO, getting that wrong is immensely costly for startups, so I would do the full 12-15. But hiring college interns is a very reversible, low-stakes decision.
Here’s my rough rule of thumb:
- Executive hires: 12 or more references
- Mid-level hires: at least 5 references
- Junior hires: at least 3 references
I want enough firsthand accounts to start uncovering patterns and teasing out where people disagree, so just doing a couple of calls leaves an enormous amount riding on who you happened to reach. But quality matters, of course, too. 12 rote, superficial calls are just as much of a waste of time.
Blend front-door and back-door references
People dunk on front-door references quite a bit. Most folks you call up will rarely want to say negative things, so feedback skews overly positive unless the reference knows you well. Given this, many assume any names the candidate offers up themselves will be so falsely effusive that they’re not even worth calling.
But I’d argue that in most cases front-door references are extremely poorly run and that is why they produce so little signal. Candidate-provided references can be just as insightful as backchannels when done well, and the most accurate read comes from blending the two.
It helps to run every reference through a decision tree that starts with one question to see how much spin there is on the ball: Does the person have more allegiance to you or to the candidate?
If the reference happens to be my best friend in the world, I can call them up, chat for 5 mins, and cover everything I need to learn.
If someone is more neutral (doesn’t know me well and didn’t work super closely with the candidate either), they may be able to tell me quite a bit about the candidate’s reputation inside the company. Getting that read on their vibe is valuable, especially if the company is in the hundreds rather than thousands of people.
But the most common configuration, of course, is that the reference is closer to the candidate than they are to you (aka the front-door reference list). You could write this off as totally useless, but if you’re ready to put in the work to get curious, you can unearth lots of useful information on who they are, what they care about, what environments they will or won’t do well in. The more aligned the person is with the candidate, the better detective you have to be.
At the end of every single reference call, I ask the reference for the names of other people who would have a perspective on the candidate and use that to expand my list of potential people to call.
Some hiring managers are hesitant to lean this heavily into backchannels, as candidates might understandably be worried about anyone finding out they’re considering a new role. (I do think this tends to be more of a corner case than people think, though.) As I collect more names, I loop back to the candidate: “Here are 5 people who came up. Are you comfortable with me reaching out to them? Why or why not?”
Don’t save references for the end
Hiring managers also tend to leave references until the very end of the process, when their ears are happy and the offer letter is essentially ready to go out the door (which further contributes to the feeling that the references weren’t all that helpful in the decision-making process). The ordering is probably the easiest tweak to make, and I’m continually surprised by how few people do it.
Normally, by the time I’ve done a project with a candidate, I’ve done 2-3 references. I aim for roughly a third early, a third in the middle, and a third at the end.
When you save references for the end, you run the risk of rationalizing away the negative feedback because you’ve already arrived at a decision. As our friend and colleague Annie Duke often reminds us at First Round, your gut and your ears are filled with confirmation bias. Starting references earlier gives you a chance to learn new information before you’ve become overinvested in a particular answer. (It also seeds useful questions to bring into your next interview with the candidate.)
And even if you do manage to avoid that bias and end up passing on the candidate, now you’ve sunk all this time into a process you could have cut short earlier. Imagine you have a 30-minute Zoom interview with a candidate that was quite positive. You then learn your best friend worked with this person for 6 years. If you ask your friend over text for quick feedback, you probably wouldn’t decide to spend another 20 hours with the candidate if your friend says they were atrocious. How can you arrive at that moment earlier without the built-in best friend connection?
After the second interview, I ask the candidate for permission to start references, loosely using this script:
I approach references a little differently. I’ve found the best way to understand someone is by talking to people who’ve actually worked with them, not just through interviews. So I like to run references throughout the process, starting now. Would you be open to me speaking with a few people who know you well professionally? If so, could you share three names? If not, when would you be comfortable with me doing references?
I also believe references should go both ways, so I make sure to always add this too: “To understand what it’s like to work here, you should feel free to reach out to anyone on our team, past or present.” Very few will actually follow through on this, but it’s an important offer.
The art of rooting around and asking high-signal questions
Many people just wing reference calls, asking a few generic openers that stick to the surface (such as “What are his strengths?” or “What’s she like to work with?”). In my experience, these calls actually require a great deal of prep work and care, largely in the form of sketching out very specific questions to ask.
The best kinds of reference-checking questions do one of the following:
- Reveal insight rather than just judgment (i.e., questions without a clearly good or bad answer)
- Prompt stories instead of adjectives or superlatives
- Encourage comparison and contrast rather than simple absolutes
- Dig in many levels deeper after a high-level answer
For each candidate, I start by creating a focused list of 10 “burning questions.” These are shaped by three things:
- The core traits someone needs to thrive at my company
- The specific skills required for the role
- The biggest open questions I still have about the candidate and their capabilities
Say I’m hiring a VP of Marketing. If I think outlier levels of agency and speed are the core traits someone needs to thrive at my company, here’s what I’d ask:
- When you think of all the people you’ve worked with, where would you rank CANDIDATE on their level of agency?
- When you think about the #1 highest agency person you’ve ever worked with and CANDIDATE, what was the gap between the two of them?
- Can you tell me a story of how CANDIDATE reacted when something didn’t go their way?
- How does CANDIDATE handle people or systems that get in the way of their progress?
- How impatient is CANDIDATE? Have you ever seen their impatience create positive results? Negative ones?
- Would CANDIDATE be more effective at your company if they sped up or slowed down?
- When things are moving slowly, how does CANDIDATE react?
- How does CANDIDATE approach speed vs. quality decisions?
If I think recruiting is a particularly important skill for this role, here’s what I’d ask:
- How would you compare the quality of CANDIDATE’S team vs. other teams?
- Which members of CANDIDATE’S team worked with CANDIDATE before and came over to your company?
- Who did they attract to their team that they probably shouldn’t have been able to?
- When they needed to build a team quickly, how did they go about it?
- Compared to the best recruiters you’ve seen, what’s different about their approach?
If I think the biggest open question is around their ability to invent, here’s what I’d ask:
- What’s an example of CANDIDATE solving something in a way that surprised you?
- How would you contrast their ability to invent versus execute?
- What are examples of ideas, processes, or systems they introduced that changed how the team operated?
- When they inherit a system, do they tend to tweak it or reinvent it?
- How often do their ideas become new norms inside the organization?
Then I supplement with core reference questions, pulling from my bank of many dozens:
- What is your overall narrative of CANDIDATE?
- If you had to rank CANDIDATE in the top 1%, top 5%, top 10% or some other percentile, where would you put her/him? (Follow up with… Just curious, why didn’t you rank in the top 1%?)
- In as much detail as possible, describe the difference CANDIDATE made in your business. Said differently, if CANDIDATE never worked at your company, what would the difference be?
- What are CANDIDATE’S blind spots?
- [If the person was their manager] Can you walk me through their last performance review? What did you deliver to CANDIDATE?
- How was this person perceived by others?
- [If the person is no longer at the company] How hard was it to recruit the person behind CANDIDATE when they left? How would you contrast the new person vs. CANDIDATE?
- What are the person’s key personal values?
- What do you think are the primary drivers for CANDIDATE – what motivates him/her?
- Can you share someone who maybe doesn’t necessarily get along with CANDIDATE as well as you do? Why do you think they don’t mesh well?
- What’s one thing people misunderstand about CANDIDATE?
Just as in customer discovery, you don’t want to fall into a pattern of sticking to your script. Reference calls require very active listening for very subtle cues, which will often lead you down unexpected rabbit holes. The questions are just a starting point. It’s your job to notice the breadcrumbs people leave along the way and follow them.
The mechanics of running the call
I aim for 30 minutes and prefer Zoom over phone calls whenever possible — you learn a lot from facial expressions and tone.
I start off by saying something like:
I approach references as a way to understand the complete picture of a candidate, not just whether they’re good or bad. I also don’t believe there are good and bad employees, only good or bad in the context of a role and company culture. I’ll run through a series of questions with that goal in mind, and everything you say will be kept strictly confidential.Then I say: “To begin, can you share how you worked together and how well you know them?”
I typically spend half the time on my prepared questions and half on follow-ups — the follow-ups almost always lead to the richest insights.I try to leave as much whitespace as possible to let people talk in an open-ended way. For example, “Tell me your narrative about Sally. Imagine I’ve never met her and I’m trying to understand what she’s all about. What is the story that comes to mind for you?” Fight the urge to fill the air and say anything until there’s truly awkward silence. (There’s a wide range of responses here, some will talk for 5 minutes and some give you a short 30-second answer.)
I end every reference by asking: “Who else at your company knows this person well?” Those names become the next set of calls, especially for executive hires, where I’ll often repeat this process several times, cold emailing the people who were mentioned.After the reference, I take the Granola transcript into ChatGPT and use the prompt: This was a reference call between Brett and [NAME]. Please turn this raw transcript into a 2–3 page summary that captures the key points and themes discussed. It should be written for others on the interview loop who weren’t on the call and include as many direct quotes as possible vs. a high-level summary. Do not overlay your personal narrative, rather focus only on facts from the reference call.
I then write down my new open questions or areas to explore, both in upcoming interviews with the candidate and in future reference calls. Then I repeat the process throughout the hiring cycle, refining what I’m learning and sharpening my thesis each time.
Weighing what you hear
The final step is deciding how much weight to give what you’ve heard as you try to complete your picture of the candidate.
In general, you are very likely to hear some variation of “I would kill to work with Tom again, he’s one of the best and made an enormous impact here.” Every company and manager has different standards, so even if you hear this type of overly positive feedback and it happens to be true, it doesn’t always translate to success in your environment.
But if you hear something that’s even a click away from that in tone and tenor, that’s something to get extremely curious about. Anything along the lines of “Overall I’d say John was very good and I’d consider working with him again” or “In the right circumstances, I’d probably work with him again” should have you leaning forward in your chair.
If a reference seems to lack specificity and details in all of their answers, no matter how targeted my questions, they generally go low in my sorting algorithm as someone not close enough to the metal.
But beyond unpacking what they tell you, you need to put in your own work to figure out whether the reference is well-calibrated. I like to:
- Review their LinkedIn to understand their career arc, time in role, and an initial sense of their own ability
- Run enough references so the chances of talking only to poorly calibrated people are low. (If 1 call out of 10 is negative, it’s more likely an issue with the reference than the candidate.)
- Cross-reference the people you are calling by briefly mentioning who else you are speaking with
- Check with trusted friends who know the reference to get a read on their talent bar. For me, doing references on references tends to look like light texting, and is something I usually reserve for more high-stakes hires. (“Talking to Jane about a candidate for our VP of Product role – any quick thoughts on her?” type of thing.)
If you do have the time to do a number of reference calls, try to connect the dots. The themes that run across many calls are generally most likely to be reflective of reality. If you do 8 calls and you consistently hear that a candidate is immensely talented, but very difficult and a bit of a bull in a china shop, that’s likely true.
But the decision of whether to hire them or not is very context-dependent on your culture — it’s less of a question of is that the right or wrong way to act, and more of do you want that kind of person at your company?
It takes a considerable amount of work to collect these different inputs, and even then, it’s still ultimately a call of human judgment. You’re never going to know as much as you would after months or years of working with someone. But I’d much rather make that decision after spending serious effort getting to learn from the people who already have.