The product wisdom every CPO should ignore | Jeremy Epling (CPO, Vanta)

Brett: All right, let's do it. Thanks for joining.

Jeremy: Yeah, thanks for having me.

Brett: Normally when you look at someone with a background like yours, basically 20 years in various roles at Microsoft, they then join not a small startup, but a scale up startup, certainly relative to Microsoft, a tiny, tiny company.

Jeremy: Yeah.

Brett: In 90 days, it's a complete disaster and the person leaves and goes back to sort of the wonderful mothership. What is it about you and what you figured out and maybe the specific things you worked on at Microsoft that you think allowed you to be effective in after such a long time, changing context so dramatically?

Jeremy: I think that for me, right out of university, I had a startup. So I guess there's a little bit of that DNA in me. Lasted for nine months, ended up getting me a bunch of job offers, but didn't really go anywhere. So this is back in the day when the bubble had just burst on that first wave of internet companies. I mean, I started at Microsoft in 2002, so it was right around then. And so I think that a big part of the learning for me was I always wanted to switch teams and I've always wanted to learn new things. I think it's just part of my DNA. And so even when I was at Microsoft, I tried to switch teams every three or four years into a brand new product area. So when I started off, I was on Windows working on security and deep technical things. I was on Internet Explorer for a while. Then I joined OneDrive and that was like a zero to one and I really enjoyed that process of zero to one. I think a big transformative thing though was going to GitHub. When Microsoft purchased GitHub, I came over right after the purchase to lead all the new product development there. And to me, Nat was just very much a founder, CEO. I mean, obviously he's exited a ton of companies, been a VC, done a bunch of amazing things. And that company was run very independently. I literally quit Microsoft. Your contract ended. It was like we used all Google tools, everything was built on AWS, everything was Zoom. When Satya sent emails the company, they didn't go to anybody that worked a GitHub. Badges didn't work back and forth. HR, benefits, levels, everything was completely different. And so I think that really helped me get a taste of what it would be like. And I think one of the things that I loved was just the tight connection across the entire business. I feel like there's a lot of great things about Microsoft, but in my experience there, you kind of feel disconnected from the go to-market motion. It kind of feels like it's on another planet. It's like you're just trying to build the best product. And then somewhere over in the ether, lots of money is made that feels pretty disconnected. And I think at GitHub, I got to see it all come together. It was like being deeply involved from our product, to the messaging, to the marketing, to the revenue, going deep with SEs, thinking about our positioning, all the pieces. And that just really excited me. And I was like, I want more of this. And then as GitHub grew, which is great, it kept growing and growing. I was like, actually the most fun part was the earlier stage.

Brett: What's some of the stuff in the context of the GitHub experience when you went from being very product centric to being much more full stack and at least owning or involved in a bunch of the commercial elements? What were some of the interesting learnings from that experience?

Jeremy: I think some of the big ones was, one, I developed a really close relationship with our head of SE, which I think was really important. I think when you're doing a technical sale like GitHub, the SE team plays a huge part. AEs are critically important for setting up the relationship or navigating the org chart, doing all those pieces, but building that tight feedback loop of understanding practically what is working and what isn't each time they're going through the sales cycle and then how they're implementing with customers and driving that feedback back into the product. I think that loop I had a little bit at Microsoft, but was not nearly as tight as it was at GitHub where I felt like I was talking to our head of SE almost every single day. And I was working on new products. You're kind of in that stage of like, you really need to figure it out. Nat, I think really pushed a culture around understanding our marketing and positioning. So I got to spend a lot more time understanding how we're telling our story, how to tell our story a lot better, how we can go through and pricing and packaging, I think was something that I spent a lot more time on than I had at Microsoft. With GitHub Actions, that was like one of the products I built while I was there. We were doing usage-based pricing. It was the first time GitHub had ever done usage-based pricing on anything. And so how do we think through that transformation? The nice thing is I could still go and talk to people in Azure that I knew and worked with of like how did they go through that transformation, but then think about what does that mean for GitHub where, hey, we're selling a product every day. People are used to just buying licenses for every developer on their team. Now we're telling them to think about compute and storage and how do they estimate that and how do you make that friendly for an open source developer? How mch should we give away for free for open source? And so I think that was a really eye-opening experience to me of just the entire impact of the product holistically through the business of the pricing and packaging side. I spent a lot of time thinking about COGS. When you're running large infrastructure like CICD for these workloads at massive scale, it's very expensive. We actually racked our own Macs in our own data centers there. So thinking about the hardware supply chain and how long that takes of like, okay, we think there's going to be a new Mac Mini coming out here. How do we need to go think about the networking setup and all those different pieces? So I think it just exposed me to a lot more breadth of the business and made me appreciate everything that goes to delivering a great product experience versus just the individual kind of features or the smaller bubble, at least in my time at Microsoft. I'm sure things have changed there that you were kind of kept in there.

Brett: So with all that as a little bit of context, when you think about the job of a chief product officer in the context of sort of a scale up startup, how do you define excellence?

Jeremy: I think for me, you have to be in the details. I know there's been a lot of talk for the last year or so around founder mode or not. I'd say I'm definitely on the side of the founder mode interpretations like you need to be in the details. So for me, I want to understand what is the experience we're delivering to customers and how it's great. And I think the more disconnected you get from those, like when you start to become more of like a professional manager, I'm just guiding the team, giving strategy and things, I think you lose a lot in making sure you're building the right thing for customers. Because if you're not in those meetings, if you're not selling the product, if you're not demoing it and using it every single day, I think that's when you kind of get this disconnect. And I think that's what I see happen at times at larger companies where they get there. You kind of end up with this kind of professional management class and then the people doing the work. And then I think that's when the product starts to get confused about what it's doing because not everyone's using it and knowing where it's going and why there's a clear direction behind it.

Brett: So that's sort of one piece. What else is in the job spec for a chief product officer?

Jeremy: I mean, I think for me, understanding, one, of being able to deliver at pace, I would say that is one of the biggest things. You need to keep up a high velocity set of new features coming out to customers, making sure that you're having a really strong win rate. I'd say win rate's like a metric that I think a lot about, especially if we lose, why are we losing? Is it pricing and packaging related? That's something product can go do work. Is it like a product efficiency? What can we go do there? I think on the side of being able to cast out the vision, I think it depends on which CEO you have. Some CEOs want to own all of that. Some of them want to delegate some of it or part of it to the chief product officer. So I think understanding where you are there of how much you're going to be doing on the vision, telling the narrative and the story. I do think even with marketing being able to tell amazing narratives, the product team needs to be able to explain, because you're the one talking to the customer, building the product and be like, "Okay, we're building it for this persona in this way," and be able to transfer that knowledge over. In my case, I also lead engineering. So for me, there's also a technical side of the job as well of understanding the architecture, how we're doing on SLAs and scale, making sure we're building for the future while trading off for short-term things that we need to go deliver, especially when you're in that scale-up phase. If you're joining at like a B or a C, I felt like a lot of my conversations with CEOs when I was trying to figure out where I wanted to be was, "Tell me how fast you can move and how you move quickly. Tell me how you will help us build into a multi-product strategy." So how do you think about adjacencies, where you can go into those adjacencies, how do you make decisions around which ones? How do you verify if the decision was good or bad because not every one's going to be the right decision. And then I think one of the things I talked about through that process was also who I am and was trying to figure out what they were looking for. So I was like, "Hey, I care a lot about product quality and building an experience that's well crafted and beautiful and usable and customers love it." And you don't necessarily have to do that to build a great company. And I was like, "But I'm going to chafe against the system if you don't also value that." So I remember having conversations with Christina and other founders around like, "Hey, what gets me excited and where do I think you get the most out of me?" Versus like, "Hey, actually you should hire someone else because I'm not going to be a good fit here."

Brett: With sort of that level of detail, when you think about the chief product officer role in the way that you define it versus a senior vice president of product or a VP of product or director, whatever in your language would be the step underneath, how do you define the gap between what is expected in those two distinct roles?

Jeremy: I think in those roles, the big thing that changes is you are more focused on the C-team as your first team. I think when I got this role as my first chief product officer role, I talked to a bunch of my friends that were founders and I was like, "Hey, what's the biggest mistake that I would likely make in this role?" And they're like, "You just focus on engineering product and design and you're not actually working with the C-team." And they were like a million percent correct in that advice. And so I think that it's very easy for me, and I even like default to this because I spent most of my time in engineering and product design. It's like my happy place. I want to go in there and work with the team and think about how to make the product better, but if we're not connecting it across the rest of the business, we're not going to have the impact we need. So I think that is the biggest step change. I'd say the other thing is just a lot of the buck stops with you, especially with the way we have it structured at Vanta where I have product design and engineering. Part of the mindset I think from Christina for that is everything that goes into product development is my responsibility. So setting the pace, setting the quality bar, being crisp on the vision and what we're doing, creating that clarity felt like when I was a VP or if you're an SVP or anything before, you always have your manager to go and bounce ideas off of or know that they're going to be setting some high level guardrails. I think for Christina, she needs to be focused on our entire business and company. And so her scope is massively large. And so with me, that 100% rests with me to set all those things on pace, tempo for execution, where we should be going, why, digging into a bunch of the customer specific issues that anyone's having, debating big strategy trade-offs around how much we're investing in our downmarket business versus our upmarket business. So I think there's just a lot of kind of weight on your shoulders probably in that role where you just have to get really comfortable with just making a lot of decisions and there's no one else to really hand them off to or you're not really doing the job, at least the way we have it set up at Vanta.

Brett: What does it look like to do an excellent job being a team member sort of at the executive level other than just having relationship, connectivity, helping out, what does it mean to do that at a 10 out of 10 level?

Jeremy: For us, I think it is us all coming in with the problems that we see in the business or the opportunities and being able to share them in safe, trusted space and to be able to work through them together as a team and then represent that out as a unit. So let's say if we were like, "Hey, I'm worried that win rates," the CRO could come in and be like, "Hey, I'm worried win rates are dropping here. What are we doing?" And she feels like we can come back. And actually I would prefer ... I mean, to be happy when Stevie and I talk about that one-on-one, but almost getting it as a group so we all are on the same page because maybe there's something marketing can help out with. Maybe finance should know because there's a thing around pricing or whatever else or some data analysis we would want them to go do. So I think being able to bring back those to the group and us spend time wrestling with what those are and the big strategy decisions. And I think the other thing you need to do as a CPO is just understand the altitude that your communication needs to be at. When I'm working within engineering, product and design, we can go super deep and everyone knows every feature of the product and get really technical and do all these things. When I'm working with our CRO or our head of people or whatever else, they are not as deep into these things. So being able to kind of re-level the conversation back to like, okay, what did they know about the product and how am I communicating with them in a way that we can move the whole business forward?

Brett: When you think about operating at the executive level and you're working on a problem that a CRO is bringing you, or a chief people officer and you have the exec team working on it, do you think you're doing your job by bringing the product and engineering hat into that specific problem or are you all just, it wouldn't matter if you were doing something entirely different at the company. It's not like I'm bringing the product to an engineering viewpoint, someone else is bringing this viewpoint, someone else is bringing that viewpoint.

Jeremy: I would say a lot of the time it feels like I'm bringing more of the product and engineering viewpoint, but there's definitely things that are just kind of objective conversations around like what are the core values that we want to have around the company and how people can work together or how do we think about remote work versus not remote work? And some of that is like each team could be different and have a different approach to that and we want to allow that or we don't, but I think that's more of like a general discussion. I think a lot of the time I'm trying to bring the product insights in a consumable way because I do think there's like a difference in the CPO role when how you think about the product versus some of the other team members is just, you're probably spending a lot more time with customers than some of the other people are. Like, we have amazing go to-market leaders at GitHub, but a lot of the times a lot of what they're doing at the executive level is building the system in the machine, but aren't probably in as or might not be in as many customer calls as I am. And so I think that, hey, let me bring this customer perspective and then let's also understand what's the general go to-market perspective and the specific things we're hearing from AEs and SEs and then try to like work through it together. I think some of the big ones for us is always like, what's that feedback loop? Competitors change sales plays, they'll maybe be dropping pricing, they'll ship a new feature. And I think the key to me would be getting that group together and being like, okay, do we see one of our key competitors making a change here, they ship something really interesting. What do we want to go do as a group? Do we need to go change the roadmap? Do we need to respond? Do we think our product is just hands down better and existing messaging works or do we need to go tweak the messaging? And if we need to go tweak the messaging, how do we get that out to the field really quickly? And I think you need everybody in the C-team to kind of be aligned because maybe there's like a pricing change we would want to go make or not. Maybe there's a different go to-market motion, maybe there's a different like marketing play that we should go do or maybe it's like, oh wow, we got out played and we should go change something in the product, or they exposed a gap or something that we never thought about and we should go ship a fast fix versus like, actually this is going to take a long time. So how are we going to try to change the buying criteria for those customers? And then it becomes like a messaging thing. So it's like, oh, it's going to take EPD a month or two to go build this if it turns out it's a gap. What are we going to go tell customers about and how do we emphasize where we're even better? I think those are the conversations to me that get the most exciting and productive. And I can bring that product lens of how we can change the roadmap, how we can't, what I'm seeing in the different customer calls, but then still learn from what the go to-market team's seeing, what the marketing team's seeing.

Brett: How do you not create a dynamic where everyone's blaming everyone else? I think it's very natural, particularly with high ambition execs. If there's an issue in win rate, the normal thing would be the CRO would blame the chief product officer, they haven't built the right thing or they didn't-

Jeremy: Yup. Yeah.

Brett: ... have the right features and functionality. Maybe other than just the culture that we're building, do you have any reflections on how do you actually create a culture where here's a problem and we're all going to work on this, not blaming it on everyone else basically?

Jeremy: I think it's really hard just to be frank. I think there's this hard dynamic between the revenue team and the product team that's fundamentally kind of at odds to some degree where the product team is always thinking further out. That's how the compensation model is based for the product team, like primarily salary, also probably heavier on equity. And then you have the revenue team that's like, especially on the downmarket focused business, it's like month to month quotas. Maybe as they get bigger, it's like quarterly quotas and things like that, selling up market, and they're extremely reactive. And I think you need both parts of the business to be successful. You need to be responding quickly to customers and you can't have EPD like navel-gazing about the future when everything's going to be better and just focus on strategy. So I think the biggest thing for me is like trying to build shared truth between those two things. So I usually come back to like the customer feedback loop I think is interesting. I think I've had this experience in the past where the product usually falls into a bad place when it feels like the product team is just like order takers from like the revenue team. You know what I mean? Where it's like, "Customers complain about these five things, go fix these five things," and it becomes very reactionary. And then you don't have much of a strategy of like where you're going. It's just like you're living-

Brett: But there's also attention because there's a lot of goodness in that too.

Jeremy: Right, and those are going to land deals. So you need to be able to do a certain amount of that. You just don't want it to go all the way there. So one of the things that we've been doing is trying to do a good job also sharing like, "Hey, the product team from all the calls we're in, here's what we're seeing from customers. Is it similar or different from what the revenue team is seeing?"

Brett: When you think about this sort of fundamental tensions that we're talking about between product and revenue, for example, how much of this solve is just a really excellent relationship between the chief product officer and chief revenue officer? And that it's two humans and if they generally enjoy and respect one another, it solves 70% or no, you can have that and there is a core, there's a physics of it all that you sort of have to manage around.

Jeremy: Yeah. I mean, I think that's the core of making it successful. And I think that's something that just Stevie and I have leaned a lot into is understanding like, "Hey, how can we quickly and easily work through things with our teams between the two of us?" I do think there's always a bit of built in tension in the system that I actually don't think is a bad thing. I'm comfortable with conflict and things and thinks like companies are more successful and more comfortable they get with conflict and don't look at escalations as bad things. I think a lot of it comes down to that relationship, like can you share what you're really concerned about? Let the other person listen and take that in and then go back, because there'll be feedback going the other way too, right? The product team could set in a couple sales calls and just be like, "Oh my gosh, we just shipped this amazing new thing and no one's talking about it. Why aren't we talking about it?" You know what I mean? And maybe that's a marketing thing or maybe it's like a sales training thing or maybe it's just that seller didn't know or we haven't gotten the pipeline going. So I think that open lines of communication is really important. And I think that's what becomes that kind of like C-team thing that we mentioned before of just being able to share like, "Hey, we're seeing this." And I think it's like coming less with solutions always works better, probably on both sides, but I can at least say on the EPD side, when someone comes to me and it's like, "Hey, we have this problem. The competitive situation has changed here. What should we go do?" I always feel like it turns me onto a more receptive side. Then I get the like, "Can you please ship these three features tomorrow?" And then I'll just be like, "What's the actual customer problem?" Maybe that is the right solution, but maybe it's not. And so I think the more everyone's interacting that way of like, "What's the problem we're dealing with? Let's go look at it and see what everyone can go do to make it better." That's the best way to make it work.

Brett: So how do you think about good conflict versus bad conflict in the context of an executive team?

Jeremy: I like just talking about things directly. So I feel like people ... If someone's like, "Hey, I don't think this is working," I would just want to go and put it out on the table and just be like, "Great, let's go talk about it." And so I think when we end up dancing around things or having too many one-off conversations just because it's exhausting for everyone, and this can be like everyone's kind of talking about it but not together one-on-one and then you feel out of the loop or you hear about it third hand or whatever else. So I think one of the things Christina does really well is like we have a lot of time for the C-suite to get together each week and we go through and just be like, "Hey, these are the top topics that we need to go through and all of us can put things on the list and it can be stuff to do with our team or not to do with our team." And I think we try to approach it all as open and respectful, but also just be like, "Hey, here's the problem. I'm not happy or I'm worried that ACVs are going down here or I'm super excited that this product is doing really well and selling really well here. I'm thinking about moving even more EPD resources to this, but that would mean I'd be sacrificing in this other area. How does go to market feel about that?" And so I think it doesn't always have to be issues that you're talking about there. It can be exciting new things, but I always try to stay in front of things of what we're doing. I think over communicating is just key. I think it's something that I still am trying to do better is communicating more at the right level so everyone knows what's going on within my team because so much of what the product is doing just kind of has downstream effects on everyone else. So if we're like, "Hey, we've done a big bet on federal this year and we've gone through and done FedRAMP 20X. We got our low authorization and have been really involved with all the changes that are happening to security from that regard with the administration and Congress and everything." And that meant trade-offs in other areas like, "Hey, that's a bet that we're making." So that was something I brought to C-Team and I was like, "Hey, here's why I want to make this bet, the intensity. Here's what it would impact in other areas. How does everyone feel about that? Can we still hit the revenue numbers? Do we agree that this growth opportunity is the same? What does that look like?" I like docs a lot, so I'd say we have a pretty heavy doc writing culture, but we try to keep them short. So I don't know, I have a bunch of catchphrases, but one of them is clarity through brevity. So I'm just like, "Just tell me exactly what we want to talk about. Can we just get some bullet points and make a decision here versus some 80-page PRD, which I just feel like is kind of pretty outdated at this point." And at that point, I'd much rather just get a v0 or Figma make prototype or just show me pictures to see the customer experience. But on the C-Team side, yeah, I think it's talking about the hard problems. So it's not like there's an elephant in the room. Trying to be really constructive around what that is and then bringing as much data as we can to it as well. I spend a bunch of time trying to understand our product revenue data, looking at how things are going on the sales side, and then also on just our product usage data so that we can pull all these different pieces together. Our finance team does an amazing job pulling together deep product revenue and then tying it with the sales pieces and bringing that in. So a lot of what I like to go do on decision making is like, what do we need? What data do we need to make this decision confidently? Because the thing I hate the most, I don't know if you've ever been in this situation, Microsoft had this term called fetch of rock, where someone's like, "Hey, can you go fetch a rock?" You're like, "What rock?" And you're like, "I don't know, but bring it." And you bring it and they're like, "Actually, it's not the right one. Well, what one do you want? I don't know. Fetch another rock." And so I think that's the worst. So I think we try to define boundaries around things and say that, "Hey, here's the decision. What data do we all agree if we had this data on ..." Like sometimes it'll be gut. Sometimes it'll be real data. We can make this decision and then we can all represent it as a group because I think disagreeing and committing is also a big part of the job. There's definitely things where I don't know if I believed it and I was wrong and times where I was right and we didn't end up doing it, but we all represented as the group decision of like, "Hey, this is where we want to go and why, and we're all in it together."

Brett: Do you think exec teams are most functional when you have all sorts of different personalities around the table or is it much more functional when everybody's wired the same way? So the whole group is like a group of disagreeable people that like to debate and argue or you want one very opinionated, disagreeable person, you want more of a peacemaker personality, you want sort of that type of thing.

Jeremy: I'm not sure. I think with us, I would say if everyone's disagreeing all the time, I think it's going to feel like really painful and grinding for everyone involved. I think when it feels like someone's just becoming devil's advocate, that becomes negative from my perspective because it's just like I'm just trying to find issues in everything and any plan you can find issues in, you know what I mean? So I think trying to cap that to like, what are the impactful things that we're really worried about? One of the things we kind of do as part of our annual planning process is we'll go through and be like, "For each exec, what are your top five questions?" For us to close on the annual plan for next year, what's the five things that the CFO needs to know? You know what I mean? And he's like, "I have these five questions about the product or about revenue or whatever else." And then we can go through and be like, "Great, let's answer these questions. Now do we all feel like we're good?" Because there's always more questions, but trying to pick on some of those pieces, I do think if you don't have some amount of challenge though, you're just not going to push yourself or the company. So I think you have to be able to ask the hard questions. And I think that's something that Vanta has in its culture that Christina definitely encourages is like, let's just ask the hard questions and have the discussion. I think you just don't want to push it into devil's advocate like, let me always just be disagreeable on everything.

Brett: What about if an exec is not performing at the quality bar? Is it just the CEO's job to manage that or is there a responsibility of every other exec? And if so, how does that work in a non-backstabby kind of ...

Jeremy: I think in that case, I feel like the more senior you get, and I think this doesn't have to be at the C-team, even below. I try to encourage my team, like if my directs ... I've got engineering product and design and if they're disagreeing or someone's not performing, they'll try to work with each other first as peers. And I think that's the best thing. If I'm like, "Hey, I need this from finance or marketing or revenue," or if they feel like, "Hey, I'm not delivering something." I think them being able to come to me and just be like, "Hey, here's what I need and I feel like I'm not getting this to go do my job or I feel like these are the gaps in the area. Here's what they are." I just feed off details as a product person. So I think generic statements aren't as good for me. And I'm like, "Hey, in this doc that we wrote, can you tell me what in the math, what in the priors that you're bringing in, what is it that is making this not resonate with you?" And it's like, "Oh, great, here actually we disagree on this point and that's why everything below it doesn't make sense because you're like, we should zig, and I'm thinking we should zag here and let's go talk about that thing." But I do think working through it one-on-one, obviously the final decision of like, "Hey, how much coaching is too much?" And you're past the point of diminishing returns with that and you need to ask that person to move on or something like that. I think that ends up being the call of the CEO, obviously, because it's the manager of it. But I think the more that group can talk to each other, and then if you kind of just see persistent gaps in an area, I think it's on the CEO to figure out, "Hey, is it the system that's causing it? Is it individual performance failures? What can I go in and do to make this thing work?" Especially if the overall business is going well. It's like to some degree, I think we've had this joke in our board meetings that the board is harsher when we're doing well because there's no real problem to talk about. You know what I mean? So it'll be pushing on all these little things that kind of matter, but not as much. But if anything isn't going well, everyone knows what's not going well and then everyone's working on it. So there's a lot of scrutiny.

Brett: Something you mentioned at the start of the conversation is that in the configuration of Vanta engineering reports to you as the chief product officer, what are the pros and cons of that org design?

Jeremy: So I think understanding where your CEO wants to spend their time and where they're good at it really changes the shape of the role. Christina's really good at product, but doesn't want to probably think about and deal with disagreements between product and engineering on things. So she wanted one person to be like, "Hey, I want the kind of one throat to choke." You know what I mean? Of everything across execution, are we executing fast? Is there a high quality service? Is it scaling? Are we building the product that people want in those different pieces? And so product engineering and design are all together. And I think it lets her then focus on a lot of the cross company problems where she can talk to me about product direction and vision and what she's excited about and she can spend more time with revenue and marketing and everything versus having to manage a deep, another technical person or another design person and wanting to be in all of those more minutia conversations and resolving conflict there.

Brett: But when you think about the benefit, one is sort of creating an organization that's well shaped around the CEO, which is obviously very important. The other piece is sort of when you zoom all the way out and you think about, well, what are the benefits organizationally or to the company broadly? When you have a chief product officer that also owns engineering, you would say you gain X, but you do have to trade off Y. What is X and Y?

Jeremy: I think what you gain is a faster decision making process. So when you want to go through and decide, "Hey, do we need to do this big technical re-architecture and that's going to trade off against some product features?" You need to go through and make those. You have one person that's just going to go make that call and it's really easy for me to quickly resolve any of those discrepancies. Like design and product are often usually under the CPO, so I think that's more familiar for people, but I think it's that faster decision making. I think the downside is there can be a perception that engineering is not as important. I don't think we have that at Vanta. I think engineering is super important and I really empower our engineering leader to go through and own that. Like our engineering offsite, they have the biggest team at the company. You know what I mean? If they didn't report to me, their team would be bigger than the sales team and everything else, and they are responsible for that discipline, the hiring, the budget, all the different pieces there. So I try to hand off that. And so I do think the way I manage it is, yes, I'm the manager, but I also kind of treat it as a peer relationship at the same time where I'm like, "Hey, the two of us are in this together of how do we go build this?" And same with the head of design, but at the same time, I'm also the manager, so I get that it'll never feel like a purely peer relationship, but I think that is one of the trade-offs. If I had a company that required super, super deep technical expertise, I think that would be the case where I would be much more skeptical of having a single person that has it all and I would want a separate technology officer versus not. And I think we have really hard technical problems to go solve at Vanta, but it's a different type of problem than if you're trying to do low level 3D graphics or quantum computing or when Figma was figuring out WebGL for the first time and how to do design software on the web, I think those were different types of problems to go solve.

Brett: Another thing you mentioned is one of the more interesting questions when you think about a chief product officer in the context of a scale-up startup, which is the relationship between product and the role of the CEO. I think the reason that it's often most interesting is most technology-oriented companies are started by founders who one of their areas of expertise or the unique part about them is their product, taste, judgment, et cetera, et cetera. That's generally why the company worked in the first place. And so you often have this very tenuous relationship between the person leading product and the CEO. And to your point that you made at the beginning, it can take all sorts of different flavors. One is that you have a chief product officer that's really just executing the vision of the CEO. You can have the CEO that just realizes there's other things that they need to spend most of their time on, and there's every sort of flavor in between. What sort of reflections do you have specifically on the dynamic of a chief product officer and a product oriented CEO working together and making that successful?

Jeremy: I think spending a lot of time to mind meld on the fundamentals of just what is their vision for what makes a great product and trying to understand decision making so that you can not end up always disagreeing or having conflict in things where you're going left and they're going right all the time. That's not going to make anybody happy in the long term. So I think I spent a bunch of time, at least when I think of early with Christina, I was like, great, let me understand the product. What do you think made it successful? What do you think will make it successful in the future? What are some of the guardrails? Where do you want to spend your time on product decisions? And which things excite you and which things bore you? Which things are you good at? What are you not? And then I kind of can mold what I want to go spend my time on and we can see how that's complimentary or not. I think with Christina, one of the things that's helpful is there are things she's super passionate about. And I don't know, I think just being accepting of product founders, I don't know if most CPOs are this way, but I'm like, come all into my shit and jump in with me. I am very much a ... We did this partnership recently with a company and Christina was super excited about it. And she wrote the initial PRD and I think did a little v0 of what she thought it would be like. And I was like, "Do you just want to run with this?" And she's like, "Yeah." And I'm like, "Great." And I was like, "I've just told my team. I was like, just work with Christina." You know what I mean? And I don't know, I don't feel disempowered or anything by that. She wants to go do it. She needs some happiness in her life thinking about ... This is something that's sparking joy for her and she has to run a big company. And there's exciting parts of being a CEO. I'm sure there's plenty of parts that aren't exciting about it. You know what I mean? And this is a spark of joy. I'm like, "Just go ahead and do that." And so I try to create no structure on my team. And this is something I learned at GitHub. I felt like working with Nat, who was a product founder, I felt like was pretty similar as well, where if someone on my team knew it better than me or anything, I'm like, "Nat, just go work with Chris on this thing or go work with Naha on this thing. Don't ask me and I don't even need to be involved." And so I think when you get in this command and control military style structure, everything has to go up and down and everything, this is what kills these big companies. And so to me, I want everyone to feel like they can work with the CEO on anything and feel comfortable working with them and that it's not a problem. And that I just know with Christine, I'm like, "Great, you're on this one, so I'm not going to go and I'm going to go spend my time on these couple other things." And so I think just being clear about what those boundaries are is to me the most important part of it.

Brett: In the way that you've seen it when a CEO wants to get involved in some new product or some product detail, do you have the same standard that you expect them to operate in owning that? Or if the CEO is going to own this and it's not the way that a diligent, engaged PM that would own this, you give them slack. Or they're expected ... If you want this, it's all you, but you better execute this well.

Jeremy: Yeah. I never give them complete ownership if in that regard, I would say I always assign a PM with it. And it's a PM that's excited because it's cool to work with the CEO and the founder on things. So usually it's not hard. They don't feel disempowered. They're like, "Oh yeah, this will be fun. I get to work with Christine on it." And then it lets her still stay at a more conceptual level. So if something big does come up like, "Oh my gosh, we're about to go do a fundraise. That's going to take a bunch of her time and she can't do the day-to-day work with IC engineers and things." So I always try to have a PM and a designer. Usually someone also that's more senior, ideally staff level for us, or if not a higher senior person so that they can just understand what a high level exec communicating something means practically and how to translate that down so Christina isn't pulled into detailed minutia decisions.

Brett: Do you think that way of operating, which is that a CEO can operate at any level and any function is just globally correct or just works in the context of some of the companies that you've worked at?

Jeremy: I mean, I'm sure it hits scale limits. I doubt Sundar or Satya can operate that way, you know what I mean? With whatever 200,000 people or whatever they have. I think those companies just become very different when they get to that scale. I think when you're still in the thousands though, you should still be doing it. And I think that's part of our philosophy. At Vanta, I think we really expect people to be in the details and that's something that I believe a lot. It's just hard for me to think about how I can coach an engineer, designer or PM to be better if I don't know what they're actually doing and how to go do that job, and I'm not in those details to some degree. I mean, I tell our designers when they come in and get hired, I'm like, "I'm going to be leaving comments in your Figma file." That should not scare you. It does not mean you're doing a bad thing, but just try to normalize the behavior of like, "Hey, we can have it." I think the thing that I've learned here, and I learned from one of our design directors, Brayden, that was really good, he came in with this mechanism that was pretty good where he actually just puts what level of feedback is in front of everything. Like this is an idea, this is a suggestion, or I expect you to take action on this. I think that's something I've learned from my team that the more I've kind of grown in the size of my team or seniority of my roles, that executive megaphone, you just don't understand how it's going off and impacting all this other stuff in your org in negative ways. So I always try to be a lot better about that and repeat things multiple times. Like I send something to a team, I'm like, "I'm expecting action on this quickly or I just want to have a brainstorming conversation. Please do not change the roadmap." And I'll literally put it in all caps. Even sometimes when I do that, they'll still be like, "Okay, we made three adjustments." And I'm like, "No." So I think that's the burden that kind of comes as your team gets larger is really pushing on them to set expectations of when you want something done versus when you don't, if you're going to allow that level of movement. You know what I mean? Where the CEO's going to come in and talk to the team about something. Or even me, I don't know how big the team is now. It's like 350 people or something like that. So the EPD organization I have is pretty large at this point. And so it can be kind of scary for someone that just started for come in and me leave a comment on their PRD and they're not sure what that means. And so we try to normalize that as much as we can.

Brett: You touched on this a little bit, but an engineer, when you think about the thing that they're producing, it's software and then ideally a valuable product that a customer then wants in exchange for money. A designer is maybe doing prototypes or mocks or finished designs and the output is design and maybe what they're responsible for depending on the organization is some version of product quality, impact, whatever. How do you define what the output of you in the chief product officer role is?

Jeremy: I mean, I think for me at the highest level, I guess there's, do customers love the product and how do you want to measure that? Some of that I think can be measured through looking at win rates and how we're doing and how many of those are like product efficiencies or not coming up. I think customer satisfaction on specific areas is really impactful to me. I'm not a big believer in NPS. I know some people like it. I don't think it's like that useful, but I think going and doing customer satisfaction surveys for core workflows of like, "Hey, did this make your life better? Did it not? What were the gaps? How was it?" Those are other important areas to be looking at there. I think the velocity of like, are we continuing just to ship new features and do we have not just a product that excites customers, but like a vision that excites customers and feels like differentiated? So I think those are some of the big outputs we look at there. I mean, I think there's per product revenue is something I look at a lot. I think the sales take, at least the way we do it and the way I've seen it in most companies is more like you have like new logo versus expansion, you have it broken out by different segments and territories and all those different pieces. I instead find that the dashboard I go to and have the finance team build for me is all based on product clients. So I'm looking, it's like, great. Okay, we did a bunch of investments in this product. Are the ACVs going up as an output metric? There's lots of input metrics of like are people using it and whatever else those output metric. It's like, is it going up? Are we selling more of it? What does the attach rate look like there? Is it selling to the right persona and segment? So in our marketing data, when we do outbounds and when customers come in, we classify them into our different personas because security teams are pretty different. A startup, you probably have no one that's actually their job is security. It's like their moonlighting as something else and it's just like the founder or the technical founder. And then you have big companies where they have CISOs and multiple layers of like hierarchy and high specialization where there's a whole team that just does internal risk that is different than the team that would be handling audits or internal audits and everything. So there's a lot of specialization. So we classify those and then we think about when we're building products like, "Hey, I'm building a product for Susie security." And for her, our persona is basically the first security hiring in a company. And it's like, how are we doing against her? Is she buying the product we want? Is she using it the way we want? And how is that growing? And then is the breadth of our product offering growing? I think Christina has an approach that probably a lot of other founders do where her first experience was at Dropbox and she was working on paper and it was like a product that they were trying to expand out to be more multi-product than they are today. And I think Dropbox has done that successfully in some ways, not in others. And I think she has brought that very fully into Vanta of we have to be multi-product and how do we get there and how do we do it really early and keep it in our DNA and always be able to build new products? So I think that lens that kind of gives me a revenue output metric that's really helpful to know, great, we launched our customer trust product two years ago. How's it doing? Okay, we added a new add-on for it. Is that selling well? Are customers adopting it? How are we doing it? And then tying back to the specialty sales team like, oh, if it's not selling well, is it the product's bad? It's like we can make it better or is it like, oh, we didn't have specialty sellers. And that was like one of the fixes we did this year where we're like, wow, when we talked to the general sellers, they just don't know how to go into these deep conversations for the new product. They're kind of scared about selling it because this thing's already making all their quota. Why try risk it on the new thing? And then we did that and then it unlocked even more growth. We still found things that the product could be better on, but I think those combinations of looking at the per product revenue, how people are using the product, are they happy? And then just do we have like a compelling, exciting vision for like upmarket customers? A lot of that can fall in product, it can fall in the CEO or it can be like somewhere between for that one.

Brett: Outside of when you're doing IC sort of PM work on a given product at any given point in time, what do you think are the actual pieces of work product that you yourself must produce that are important?

Jeremy: I think for me, I have like a couple docs that I end up owning and work with my team on. One of them is this kind of meta doc now that like Google Docs have tabs, it's kind of all in like one doc with all different tabs that we call it the product plan doc. And so it basically goes through and looks at every core product that we sell independently and it defines out like, here is the persona, here's the target segment, here's the core customer problem, the value we're trying to deliver, what our differentiators are, when those are shipping and how we're unlocking new types of customers. So whether it be like a bigger headcount tier, a new special type of customer that we're unlocking. So I try to think of this as like the raw material that EPD and the product team needs to be aligned on of like, who are we building and how are we going to go win? And then there's a time element that then informs marketing and sales, right? Where we're like, "Oh, great. We think that customers that are shaped like this, we can't win them today, but we think we're going to go win them in Q2 of next year." And then I can go through and tell the sales team and marketing and be like, "We should have a campaign about this. We should be pushing it here. We should do a moment at this thing." And then we can see maybe it works out, maybe it doesn't, you know what I mean, on that bet. And then they, when they think about revenue forecasting can then go and be like, "Okay, on Q2 or in Q3 here in this segment, I'm expecting things to improve and this is how they're going to improve and we work with finance on a model." That document's really important to me. I think the things I've learned about that document is it is really great raw material for PMM and some of the people in the field, but a lot of times it gets way too detailed and it kind of glosses over for them. I would say we're shipping so fast right now where it's like each quarter we're shipping over a hundred feature updates and feedback I'll get from the AEs is like, "Oh my gosh, we are shipping so much. I literally can't keep up. I'm not sure." And so they kind of lose the signal for all the noise. And so I think this document is the raw material that then helps enablement and PMM and everything. Us pull it in to see like, our downmarket deck should look like this or upmarket deck should look like this. Here's where we should expect inflections and win rate or when we're going to enter a new market and how well we think we're going to go do. So that's a big doc for me that I spend a lot of time on. I spend a bunch of time with the finance team on that product dashboard, just analyzing that every month around where is product revenue coming from, where's it not? Are we surprised? Is it different personas than we thought, different headcounts than we thought? I think the other thing is a couple, we have what we call the product pipeline as well. This is like in Figma where I go through and think about over the next three to four years, when are we introducing new products and where and to which customers? And so I have like little emoji when it's a new ICP or we think this is a product that can be like a hundred million dollar plus business versus one that we feel like is more of an add-on. And so that helps me get the strategy view with the C-team around like, great, this year we think we're going to start incubating two products in this segment. We think next year they can generate X revenue and then after that and we can start to plan out our hiring plan. Do we want to do M&A? Do we want to build, buy, partner, and start to have those different discussions? I think the other thing is just the high level kind of like vision click through for some of our core exciting scenarios. So this is like, okay, what are these three or four really good things that we can just show to a customer about the experience of the product where it's like, "Okay, great. Here's a cool scenario where the Vanta agent is going to do this amazing thing that's unique to us because we have all this data and it can go do it. Let's have a full click through of what that is and maintaining a couple of those big walkthroughs." That's another big thing for me. And so those are some of the ones that I would say I'm the like IC on. And then there's a lot of frequent meetings I do. So I do these things called deep dives. So once a week we have a big block of time where any of the product teams can kind of like come in and walk through different areas. And sometimes I'll ask them, I'll be like, "Great, I want to update on our auditor experience. Can someone walk me through the whole vision of what we're doing and why? And let's go through those." And so that's another good checkpoint mechanism for me where sometimes I'm asking for things. Sometimes the team comes and says, "Hey, we want to show you this. We've got this new idea about how we can think about third party risk in this new exciting way. Let's bring it in and let's walk through." In those, I try to keep very visual oriented. So it's very much like, show me the customer experience, how fast can we get to visuals or a prototype?

Brett: Maybe building on this, if somebody were to watch you work for a given week and all the meetings and IC work and when you're writing a document out, when you're joining an exec team, what would they notice? What are the big chunks throughout the week where work's happening?

Jeremy: Because I'm based on the East Coast, I get a lot of IC time in the mornings and I really try to protect that. So I would say I try to start off my day with like two to three hours of either I'm catching up on docs or Slack or I'm writing something or I'm designing something or leaving feedback or I'm just walking the store. I like to spend a lot of time just going into the product and just using it. And I kind of have a set of scenarios that I run through and I go through and I'm like, "Okay, today I'm going to focus on our risk management product or I'm going to focus on our audit product or customer trust." And I just go through and find and log bugs and think about like, "Is this feeling right? Are we headed in the right direction? Are we doing that?" I think that might kind of like surprise people. My calendar gets really time sliced. I'd say that's another thing that's hard about this role and something I'm always trying to improve, but I usually have a few customer meetings every single week, whether it's like a roadmap conversation, an escalation conversation, or someone else just wants to go meet and talk about something. So there's always like a chunk of time there. And then I do believe in one-on-ones, so I do have a decent amount of one-on-ones.

Brett: For the last 25 years, if you didn't have a one-on-one, you're a horrendous manager or a leader. And then Jensen has no one-on-ones now, no one has any one-on-ones. And the lesson is no one has any real original thinking as to like, what are we actually trying to do with any of this work? What's sort of your theory as to why that's effective for you?

Jeremy: Yeah. I mean, I just think people build products and you need a relationship with people and just having that time, even if it's like a tight 30 minutes really matters. I try to make them working one-on-ones. So usually, sometimes I just always try to want all status to be async. You know what I mean? Just DM me, let's get in a doc, let's do whatever so we can get into the meeting and solve a problem together or have this discussion about like, "Oh, should we do this or that and really wrestle with the problem?" I find them just to be really important. I would say I do one-on-ones with my directs every week and then I have monthlies with a lot of other people in the org. I also try to look for the kind of influencers in my org and I probably spend more time with them and that can be better, that can be worse. I've seen it turn out to be negative at some companies where it can feel like an in crowd or outcrowd. I think we do a really good job controlling that. And I try to be really thoughtful about it, but there are people that kind of have an outsized impact on the organization and I try to be thoughtful around like, "Okay, great. Where is their head at? What are they thinking about? What time should I go spend with them?" Our staff engineers I spend more time with than most of the other engineers on the team, just because I really think they have a massive opportunity to have this huge leadership technically across the company and beyond that. And I want to make sure we're really aligned on what we're all trying to achieve from the product perspective and how that ties into engineering. I do find one-on-ones that are helpful. I still have office hours too, where it's like anybody can just go in and book my office hours. I think it's like an hour and a half every week. And sometimes people don't book things. A lot of the times they do and we come in and just kind of talk about anything.

Brett: And it's open to the entire company or you're-

Jeremy: Yeah, it's open to anybody in the company. It's pretty much always people in my org. There are times when no one books it and I'm just like, "Just get the time back." I find it just to be helpful, especially when you want to have this culture where there's not as much hierarchy. I think just demystifying who I am and what I do. And I try to be vulnerable with my team. I try to tell them areas where like, "Hey, I'm responsible for this and I think I messed this up." You know what I mean? And here's a mistake that we make that I'm responsible for and what I learned from it. So I think having a culture like that just makes it easier for everyone to work with people at different levels when they're just like, "Oh, my leader's doing this."

Brett: How long is each one of the slots that somebody can sign up for?

Jeremy: Oh, on those, I think it's 20 minutes.

Brett: And you get all sorts of range of things that people want to talk about?

Jeremy: Yeah, yeah. I mean, I've had brand new ICs that started the company that week and are like, "Hey, I'm so excited to be here. I just wanted to meet and talk." And I'm like, "Great, let's go have that conversation." Versus someone's like, "I have this super opinionated design decision. I would love to know if you think this makes sense or not." And we're going into the pixels, you know what I mean? Very detail oriented or it could be like I had one a week or two ago where our engineer that's leading how are AI enabling our code base just had a bunch of things you wanted to go talk to me about around like, "Hey, so we've met with these different companies that are doing the same thing. Here's what I've learned. Do you have any strong opinions on these? I'm drafting out these comms with the entire team. Does this seem right? Are you aligned with it?" And I'm like, "Yeah, this sounds great." So that one was more of a working meeting. I mean, I'm sure you can go do this. I don't think that ... I just can't imagine giving some of the feedback and conversations I want to have to people in a group setting and that being the best way to get the outcome I want.

Brett: Why do you think that is and have you tried it?

Jeremy: I have done it a couple times. I think sometimes I've done it good and just times I've done it not good, you know what I mean? Where I've maybe been too harsh in a group setting. I think there's a set of people that I've worked with who they kind of get these ... There's these triggering moments when it feels like I'm being criticized in front of people where that produces them into a really negative, frustrated with their job, unhappy with you, feeling like you're attacking them. And then there's other people that are just like, "Oh, that's fine." I would say I'm very much in that latter camp where, I don't know, Christina, I tell her, it's like I was like, "You can disagree with me about anything in any meeting." You know what I mean? It doesn't bother me. Obviously, I'm going to feel bad for disagreeing all the time and then I'll be like, "Okay, I have a performance problem that I need to go improve." But I'm very comfortable with that and I've just found there's a set of people that aren't. And if I want to get the best out of them and I don't necessarily have to force them to work the way that I work. So there are people that I have longer one-on-ones with, specifically because I know it helps them do better results and I think it's worth it. I'm like, whatever, another 30 minutes a week for one of my directs. That is completely worth it if I know it's going to help them do their job a ton better. I can adapt to people and not have them all be forced to adapt to me.

Brett: Yeah. I mean, I think that companies and teams are these complex organisms. So my guess is if Jensen has been running the company for decades like this and everyone's indoctrinated or there's positive selection bias where that's fun and enjoyable for them, then it can work incredibly well. I think the dangerous thing is to assume that in that organism it's successful that you can just yank it out of there and jam it into your company and it's going to be successful.

Jeremy: Yeah, I agree. I think there's like a whole ... Yeah, when you already have the system working that way, there's a bunch of people that are opting into that system and then versus you're like, "Oh, I decide I'm going to go do this." And there's a bunch of people like, "I did not sign up for this. This feels bad." And maybe that's the right decision and you want to make it, but yeah, it's a lot different.

Brett: When you think about influence in an organization, you have what the org chart says in most companies.

Jeremy: Yeah.

Brett: You have some sort of ...

Jeremy: There's definitely the shadow org chart as well.

Brett: But on that point, you have sort of a director, VP, et cetera, et cetera, and they have influence and authority. You were mentioning this a little bit, but in any company, there's all sorts of people that aren't a director of this or VP of that who end up having an incredible outsized influence on the company. Who are those people generally? How do they end up in those positions of sort of informal authority or sort of these people that people end up going to even if they're not a chief this or a VP of that?

Jeremy: I think this is the opportunity for ICs. I think a lot of companies don't celebrate ICs enough. And for us, I want ICs to be able to get all the way to the VP level at Vanta and have that level of impact. And so I think a lot of people that end up in these roles are often like ICs. You can read the books about Apple's past with Jobs and different people that have done amazing work there and how they were able to go and influence. I think one of the things about them is they're usually extremely good at what they do. They can communicate it well to executives. And I think that is a skill. You know what I mean? They know how to, in our case, go or talk to Christina or in the GitHub case, talk to Nat in a way that Nat will understand or Christina will understand what they're saying-

Brett: Do you think those are two separate things-

Jeremy: ... and see the value of it.

Brett: ... between the two of them?

Jeremy: Oh, yeah.

Brett: Or there's a correct way to communicate to a C-level person or a CEO?

Jeremy: I think it depends on the CEO, but I do think that being great at your job and being able to talk about it to the leadership team are two different skills. The other thing that I think they do is they're usually more on the, they're thinking about where we should be going and more visionary where they're like, "What is happening in the industry are pretty tapped into that and bringing that knowledge back and trying to affect change." They will go through and be like, "Hey, this awesome thing happened in AI when computer use came out for the first time and they're like, "Wow, this could be huge." I went and spent the last week prototyping all these things. Here's what I built, here's what I learned, we should be doing more here, here's why. And they kind of set these things up on a platter and then they'll go and get a bunch of other people excited and you can usually go to them. And I think there's about a bit of social influencing here too, where it's someone that I know I can go to and be like, "Hey, I really care about this." And if they care about it too, I can be like, "Can you help us drive this message across the team?" And they're just kind of socially connected enough on the team. They'll bring it up in meetings, they'll bring it up in all hands or they'll do different things and kind of carry that message forward. So I think some of the things, and I don't think it's like someone that's like a perfect social butterfly. I think there's like your stereotypical engineers that are less that way that are still really good at this, but I think that's part of it. I think the negative, the bad part that can come of this is there kind of becomes like an in crowd and like an out crowd, but I think you can control that pretty well if you're thoughtful and don't let the system getting abused. I think it is making it so it doesn't feel like they are invited to all these certain meetings and kind of get pulled in to discussions that like no one else is. I try to make it more informal. If you're in an office setting, it's just like, hey, we have some engineers that will just always come to me if I'm in the office. I'm remote, but we'll come into Christina and talk to them and go do those things. And I think that keeping it there, but we're not also going to be inviting them to every C-team meeting, you know what I mean? And be like, "Oh, what is your thought here? And can you write this paper? And can you do all these things and we're not going to tell your manager that they're doing it?" And so I think that's when it starts to become negative where it feels like this person now has been kind of like elevated into a status that no one else has been versus someone that's just doing really good work and trying to make the company better. And whether or not they're a manager or not, I don't think matters.

Brett: When you think about being effective in your role now at Vanta as the chief product officer, and you go back to the last 20 or 25 years of product building and engineering, what are some of the stories that come to mind that were really formative, that have helped shape who you are in the role?

Jeremy: I remember I've definitely had some of those harsh feedback conversations, you know what I mean, that I thought were helpful where I shipped something, I can remember one where my team shipped a product, I knew it was kind of like a B class thing, but just wanted to get it out the door because I was kind of tired of trying to iterate with them on it. And I remember my ass kind of getting handed to me for letting that happen afterward and not forcing the bar to be higher. And I think that was a really helpful thing for me. Well, two, one, it never feels good in the moment. Two, the thing my manager did at the time that was really good was like he warned me ahead of that meeting where he's like, "Go into this, it's not going to be fun." You're going to get some really direct feedback from like ... You know what I mean, from the CEO on what this is and why it didn't meet it, but that at least got me in the right mental mindset. I think if I would've walked into that meeting and not know that was going to happen, I would've reacted less good. And so I think that really helped me understand. I think I had some early managers that really focused on the craft of design and helped me understand what does it mean to create a product experience and why it's helpful to have a product experience that people love and isn't just functional. I think you can have successful businesses without it. It's just not as exciting to me, but helped me build those skills. When I thought about my first years at Microsoft or even before I went there and had like ... I mean, I taught myself how to code in high school and build websites for people in the late '90s as my high school job. So I thought about the fact that I was for so long did engineering, product and design, I think it helped me really appreciate all three. And I think that was really formative to what I do today because I remember coming up with an idea, writing all the code, doing the design, realizing it, showing it to a customer and going all the way back through. And I think it gave me an appreciation of everything that goes into making a product end to end so that I was able to make better decisions. There's definitely product managers who I think they don't know enough about the technical side, which can be fine, but can't really appreciate the pain engineering is going through and what on call is like because they've never really done it and why we need to make this more sustainable and how hard it is to keep shipping features on a broken architecture. And so I think the fact that I've had a lot of those experiences give me a little bit more empathy and I think a little bit more insight into when it is time to be like, "Hey, we need to slow down in this area and fix the architecture, go do something here because it's going to make kind of the go slow to go fast later thing versus just having to rely on engineering." Because sometimes you can get bullshit as a PM, you know what I mean? And I have the high detector for those, so I know enough about those pieces to kind of go in. I think those were some of the big ones. I mean, I think at GitHub, the big thing that for me was really understanding what the bar was for high quality experiences. I felt like GitHub's always done that really well in the past with pull requests and all the different things, and I think it's done a good job with them. And I think Max, who led the design team when I was there, I think really opened my eyes to like, what is that bar? How do you hold that bar and how technical a design team could be as well. We really pushed hard to have designers that wanted to go in and own the front end and to go build these experiences and would really understand all the details of how to make something come to life. And I think in my experience at other companies, I hadn't seen that even be an option. I just didn't even know what was possible. And now I'm like, oh wow. Now with AI, it's hopefully going to get ... It's just getting more and more easier for people to do that without all that technical expertise. But I think those were a couple of the big moments for me.

Brett: Given you've operated at sort of various scales now and you're sort of at a scale up company, one of the things that seems to always happen is as you grow, you get slower. And as you were describing, one of the things you think a lot about in the role of chief product officer who also owns engineering is how do you sort of accelerate cadence? What have you figured out in this role about how you move quicker and not just get slower and slower in terms of every next 10 people that you add to the company?

Jeremy: Yeah, it is very hard.

Brett: Why is it so hard?

Jeremy: I think that part of the reason it's so hard for us is you have so many new people. So a couple of things that I think about is we've been hiring a lot. I think within the engineering product and design team, we've hired 50 people a quarter for over a year and we'll continue that all the way through the next 12 months. And so the team has just grown substantially. And when we look at Vanta overall, I think if you've been here three months, your 10 years already [inaudible 01:01:11] than 20% of the company or something like that. And so you have so many new people. So one of the big things that I've tried to go do, and I learned this lesson negatively, I felt in my last year, or maybe it was last year and a half when I was at GitHub, was we hired way too fast. And then we hired so many people at just the point at which they were all ramped up, we laid off a bunch of people. And I was like, this was beyond bad. One, I just feel bad human-wise ever laying off people. Sometimes you need to, but then you're like, we spent all this time hiring them. It's like multiplicatively worse, you know what I mean? And like ramping them up only for it all to be gone. So the thing I really focus on, and my team probably hears from me all the time, I use this term absorption capacity. So I'm like, how many people can you absorb and still deliver at the rate that you're delivering at now? And so I try to look at-

Brett: Is every team different?

Jeremy: Yeah. I mean, I think there is some. I think for ... I would say it depends on the quality of the code base. There are some parts of our product that we've built more recently and they are built well and more forward-looking and those teams can absorb people super fast. We have old curmudgeonly, nasty parts of the code base that we need to have more senior people and we need to re-architect and fix some of them. And so absorption becomes really low. So those are the areas ... That's the big thing that I focus on. I think the other thing we talk a lot about ... Luckily Christina had a bunch of this culture on lockdown before I got here. She was very much like deliver, deliver, deliver, deliver, to the point at which my team was almost scared when I came in talking about quality of like, oh my gosh, is Christina signed off on the shit you're saying? You know what I mean?

Brett: Because most people think quality and speed are-

Jeremy: They can't go together.

Brett: ... fundamental.

Jeremy: Right, right. Yeah. And I was like, "Yes, I've talked with her. We can do both. Here's how we need to think about doing both and how we're going to go do it." So there's metrics that I have is like, how many feature announcements do we do per ramped engineer? And then we have a certain amount of time we think to use ramped engineers. We use this startup called Span that helps us with developer analytics. And we also look at the ramp time for engineers. I'm spending a lot of time talking about engineering because to me it's the vast majority of the team. So it's like the most important thing to optimize. So I look at like, can they ... Like our expectations, you're going to write a PR your first week and ship something to production. And then the ramp ups are actually different for product and design. They're actually almost the opposite ... Or sorry, for product design and engineering. If you think about product and design, I want them to understand the strategy and where we're going and the overall context of the product because they make big, impactful, broad decisions. For engineers, especially more junior ones that start, I just want you sling and code as fast as possible. How do we get you ramped up so that you can start executing and building as fast as possible? And then over time, you're going to build more context. And it doesn't mean engineers shouldn't know the business. They should and they should be like experts in it and be able to give feedback on the product. But what I'm trying to optimize then is our throughput capacity for the engineering team versus the product team. I'm trying to make sure that they don't make a decision that's going to confuse 30 engineers because every decision they make has a massive blast radius. So I think that's one of the things, absorption capacity, understanding your ramped engineers, what is the productivity per ramped engineer, how you go do that, and understanding where the code base is making you slow. No developer wants to have to go get a cup of coffee when they're running the build or when they're running tests. So how do you go optimize like build, test, deploy, drive those different pieces, looking for hotspots in the code base. So we also look at percentage of delivery of every single squad. So a squad for me is an engineering manager with 8 to 10 reports, one product manager, sometimes one ... and a product designer. Not every team's exactly like that, but roughly. And I look at what percentage of their time is spent on keep the lights on activity, what amount of it is spent on deep architectural rewrite tech debt things and what is new customer facing functionality. And even if it's a platform team, you can still tie it back to the customer like, "Hey, we made the experience faster," something like that. I look at that across all the squads every single month with our head of engineering and we go through and then just ask probing questions on like, "Hey, how's this going?" We ask people to write dates and commit to them. We expect us to be able to hitting 90% of the dates, which I think is reasonable. If it drops down to 85%, I'm not worried. If it drops down to 80% or 75%, I start to get worried. I want people to take risk too, and I don't want to build a culture that sandbags everything so that they can say they hit all the dates.

Brett: Last couple of things I wanted to talk about. When you think about the trajectory from being an ICPM or engineer all the way through running product design, engineering, what learnable skills were the hardest? Now there's a lot of innate things obviously in you that you are well-built for this role, but what are the things you actually had to learn from a knowledge or skillset perspective that you found were both important and hard?

Jeremy: It's hard. Some people just aren't natural at it. I think I had some natural ability, but then a lot of it I had to just learn and practice.

Brett: And it's just reps?

Jeremy: Yeah, you just put in the reps. I remember when we announced GitHub Actions, I did the big demo and I remember being super stressed before that demo. I must have run through it at least 40 times going through the demo, saying it all, was super nervous. I think it came off really well, but you know what I mean? I had to put in the reps and it's like, we've got VantaCon this week and I've done this now enough where it's like, okay, I'm just going to need a rehearsal or two and I'll be fine. And I've read the script already. But I think those were things that you put in the reps. Those are the more performance ones. But then there's just the, "Hey, you need to be able to get the team jazzed and focused at the all hands." However often you do those, if you do them weekly or monthly or just be able to turn it on with candidates and get into that sell mode. I think that generally certain people just maybe don't have that and it's something that you've got to go and learn. And it was something that I had to go learn definitely parts of that.

Brett: So just want to wrap up. When you think about being effective in the role of chief product officer, who throughout your career has been the person that you've learned the most from? And is there a tangible thing that is expressed in the way that you sort of execute in the job?

Jeremy: I think it would probably be Nat at GitHub. And I think the thing that I learned the most from him was just what does it take to build and deliver a product quickly that people love? And I think I didn't really understand how I could do both of those. And I remember the first time when I signed up for the big mission that we had, when we came over, Microsoft bought GitHub, I'd been part of working on that acquisition. So I kind of assumed that I'd have a role when we were done with and talk to Nat and he's like, "Hey, we've got GitHub Actions." And he was like, "You have nine months to get this to GA." And there was no question of like, could I ask for more time or not? But it's like basically build it and get it to GA in nine months. How do we go and do it? And I think that working with him through that pushed me to the point where I was like, wow, I can do good work faster than I ever thought that I could and how helpful that deadlines were and him being really involved in the beginning of giving coaching of like, "Okay, here's how you need to think about this or do this or that." And then just seeing him fade away more over time gave me more confidence. But yeah, I think that project was like a career defining ... I mean, there's a bunch earlier in my career, but I would say more recently, that was like seven years ago or something, was there where it was like a big business outcome we had to drive and it was really hard and wasn't sure I could go do it. But having someone that believed in you and pushed you and was like, "Here's how you can go do it," was really helpful to me.

Brett: Is seeing him engage with you in that way, the learning there is how to play the role that Nat played for you and then you try to play that role for people on the team.

Jeremy: Exactly. Yeah. And I think that's the thing that I try to take forward is one of the things I want for anybody that comes to Vanta or works with me anywhere is to feel like that they, whenever they leave or move on, they're like, "I did best work, my best work there and I did better than I thought I could." And I think that was a thing that Nat was able to extract out of me out of that process where it's like, I did better work than I thought I could on a faster schedule than I thought I could and had more impact than I though I could. And I'm like, "How do I become that for everyone on my team?" Where I'm just like, "Okay, great. How do I inspire you, give you the clarity, give you the freedom to create your own clarity and push on dates and ask hard questions, but still make it reasonable enough where it's achievable?" I think that's what I try to bring to my teams is how to play that role for them so that they walk away and it's like, "Yeah, there's going to be hard days." You know what I mean? It's like there's going to be not fun days, there's going to be late days and everything, but I want people to walk away and just be like, "Damn, I did stuff that I never thought I could go do when I was at Vanta." And that's what I want them to walk away remembering.

Brett: Great place to end. Thanks for joining.

Jeremy: Yeah, thanks for having me. It was great.

Brett: Great.