Building a highly-technical enterprise product? Essential advice for product leaders — Nate Stewart of Cockroach Labs

 Brett. Thanks so much for having me really excited to be here. 

 So I wanted to start by talking about maybe some of the intricacies or differences. When you think about building products in an open source or developer infrastructure context versus maybe more traditional SAS. And you've had experiences in a variety of different shapes of products. And obviously most recently have spent a tremendous amount of time building and scaling enterprise infrastructure with an open source component.

And so I just wondered to start kind of a little bit more open-ended and maybe have you share some of the things you've figured out as you've been building and scaling products in this enterprise infrastructure space.

 The good news is that a lot of the principles for building a product organization are the same across both of those. So you're still putting in place the people processes, culture to take market inputs and ultimately to create a roadmap that should create value over the long-term. But when you start drilling into, you know, what does it mean to build a capability in a infrastructure company?

 The biggest difference that I've seen is that you're not just designing for human users. You're also taking into account, uh, machine users as well. Right? 

So as a specific example, Uh, early, when I was at cockroach labs, this was like 2018. There was a big trend around unstructured data, And we were adding support for unstructured data. so people could store data within cockroach and use that to power some, you know, machine learning experiences. And what we saw was that we understood the requirements of the feature, how to write this data, how to read this data, how to edit the data. But when we actually released the V1, we didn't see the reception that we expected.

And as we started to drill in, we, we saw something pretty interesting where in a traditional B2B SAS company, you know, your, your features would be used, you know, once a day and twice a day, as part of some core workflow in a, an infrastructure company, your feature may be used a hundred thousand times a second.

And so you really need to start thinking about the performance profile of the capabilities that you're building, because that can be the difference between having product market fit or not. You have to think about how your features interact with the underlying cloud infrastructure, because if they are not using resources efficiently, again, at a hundred thousand uses per second, things can get cost prohibitive.

And then that again can be the difference between having product market fit and not so starting to have a broader view of how the features exist. Both for humans and machines is one of the big differences in working at the infrastructure layer.

 so on that point about thinking about the performance profile, or how does it interact with the underlying cloud infrastructure? How does that actually express itself when it gets into the nitty-gritty of actually building a product or product strategy, how does that reality show up?

 part of the challenge at that building a database from scratch is. Way that your customers, uh, behavior changes over time. So early on at cockroach, we were just trying to get the first users, the first customers, and they were, they were building small applications, not really, um, using cockroach for what it was designed for.

 You know, do you have these scalable, uh, data intensive applications? And then what we started seeing is that okay? Yes. You know, we were able to understand the initial needs to kick the tires, but once people started getting into production, once they started scaling and going from supporting a hundred users to a thousand users, to a million users, we started seeing where the performance issues could, you know, really cause a high operation.

Uh, burden for the user. And so we started thinking about, all right, what does it mean to test the edges of our product or test the performance of our product in different scenarios and how do we actually use that to differentiate versus our competitors? So, as a specific example, one of the biggest decisions that we made early on at cockroach labs was very simple.

You know, what types of queries do we want to make fast? Um, when I joined, we had an option. We could build a database that was more like a analytic system like snowflake. It was very good at running large aggregations, doing math and computations, across many, many roses. Or we could build a database that was, um, very performant when it came to writing and reading very small amounts of data.

Um, but in a way that's consistent in everything ties out. And so that was one of the big decision points. You know, what types of use cases do we want to be a performance for? Um, ultimately the, the big decision that I made was to focus on those transactional use cases because the, the thing that made a cockroach DB unique and makes cockroach DB unique is the resilience.

So if your analytics database goes down for two seconds, that's okay. You press refresh, you run the analytics again. If the database that is powering your payments processing goes down for two seconds, you can use millions of dollars. So, um, that starts to show how you can start to build a strategy around the core capabilities.

And that's how it surfaced for thinking about performance and cost and things like that.

 And so w with that sort of anecdote, what was the process that you went through to land on that opinionated stance? Was it organic and like highly judgment based, or it was kind of rigorous.

 So I joined as the first head of product. And one of the things that I had to do was put in place the processes for making decisions around what to build, what are the types of capabilities that we should say yes to and what, what are the things that we should avoid? And so when you have a database, I mean, databases work in every industry, they work in many different use cases.

 You need some way of saying, no, you need some way of filtering. Uh, some of these are requests. And so I was looking at what is it. What is the first part of a decision tree that we could use to filter out things that aren't going to be in our strategic interests over the longterm. And so this is something that I backed into.

I looked at different ways of segmenting the database market, but the one that became the most clear in the most natural decision point was just, what are the types of queries that people want to run? Are they analytical? Are they transactional? And just putting that line in the sand and getting alignment with the founders and the revenue team that we are going to be a transactional database that helped us not only say no to a lot of things that would push us more into say a competitors to snowflake, to start to say yes to things that could build on our strength over a very long period of time.

And we're starting to see the results of that with our recent customer growth. 

 W did you find that maybe you as a leadership team or you along with the founding team, did you find that decision difficult to make, or did it kind of become quite clear and it wasn't particularly controversial?

You know, There's a lot written about startup strategy and execution and how you use OKRs to go from strategy to, activity. But there's not a lot of literature around why do strategies fail in the, the real world? And there were some real world pressures that were making it difficult to hold the line here.

Um, the first is that it was very early days when we made this decision. This was pre revenue. And so as soon. as We said, Hey, we are going to be a transactional database. Of course, we have large customers that are saying, you know, we'll give you your first $100,000 $200,000. If only you build these analytical capabilities, you're talking about your first revenue.

That's very hard to say no to in, you know, at that stage of a startup. So there was enormous pressure there, you know, and as we got a little bit further along, there were some other pressures on the, the strategy. Um, for example, recognizing that there's different possible strategies that would be successful In a market as large as ours for our competitors that were starting in, a similar space.

You know, when we decided to focus on, mission critical transactional workloads with a focus on keeping data close to users, Other feasible strategies opened up for our competitors. And so you start to see customers that you say no to go into the arms of your competitors, and that can be demoralizing if you don't have conviction.

And if you're not able to show how the strategy that you've outlined is ultimately going to create success both in the short term, but over the long period as well. So those are some of the things that made it difficult to stick to that strategy of transactional workloads as opposed to something else.

 In in, on, on that point, other than, than what you've outlined in terms of some of the, some of the early process that you put in place or the process that you followed to land on the transactional use case. Was there anything else that you, you did to, to give you where the team conducted? Or to make it okay.

 Yeah, there was a, there was a big story telling element here. Um, you know, I started with some basic, you know, MBA style, uh, reports around just market size, right? Like what's happening in the analytics space versus what's happening in the, uh, the transactional space. And really, you know, w what I initially said was leaning on our differentiators.

We're not at, you know, we can build a great analytics and database, but we're not as differentiated. Our secret sauce is in keeping your data online, up and running and close to users. That's something that you don't really get if you're focusing just on analytics. So there was a, there was a storytelling element there, but also starting to pair that with some real design partners.

So these aren't people that were ready to pay us now, but I could use. Um, interest to show a possible future. So for example, we had a customer that was doing a significant amount of payment processing and all of that processing was happening in a single data center. So if for some reason that data center went down for any reason, whether there was a operator error or an earthquake or a power surge, their business would be completely offline.

They had to get outside of a single region. They had to build a resilient system because it was only a matter of time before that happened. And when you start to see how many opportunities there are like that, and when you start to see the conviction of these customers that are trying to move off of legacy databases, you start to get a picture for not just academically how big this market is.

You can feel it viscerally. And that was a big part of the storytelling to build conviction around the transactional. 

 Anything else that you did in terms of that, did those design partner relationships in terms of how you structured it or how that, um, fit into this, this process of really trying to own this specific transactional use?

 Yeah. The, you know, you asked at the beginning of the podcast, what are some of the differences in the infrastructure space? And what I learned Was that people who buy enterprise infrastructure tend to be more conservative, you know, by definition, the inf infrastructure is what supports some core business process.

So if your infrastructure is unstable, if your infrastructure goes down, your core business is unstable or your, your business can go down. And so we found that. It wasn't as easy as I would have expected to get people to give feedback on beta software, there was a huge engineering investment that would be required to kick the tires of a new feature that wasn't fully baked.

And it really wasn't worth it to our customers. They just wanted the product to work and they wanted it to work very well. Um, we also found that when our customers were rolling out or, uh, purchasing the software, they often didn't want to run the latest and greatest. They want it to be six months behind or a year behind.

So a lot of the feedback that we got from our customers was based off of decisions that we had made a year or two years ago. So it was really, um, you know, driving through the rear view mirror. In a world like that, I ended up splitting our design partnerships into three big categories. We had a strategic design partners where we really talked about where we thought the database market was going.

What were some of the macro problems that people were experiencing and the pains that their organizations were going through? Um, the, the second portion of the design partnership is around the actual roadmap, the capabilities that we want to add to the database and getting buy-in around that. So for example, um, it was clear over time that if people are installing a mission critical database, they want it to plug into the rest of the organization.

Ideally sending updates to other systems in real time. So we talked about adding a real time data streaming capability. There was a lot of excitement around that and that helped us de-risk the capability. But then you have the last part of the design partnership, which is giving feedback on the particular feature, getting engineers to use it, try it, kick the tires, see the performance profile.

That's where we ran into more difficulty. And that really led us to a mode of spending more time dogfooding right. Figuring out ways to use the database like our customers would to power our own business processes. And that's one of the last ways that we could de-risk the product.

 Was there anything else about the way that you structured the design partnerships, um, in those early days that you think is extensible, I guess put another way if, um, if you had somebody that was starting another infrastructure company and they're like, Hey Nate, we're just getting going on this design partnership process.

As we build out our early product, what are some things to keep in mind to make that successful?

 So the first is just recognizing that there are different levels of, uh, eliminating risk or reducing risk at the strategic level. Are you going in the right direction at the roadmap level and then at the feature level? And so before getting design partners at the feature level, that is the most difficult part in a, an infrastructure, a company it's identifying the companies that have the most pain.

 So for example, I talked about the customer that absolutely had to get out of the single region. They were willing to. Invest a couple engineers to make sure that what we build would help them on that journey to their, you know, cloud native architecture or their, their modern architecture. So that was a good design partner, but I can't emphasize enough the importance of eating your own dog food.

You know, there's a great examples that came out of Apple's design process, right? How do you build a product where you can't roll it out to, you know, millions of people early in and get early feedback? How do you build a product in secret and the way they do that is by living in the product, which is another way of dogfooding it.

And so that's definitely something that I would bake into the culture of the product and the engineering team, uh, very early on.

 And on that point, it was that easy to do, given the nature of your business, or did you kind of create some fake part of the business or do something so that you could fully utilize the product in.

 That's a great question. It was very uncomfortable because in many cases, early on it slowed down some of our product development and slowed down some of our, um, you know, core business processes. There was a point when I insisted that all of the product analytics that we do are done on the cockroach DB data.

And know, I just said Congress, DB is not an analytical database, but it was a way for us to see what that SQL interface was like, how easy is it to get data into cockroach DB, take data out of cockroach DB. Eventually once we were confident that cockroach DB could handle a basic queries, we didn't spend so much time running analytics on cockroach.

We ended up going to an analytics database. We did use cockroach DB for keeping track of all of our user telemetry from around the world. So if. Anyone deploys a database and they have telemetry turned on that is phoning home to a centralized cockroach DB database that is handling, you know, very high, uh, writes per second.

And that is helping us test the, the core product. Then we can see, Hey, how do we upgrade this database without losing our user, uh, telemetry? How do we stream updates from what is now our telemetry system of record to an analytic system, which is a core use case. And so initially some of those dogfooding opportunities were a little contrived and, you know, quite frankly painful, eventually we found opportunities to plug cockroach DB in, in the more natural place.

 One of the interesting things that you mentioned a second ago was, the idea that, strategy can fail in the real. And you gave one example when you were talking about crafting strategy in the early days around this sort of specific use case, but I'm curious if you zoom out a little bit more, are there other areas that you've found that strategies tend to fail or ways that you think about that translation layer between, you know, maybe a five page memo and one that what ends up happening in the real world?

 Yeah. You know, our strategy has evolved several times over the last five years. And one of the biggest failure modes that I've observed in other companies as well is coming up with a strategy without taking into account the capabilities of the organization to execute it. Right. So you hear a lot of people talk about strategy.

You hear a lot of people talk about execution. What skills do you have that make this particular strategy, a good fit for your, your team? And you have a couple options here. You can have an honest accounting of your capabilities, given your strategy and figure out how do you close that gap? Or you can say, Hey, this is the team we have.

How do we figure out a strategy that can make this team successful? And, and there's a, there's a give and take here. And this is something that we experienced that pretty directly as we started moving into more of a SAS product and delivering cockroach DB as a service where our customers don't run the database themselves, we run the database for them.

And so when we realized that, Hey, over time, we're seeing that there's way more developers than people who have the capability to operate a database. And this macro trend is pushing more people to offload operations to vendors. It became increasingly clear that we needed to figure out how do we offer a cockroach DB as a service.

That was a big part of our strategy. But then we had to say, okay, what needs to be true for this strategy to be successful? Um, and this is something that we had to figure out the hard way, by the way. Um, the first thing we realized is that, okay, our customers are having trouble hiring these operators, but now we have to hire them.

And these operators, these site, reliability engineers, they're not going to run just one database. They're going to run thousands and thousands of database basis. How do we find these people? How do we interview these people? How do we integrate them into our culture and make them successful now that this needs to be a core competency.

That was a huge capability that we had to build. Another one. Which he may be able to guess is that we are in the business of handling mission-critical data payment data, ledger data. If we're asking people to trust us with that data, we need to build a substantial security skillset. We need to figure out how to do network security, data security, there's all manner of certifications that we need to be able to pull off.

 how do you hire security experts on the product side, on the engineering side that can do that. So those are some of the things that we realized we had to build. If we were going to be successful, moving into, uh, becoming a cloud company.

 Another part that you hit on in this topic of early product strategy was the role of competition. And it's something that I'm, I'm kind of endlessly fascinated with. And I think that. Uh, my own point of view, is that the idea that like competition doesn't matter, focus on your customer, built for your customer and everything works out is, is actually quite damaging advice.

 particularly if you're operating in a market where there are sort of other players, which definitionally is most markets. And so I'm curious, how do you think about competition and maybe how has that changed over time and how does that fit into this sort of product strategy thread?

 Yeah, it's a good point. I also initially thought that competition didn't matter, just, you know, focus on the customers, work through first principles and you'll end up in a good place. But when it comes to you actually landing an enterprise sale or looking at a real evaluation process, competitors are involved and it's useful to understand.

Where your competitors, competitors are strong and where you're strong and how to talk about those differentiators in a way that, uh, favors you. Um, the, the place that I've spent the most time thinking about this is when it comes to strategy, because we have a class of competitors that have made fundamentally different strategic decisions than us.

Um, as an example, you know, Amazon has a great database, Amazon. And rather than make the entire database cloud native, they have just made the, the lowest level of cloud native and they've taken the execution layer from an existing database. And what this means from them strategically, or for them strategically is that they have a much simpler migration.

Right. It is very simple to take an existing database and run it on Amazon. Aurora. The thing that is in our favor is that those databases aren't cloud native, you don't get the same resilience characteristics. You can't move your data anywhere in the world. You, you can't scale to the same level that you can with cockroach DB, and that is a fundamental trade off.

And so this is helpful as we think about the investments that we make and where we can invest in our differentiators. But it's also helpful when we're arming our sales team to really talk about the real contrast between us and our competitors and position us to win. And in the ways that our customers find valuable. 

 Maybe we could sort of build on this and talk a little bit more about how you actually build product, how you. And maybe we could talk about it a little bit, what you figured out in the early days, and maybe what it looks like today, like how you actually run the product team, how you set direction, what are the intervals that you're building?

Like what does that look like? Maybe in kind of a step-by-step fashion in terms of, you know, if at the end of the day you have a new product release, what what's, everything that happens before that.

 So for some background, I lead the product team at cockroach labs and that includes product management, product design education, and our data and analytics function as well as product operations. the way that I think about building product is as a, uh, system.

 So I think about the, I call it the product operating system, where you have a product team for now. Let's just focus on product management. You have product manager. And what they're doing is they're taking inputs from the market, right. Externally, right? So what are our customers saying to your point, what are our competitors doing?

They're they're taking these external inputs, but they're also taking internal inputs. What are we hearing from our sales team? What are our pre-sales folks talking about versus our post-sales? What are the, what are the trends there? And they're taking all of this information, looking at it through the lens of a strategy, and then figuring out what are the ways that we want to change or enhance the product that will further our interests.

Right? That'll move us closer to our, our mission. So in this process of. Taking this input and turning it to, uh, this output. The question is what actually makes it on the roadmap. What are the questions that PM's need to ask themselves? So, you know, big things are outside of strategy. You know, how deeply do we understand the problem?

This is huge for us because we're in a space where it's very easy to look at another database. Let's say Oracle, and just do what Oracle did, but that was based off of the world 30 years ago. If you truly understand the. Now you can come up with a much, much better solution. And so we really put a big premium on that.

And this is especially important when you're working with engineering teams that may have a closer or a tighter, uh, intuition about the customer than the PM's that the PM's can frame the problem. Right? All sorts of innovation can come from the engineering experts. So framing the problem is huge. Um, there's also your standard ROI.

Does this make sense to build given everything we know, you know, risk adjusted? Is this a good investment? Given the engineering cost and then crucially there's the opportunity cost element. And this is all baked into our product process. If you add something to the roadmap, if you add something to something you want to deliver over a period of time, something is going to fall off.

with that, that trade off? So those are some of the things that the PM's are asking themselves. And then what they're doing is they're, they're partnering with engineers. You know, we, we go through a standard agile process. We have epics, we have user stories. So there's not much to talk about there, but the, the big question is, as these features, as these updates start to come to fruition, how do we de-risk them?

And that goes through the, the design partnerships that I talked about, acceptance testing, living in the product. Dogfooding these are all things that help the PMs build the right things. And then we roll the product out. The challenge for us as an infrastructure company is that we only do major releases twice a year.

Right. So we have to figure out given the, the risk of the feature. Is this something that we can put into a current release? Because we're very, very sure it won't break anything that helps us get it into more customers' hands faster, but it also limits the scope of what we can do or are we okay. Winning six months?

And so there there's that calculus. But that's a big part of the process. And then as these updates go into the market, we're seeing the product analytics understanding what's working, what's not working again. That's another form of input that helps us understand where do we iterate? And so this cycle just keeps going.

And at this point as a product leader, I don't have to understand the, you know, what a product manager is building, what feature, as long as the system is working, as long as we're getting high fidelity inputs. And we have this feedback mechanism and we're correctly plugged into the rest of the organization.

I know we're going to have a good outcome at the other end.

 So how does that process fit into annual plans? And do you start with a set of themes or areas? 

 That gets filtered down through the line level PM or what sort of like an annual cadence and maybe how does this kind of tactical process fit into it?

 So I mentioned that all of this happens in the context of a strategy. And if we drill in there, the way this ties into our annual planning is we have a strategy that updates very infrequently. You know, maybe we'll make tweaks once a year, but this is something that is fairly static. We codified that strategy into, um, okay.

 Ours and those OKR is, are what ultimately set the, the roadmap, right? So if we want to become the developers database of choice, and we're looking at new serverless starts, or new database starts, um, you know, top of funnel as one of the leading indicators, there's a PM that is responsible for thinking about that entire funnel.

That, you know, moving those key results is one of the things that they're looking at when they're, they're setting the roadmap, what are the epics? What are the user stories? What are the enhancements they are going to do that are most likely to, to move the needle there? And then what happens is we have a monthly business reviews of things that we call a product councils, where the PM that is responsible for a particular product area or a set of user journeys.

We'll sit down with a cross-functional group of stakeholders to talk about how things are going. You know, at the beginning of the month, we were trying to move this particular conversion rate. Here are the activities that we took to do it. How is this playing out? Here's what we think the next steps are, you know, sales engineering is this aligned with what you're seeing?

What are we missing? Marketing? How are you feeling about this? So this is a cross. Uh, meeting that happens with them once a month and it's a chance to reflect and, um, really understand what are the next steps, right? a, it's a course correction and, uh, information sharing mechanism that we have on a monthly basis in the context of this broader, uh, annual plan. 

 Anything else you can share on that format of product councils? I think that's a really interesting idea. And I'm curious to learn maybe if somebody wants to implement a similar idea at their company, what are the things they need to know about how it actually functions?

 Yeah. So the way that we run our product councils are primarily through, uh, docs and, uh, Read, um, a meeting. So what happens is the product manager for their group. They, they have a set of, okay, ours that are scored and rather than just creating a deck or providing a scorecard for how things are going, they write a six page narrative around what's working and what's not working.

62857ca281b9dd50da3c191d

Nate Stewart is the Chief Product Officer of Cockroach Labs, which is the creator of CockroachDB. He unpacks the ins and outs of building a product org for a highly technical enterprise product — from crafting the product roadmap, teaming up with the right design partners, and avoiding a product strategy that fails in the real world. 

Essential advice for building a highly-technical product

Nate Stewart, CPO of Cockroach Labs, unpacks the ins and outs of building a product org for a highly technical enterprise product — from crafting the product roadmap, teaming up with the right design partners, and avoiding a product strategy that fails in the real world. 
