How to measure product-market fit with the REV model — Artem Kroupenev

 All right. Well thanks so much for coming on.

 Thanks for having me. Really appreciate it.

 Maybe just for context setting, given, a lot of folks might not know what Augery does or sort of the scale of the business, could you set a little bit of context and talk about what the company does today and maybe some sense of the scale.

 Sure. Absolutely. So today, Augury is over 400 people, approaching 450 people. To give you a sense of the growth, we were at about a hundred people, two and a half years ago, So we've experienced rapid growth in the last few years, especially through the pandemic. our companies focused on the industrial sector, industrial AI, and iot and hardware.

And we're a leader in a category that actually we helped create called machine health. meaning that we make sure that machines and industrial world manufacturing world, Don't fail or fail much less frequently. And that enables factories to keep running, to run more efficiently, to run more sustainably.

and we do that through a combination of, devices, sensors, and machines. And then ai, they can predict and prescriptively tell you how to fix machine failures way ahead of time with a very high degree of accuracy. in addition to that, we've expanded this category into something that we call process health, which is essentially geared towards process engineering to help you run production lines.

If you're a manufacturer to help you run your production lines, much, much better. And if you're producing a certain product to help you understand how to produce that product, consist. and more efficiently and again, more sustainably. and, that, kind of combination of, the health of equipment, the health of the processes, and also the health of other aspects of how you run manufacturing essentially comprises, what, what AUG does, and the value that we provide to the industrial manufacturing world by helping specific functions around maintenance and reliability and engineering, process engineering to do their job a lot better with the help of artificial intelligence and the kinds of products and technologies that we provide.

 So with that as a little bit of a backdrop, wanted to start by winding the hands of time back to the very early days, the pre-product market fit days. And maybe as a starting point, you could kind of walk through your journey into product market fit and what those very early days tangibly looked like.

 Yeah, absolutely. like a lot of companies, that journey was not simple or easy or straightforward. I think a lot of the, things to consider in this industrial space is that there's more complexity

The other, part is in the early stages we were thinking about solving a pretty big problem around the health and productivity and efficiency of equipment. And to get there, it was very clear that artificial intelligence and combination with very clear dataset and signals from those machines is the way to really build something that can provide a lot of value.

But to get the initial dataset, was very difficult. It just did not exist. So part of the challenge early on was, well, how do we get the right dataset to create a smart system, a system that is based on artificial intelligence that's also highly accurate and dependable. While we don't take 10 years of investment just to build it out, but rather way I can actually provide service, that provides value day.

So, we created a device that can be attached to a mobile phone, a smartphone that technicians that are in charge of repairing, rotating equipment, would go and then record that equipment with that device, with that sensor. And then on the backend, we would have actually people providing the diagnostics for those machines.

And we created a system that can, enable experts to provide that level of diagnostics. But as we were doing that, we actually, got one of the largest databases of machine, health data in the wild, that then we were able to utilize in order to create algorithms that can analyze those machines automatically at a higher and higher rate of success and accuracy.

And so we have essentially built a product that transitioned from a manual. Human-based diagnostic, tool and capability to something that is enabled and enhanced by ai. and then eventually, almost completely diagnosed, diagnosed based on algorithms. So that transition, how do you build a product that goes from a massing database that can, you can build algorithms on to then providing your service and gradually scaling the people within that service so that, at the end of it, you have a really highly accurate and independent and autonomous AI system that can do this, from the first, from the minute you have a, a signal coming into it, into the highly effectively.

That was the transition that we had for the first product that was part of the product strategy. and, and also part of its challenges. So that's what happened in the early days in terms of kind of thinking about the, the product market fit.

 So let's go back to the first sort of set of ideas that you shared. Where did the actual foundational insight come from? And maybe you could go a few clicks deeper in terms of what the actual first 12 months looked like and the interplay between maybe talking and working with customers, building early product, that type of thing.

 Sure. As I mentioned, when you talk about the, the initial insight to build a product that is an AI driven product, an IOT driven part product within the space, that came from the founders, and having a good amount of validation with customers just speaking to various, people in different industries who are responsible for the health of equipment and can also hone in from a technical side on what is the, widest possible application for this type of technology.

So that was the initial understanding, kind of top down to, what market to focus on first, to be able to get a great, a good enough database and also to provide the product that provides, value widely. when we went, and started looking at specific segments over of where this is applicable, we traveled and then I've traveled to over a hundred different manufacturing facilities and spoken to hundreds of people in the space to really provide a good map of what are the different segments that, have a high level of pain, if you will, around this type of equipment where this equipment is responsible for the production process and production outcomes.

where breakdown is very costly and also at the same time, they have some level of understanding and maturity. Of the value of avoiding failures around this equipment, and also maturity in terms of what is their willingness to, actually acquire a solution and change the way they work today.

and for some segments, that level of maturity and understanding was high. And for some of those segments it was very low. So we created that map over a period of about six months. And from that map, it

 Before you can, can you give us more of the architecture of those type of conversations? what specifically were you asking? What were you listening for? How did you avoid normal confirmation bias or happy ears where you know you like your own idea and you want them to like it, to that type of stuff.

 Yeah, absolutely. well, one thing that was helpful is that, we had to get an understanding of the field itself, all right? these are professionals and the work within the field of reliability and reliability, engineering and maintenance. and so one thing was really, being on the ground with them and observing and asking them about what is it that you are doing to solve this problem?

Or what is it that you're doing in your day-to-day, in order to figure out how to fix this? Or is this really a problem for you? and in some cases, a lot of users would not understand, or customers would not understand that this is a problem because that's the way they would do it.

But when you go a level deeper and ask them, well, how much time does it take you to do this? What happens if this fails or will I have to wake up at 3:00 AM to go and fix it? and well, what do you do to, to avoid waking up at 3:00 AM In some cases, there's no answer. In some cases, there's a lot of manual work that has to be done.

So kind of chipping away at the kind of the five why's, if you will, of what the problem is and really listening. and there's some methodology to it, right, to avoid confirmation bias and then, and so forth. But, really listening, not just for the functional outcomes of what is it that they do, to solve a problem today, but also for the emotional connection, for emotional, outcomes.

and then having the ability to have a, a working product or a version of a working product, as an MVP to help them, alleviate some of that pain, then you can actually start testing that in use, right?

So when I would travel to, to these factories, we would have a product that would help them see how a failure could be avoided. And then once you see the kind of the eyes open up, it's a very different conversation than just thinking theoretically about what could be done to avoid that problem. In this space, my experience is that really nothing replaces those face-to-face in context conversations, because it's very different to talk about high level what the pain points are versus really walking the production line with a person or really getting under into a basement where the H V A C system, where the compressors or the chillers are, where it's dirty and loud with the person who's responsible for the health of that equipment and really understanding what is it that they experience.

and that helped us really hone in on, on a lot of the user level pain points at the customer level. then it, the validation for the product and the sales process at that stage of the product are really one and. Right. So not separating the sales conversation from the product discovery conversation, but rather combining them in a way that, is helpful to both, was really key to, to get a fast learning cycle.

 do you remember the specific questions that you would ask in those early conversations? You talked about the five why's. Were there any kind of specific line of questioning that you found particularly useful?

 I think one of the key ones to avoid, bias was to, instead of asking, for instance, have you experienced this problem? Or What is the biggest problem that you experienced? Asking more specifically, how many times have you experienced this in the last. or give me three examples of where you have experienced this.

and then from there, then that kind of removes some of the biases that the person might have and trying to come up with the best answer, but rather speaking facts and laying on top of that, the questions that hone in on the functional impact on functional jobs to be done and the emotional impact.

So once you get the facts of when and how this has happened, let's say we're talking about specific pain point, then say, well, how did you feel at that point? And how did you feel when that, when it was later resolved? And the same time, what was the impact of that on, operationally?

What was the impact on the business? What was the impact on your team, and how did that get resolved? Or how you resolving that today? So really that, I think that kind of approach. of getting first to the facts and then help understanding the emotional functional impacts of those facts.

That was a way that, I would structure those interviews.

 I wanna get into the details of your first or second or third customers, but one thing that you mentioned was this idea of combining customer development and sales together in some way. Could you share more about what that looks like? And I think it's such an important topic because when I think founders are bringing something to the market for the first time, there is this sort of delicate balance between when does customer development stop and sales begin?

 Yeah, I would argue that, customer development really never stops, but it becomes a smaller portion as you become more and more confident within your product market fit. Especially if we, where we are talking about the early stages than it would be founder led sales, right? Primarily, you have a senior person that would, be responsible for, for selling and especially in within the enterprise space, right?

where we live. when we talk about customer development, the process of understanding the things that you would want to know and need to know for go to market fit or for on the sales and go to market side, are extremely relevant to how you build a product and what the interaction is with.

so you have to have a combination, an understanding, and a combination of both. So when, when we had a conversation with a customer, we would ask questions about obviously functionality, user level, pain points and so forth. But we would also combine that with questions about the buying process, how budgets structured, who are the teams and people and personas who would be involved and influence that buying decision.

And then also understanding where the value lies within their organization.

So having a list of assumptions, hypotheses around the customer side and the customer journey, and the personas as well as at the same time, the user side, the user personas and the user journey. and having the ability to validate both within that, those conversations, is critical.

I think one of the very tricky things in kind of the first month, 6, 12, 18 months of building a new product in a company is figuring out how fleshed out that product actually has to be to get the data that you're looking for in kind of the interplay between iterating on the product and having customer conversations and how many to have before you go do a lot of building.

 and I think you have these opposing forces. One is that you get a lot of data when you show a product to someone or give it to someone. At the same time, you don't wanna go off and spend three months building a fully featured product only to find out that you missed something. And so what did that early product look like?

How did you figure out what you actually needed to build in order to get in front of an early customer for some of these conversations? And any key learnings around that.

 Yeah, absolutely. So, there there's not a clear cut answer. What is the fidelity of your mvp? because, the, the technology matters, or at least in the space that we're in and the kind of product that we have, technology matters, and the deep technological capabilities that we had to create and engineer and some cases invent, they really matter as well.

because, if we would've just come up with a mockup, a concept of something, and this really happened in the very early stages, there was a lot of doubt. I would say 80% of the people would doubt if this was even possible to predict machine malfunctions based on, basically on sound, the vibration, uh, and to do that effectively at scale.

So we had to show, so. in the space that would show that it is possible, technologically possible. one way to get around it, for us was to instead of having, these AI algorithms that are highly accurate, that took many years to develop and, and perfect. we had an application where you had a human expert on the other side provide that level of expertise.

So we simulated the experience as it would be in the future by having a person on the backend, provide those diagnostics. and it worked, right? so that basic function of what is the core view product, from a technological perspective and maybe the simplest user flow that you have, I think for us was very important to have in order to be able to have customers and specifically users.

Who are used to working a very different way to believe that this is possible and that this, this can actually work. I can imagine that for some other applications it might be different. but for us it was very important to prove that in the show that, that possibility exists.

once you do that, then you can actually move beyond that disbelief, that initial anxiety, then the conversation becomes very different. They're, you say, okay, this is what you're bringing to the world and, significant innovation for this world. And you can actually build that out.

Now I can believe how all the different aspects of my work and my day-to-day, and also even the function itself can change in the future.

 You mentioneda really important point, which is this idea of early customer skepticism and potentially the more skeptical a potential customer is, the more important it is to have some version of a functional product. And I'd be curious, in the case of Auguery, what was the root of the skepticism and was it that other companies had made false claims where they said they could do this or they couldn't, or just in the abstract early customers couldn't get their heads around this being possible.

 Yeah, there's definitely a combination of both. in the space. I think when we just started then it was more, a lot more around is the safe, impossible. But then as we started to grow and some other companies came into the space of industrial ai, a lot of promises were made. and so the market or the hype within the market increased.

and so that created a lot of noise. And so then we, adjusted our messaging and our perspective from okay, from just proving that it's possible to saying, well, the way we approach this and the way we build a company around and the way we're building our product and the way we're working with our customers and users, that is what actually makes it not just possible, but very tangible and useful.

So, part of the messaging became not just, the product itself, but also our approach to building that product and our approach to working with customers. And so, as we evolved from just pre a pre-product market fit company into a company that has product market fit and has go to market fit and has created its own category the ability to be a leader, not just in the technology.

but also a thought leader of sorts in what is the right way to approach building products in the space became increasingly important.

 Can you tell us the story of the first customer

 I think, I don't say that, this customer was the first, but there were definitely one of the more influential customers for us.

and so this is a large, manufacturing company, cpg, so consumer package goods customer, a company. And what was really prominent about them was not the size, so much the size of the revenue that we initially got from them, but their belief and dedication. Most primarily in our team, but also in our team's ability to solve some of their bigger problems longer term.

And the relationship that we had with them that, made them, one of our best customers and also one of the customers that, without whom it would, would've been very difficult for us to, to really grow as fast as we have and to provide the same level of value. the initial engagement that we had with them was one of, well, skepticism, right?

they definitely had some anxiety around, will this work, will it pay for itself? And so forth. But once we pr, if we've proven that part, we really got some key champions within this organization. And when I say champion, champion is somebody for us, is that's at a. Let's say corporate executive level that has the ability to take our product from one place to the next and kind of create, oversee a level of scale.

They won't necessarily fund all of it, but they can drive it within the organization because they believe that their personal interest, the interest of their, organization are aligned with what we can provide. So, once we prove the initial value at one manufacturing site, and we avoided a few failures for that company that essentially provided enough ROI for the program to pay for itself for the whole year.

We immediately had the kind of the confidence within their leadership team and one specific VP level champion became very vocal about our solution internally and more importantly externally to other customers. So if I would say one of our best customers was not the biggest deal that we closed initially, even though that's very important, it was also not, the customer that helped us, improve the product the most.

even though they had a lot of input. But it was the customer that was willing to talk about our solution to their peers and lend their credibility to our solution in a way, that unblocked something in terms of, the marketing side of things, the strategy side of things, but also gave us confidence to speak to.

Their peers in a different way, different level. And that changed the way we go to market, or it accelerated the way we go to market. So we, instead of selling factory to factory, we were now able to sell top down and bottom up at those factories.

So I think, to me that was one of the, one of the more important partnerships because they enabled us not just to unblock our product, to unblock initial revenues.

They helped us unblock the whole market in a sense, in that, in that early stage of a journey.

 And did you structure the original way of working together as a design partnership?

 We did not structure it initially, but we very quickly moved into that space. Um, I mean, most of our early customers, we were approaching them speaking to them as design partners in a way, right? Because I, I mentioned the ability to, do the customer development piece and the sales piece at the same time, and under promise and over-deliver, but at the same time speak about the grander vision for the future that we can build for them and gradually prove towards it with, with this customer.

We very quickly moved into that design partnership capability where we did not just talk to them about the value of our product today or even in a year. We talked about what are the biggest problems they have in terms of their digital transformation around manufacturing, and created even a mutual roadmap of sorts of where are there areas we can actually help in the future? so it evolved into a true design partnership in the sense that, that whole organization started basing their strategy in large parts on some of the things that we can provide, in the future. So, yes, and we have a number of companies where those are design partners, and now that we have a portfolio of products, so we're bringing more products to product market fit, we utilize exactly the same model.

 Any other quick thoughts for founders who are structuring or thinking about design partners, things to do or things to avoid, or things to watch out for.

 I think one of the key things that has been helpful to us, or has actually driven part of our success is a clarity of vision.

So when we, talk to design partners or to customers in many ways, they're buying into our vision of what the industry, the market. Could look like. And obviously it needs to be aligned, right? That vision is not completely disconnected from what they're looking to achieve. It takes into account the pain of the transformation and the kind of the day-to-day what the customers are looking for.

but typically, as a, as a technology provider, our vision is going to be more focused on kind of the five years from now. and it's, it's ambitious, right? But it also has provide some clarity. And so what, when we speak with, uh, let's say it's a new design partner we're talking to, we start with vision.

We start with, here's what we believe the market should look like in the future. here's what we believe. A lot of companies that are early adopters are going to be, and they're going to have a huge competitive advantage. Here's what that might look like. How does this align with what you have?

But the way you were thinking about. And so that level of vision alignment is critical. And if you don't have a point of view on what that looks like and you're trying to back into what the customers are telling you, in order to develop that vision, it's going to be more difficult because you are not going to sound like the executive at their company.

But rather you're going to sound like, somebody who's in charge of a niche or a part of the space that's more execution focused. Once you start speaking vision, you sound like an executive. And, you sound like somebody who can help them lead and guide the company into a version of that future.

 So I wanted to, to revisit a. With the terms that you have woven into the conversation thus far. You talked about product market fit, go to market fit, category creation and product portfolio category expansion. Could you pick apart each one of those? And if you think about the term product market fit, what?

What precisely do you mean by it? What does it mean to you? Cuz it's one of those squishy terms. And can we do the same thing for go-to-market fit, et cetera?

 Absolutely. Yes. So sure. this is the model that, comes from a number of different places, right? some experience and some, some other places and, books like, survival of Thrival and Scaling Up and, partly Empowered and, founders at Work and the Dean Startup and so forth.

So it's a combination of different, different materials and also, Through some of those problems within our space. I don't think it's unique in any way to, to us, I think it applies probably to a lot of companies in the same enterprise space. The consumer space might be slightly different, but for product market fit.

essentially today, if you ask a bunch of people what product market fit it is, there isn't a very clear definition. there are some people who define it better than others. one of the definitions I heard is when you, you can transition from a product, being pushed onto customers to being pooled by customers, I think that's a pretty good definition.

we, have a more measurable approach to it that we've developed over time. But product market fit, I believe is me. definitely measure measurable if retroactively. and that is a point where you have the revenues, the engagement, and the value that's clearly visible and shown to customers.

I call that the rev model, r e v revenue engagement value that you can point to and say that this, this is a viable product that produce, the business models, right? They're engaged users that will be very upset if they no longer have access to the product. And there's, measurable value for them personally, for the customer or me, the organization, as a whole and also potentially wider for the industry in the world that you can actually measure.

So that's revenues, engagement and value for product market fit. And you can set metrics around each one of those, that, forward looking. That's saying once we hit. Some of these metrics. So we have early indicators of the metrics that we're hitting around revenue, engagement and value. We're actually on the path and we're hitting product market fit.

 before you move forward on that rev model, can you give an example of what metrics looked like in the early days or if you were building a new product? When you talk about setting metrics against that, I think it's important to be specific and give some examples to, to help bring it to.

 Sure. So, here's, here's the approach that we took with our most recent, product. And say, okay, so what we say, there's revenues, engagement of value. Those are the three categories of metrics that will provide us a measurable indicator of product market fit. And then what is the timeframe?

Let's say that timeframe is two years from now for revenues. what is the revenue model that we're going after? And for us, it's land and expand, right? Meaning landing and expansion. we should ha see above a hundred percent net dollar retention in that space. And, for land, the metric is four quarters of increasing revenue and the segments that we defined.

So let's say we've defined five segments and at least three of the segments that we've defined for our. We want to see four quarters of increasing revenue growth, expansion growth out of those segments. We want to see at least one account over the two year period that is expanding. And again, this is, more, centric to what the way we operate and the sales cycles that we have.

If you have much shorter sales cycles than you have different timeframes, you can define the segments differently. You can define the metrics as is fit to your market and so forth. But, for us, we kind of break that down into land metrics, expansion metrics. And we also take into account other means of exchange of value.

Meaning that if we're getting testimonials around marketing referrals, advisory lighthouse sites and so forth, we take that into account. And so we define metrics around the other types of exchange of value and mostly around marketing. that is. An indicator of that part of the equation, the revenue part of the equation, and market growth, right?

The engagement bucket, is really around first figuring out what is the engagement model. So it said revenue model. What's the engagement model? The engagement model is there, what's the component of direct usage and indirect usage? And if is the direct active or is it inactive in some way? Right? So, meaning that if there's directed usage, somebody logs into your platform and what's the active model?

do they log into the platform every day for a number of hours every week, every month, right? and maybe it's a different type of product where it's more of a, push kind of scenario where they only get, get in there when there's an event or so it's an event based indirect or, sorry, inactive usage.

And so, Then you need to have measures around that type of usage. And then there's indirect usage where somebody is, like, an executive gets a report, right? Or a dashboard once in a while. And so what does that, engagement model look like? So once you understand what does your hypothesis around the engagement model, direct, indirect, active, inactive, push pull and so forth, then you def say, okay, I need X number of daily active users, monthly active users, weekly and so forth.

And once we hit that number of users, based on that model, you will achieve the, engagement piece of product market fit. The last part, last bucket is value. And value is divided into two parts. One is operational value for the customer, and the second bucket is, Right. So operational value should be tangible.

It's ROI or total cost of ownership. What is the unit economics? What are the unit economics of roi? What is the value specific time period or effort or money that you're saving the customer? How does that translate into impact?

And so let's say if you have a confirmation of a certain level of ROI for the customer, let's say across five to 10 customers, then you hit the operational value piece of value and then the satisfaction piece. You can measure it in various ways. Some are more lagging, and some are more leading indicators.

But, one of my favorite ways is to, let's say you have, three or four customer users, and you go to them and say, if I took this away from you right now, how upset would you be? Right? So that's, that's a common way of doing it. The other one is nps, and so forth. So, revenues, engagement, values, set those metrics across those categories, say two years from now, once we hit those, we can declare.

Victory. Right? Same way we have product market fit. At least we're tracking on that way. And as you figure out, that some of these metrics are not right. Exactly. Or maybe they are revenue models. not exactly as you thought. Maybe your engagement model, not exactly as you, you imagined maybe your value model is slightly different, you can kind of adjust it.

644a87d544588c340a959a36

Artem Kroupenev, VP of Strategy at Augury, shares by-the-book frameworks for how his company measures product-market fit in the complex field of industrial equipment.  