The secret lever Replit pulled to scale ahead of its competition | Amjad Masad (Co-founder and CEO)

Todd: All right, Amjad, welcome to the show.

Amjad: Thank you. Excited to be here.

Todd: So if I understand correctly, I think you told me that you came up with the idea for Replit, you know, way before you started the company, I think when you were a student. Do you mind kind of taking us back to the very beginning and sharing that origin story?

Amjad: Yeah, for sure. I was, uh, doing computer science degree at, uh, the Princess Sumaya University for Technology back in Jordan, where I grew up. You know, like many computer science students, the first thing that you're faced with when you're taking a computer science class around programming, whether it's object oriented programming in Java or distributed systems in, C or whatever it is, there is this initial slog of setting up the development environment.

And it's not this one time thing. It's, it's a constant thing. you know, if you talk to any engineer, if you're an engineer yourself, you've had the struggle of like, it works on my machine, I have the right libraries, it doesn't work on my, project partner's machine, it doesn't work on my professor's machine.

It's hard to code together. It's hard to share programs. And, at the time, I was, really excited about the web. Like, this was 2007, 8, and, Chrome had just come out. And with it, their V8 JavaScript engine, JavaScript was getting really powerful. You had, um, what we used to call, I think we just call them apps now, what we used to call single page applications like Gmail.

 Thick clients app and things like that. The idea that you can build

applications applications in JavaScript was new. Google Doc had just come out and wow, you could like share, you could share a document and you could like be in the same document together.

Todd: No, I remember that time, that's when I was at Google working on Gmail.

Amjad: amazing. Amazing. I mean, yeah, you guys, uh, really, uh, inspired, an entire generation of people to actually imagine what, what, what the web could be. And I thought surely someone was working on a collaborative web based code editor, of course. And then I looked around and there was nothing.

There was nothing. I think there was like a project by Mozilla to like create the editor component. But, you know, there was nothing that allows you to run code, to share programs, to collaborate with each other. And I sort of, in this, naivety of, being young and inexperienced, I was like, "I'll do it." You know, "How hard is it?" And, turns out the prototype was easy. I just created a text box, you can put in some JavaScript there, click a button, evaluate, and it will like literally evaluate the code. and, you know, there's a way to kind of save it and share it, and my friends loved it. And so I immediately got instant feedback that, you know, what I'm building is, is useful, which is what you want to hear as a developer. And I was like, okay, well, you know, let me add a few more languages how hard could it be? And I started writing interpreters and compilers. 

 Python was becoming really big and we were starting to use it at the university and data science and machine learning was starting to be a thing. I was like, I want to add Python.

and yeah, that was incredibly naive of me to try to write a JavaScript interpreter for Python. I actually went down that path and it took me six to 12 months to give up. Uh, you know, I got like Hello World compiling and running and all of that. you know, didn't pursue the project anymore which was like really difficult. Then one day I was like on the internet and I saw this project by research group at Mozilla to compile different languages to JavaScript. And by the way, Google was working on on NaCl native client, I remember that, that allowed you to run, native programs in the browser.

But there was this alternative vision of actually, can we compile these programs to JavaScript? Can we treat JavaScript as a bytecode interpreter? And that captured my imagination. Okay, okay, I'm going to go like compile all these languages and it actually worked. I got a prototype running and we're the first to put Python, Ruby and a few other languages in the browser and I open sourced it.

Todd: And this was all still in college? 

It 

Amjad: It was all in college, you know, nights and weekends. 

Towards the tail end of it, I think after I had open sourced it, but before it sort of started taking off and people paying attention to it, I had finished my degree and I was working at Yahoo in Jordan and it had acquired an email company called Maktuba which was one of the biggest acquisition, 200 million dollars. It was like the biggest national acquisition for us, back there. And so I continued working on it nights and weekends, and it sort of exploded on Hacker News and GitHub and everywhere on the web. Around that time in 2011 there was a demo app, it was called Replit, but the thing that really took off was the underlying engine, called JS REPL. And a bunch of companies started using it in the US. we had, uh, Udacity, reach out to us. They were just starting out. There was this MOOC revolution that was happening in the US and Silicon Valley where, this idea of being able to, learn online, is capturing people's imagination.

Codecademy was part of that movement and they had just finished YC and they were using a lot of the packages that we wrote for, for Replit at the time. And so, I had the option to either start my own startup and do it in Jordan where there was no infrastructure for it, no venture capital, or join this company that was just starting out and that was more exciting to me to be able to come to the US. It's been a dream of mine. And I watched, the first time I had this idea of like, I want to be in Silicon Valley was when I watched, Pirates of Silicon Valley, this like low budget movie of, uh, Steve jobs. That's how I got my start. I came to the U S and worked at Codecademy for a couple of years. 

Todd: So Codecademy was actually using pieces of Replit and that's how you sort of got connected to Codecademy?

Amjad: Yes.

Todd: What were they using?

Amjad: you know, Replit from the start was sort of this boil the ocean project and we had to write a terminal emulator. We had to write a lot of just web infrastructure to be able to run this thing. We had to contribute it to Emscripten and write a lot of, Emulators around, making Python programs run on the web, for example. So they were using a few of those packages, and then they contracted me to use the core engine to add Python to Codecademy. And I did that back in Jordan as a contractor. I was getting paid 15 an hour, which was a lot of money for me at the time.

Uh, and so by the time I joined, they'd already been on that, stack.

Todd: Cool. Okay, so Codecademy 2011, and that's sort of enabled you to, immigrate to the United States. and then you were there for a couple of years and then you joined the JS infrastructure team at Facebook, right?

Amjad: Yeah, I actually started the Jetson infrastructure team at Facebook I joined, Facebook in New York, it was a really small office. It was kind of a startup in 2013, late 2013. initially, actually the first thing that I worked on was the Android app. One of the things I was excited about, and the reason I joined Facebook actually was internet dot org uh, and like Android as a platform, I always had this view that, the world was, in the middle of this massive shift where everyone would get access to computers via phones and tablets. Everyone would get access to the internet. And we're actually almost at the tail end of that, now, you know, more than a decade later, which is true. Like there's almost universal connectivity now. So I wanted to contribute to that, uh, you know, Zuck's vision of, internet dot org really captured my imagination. But, uh, one of the things I really hated about working in the Android app at Facebook was the development life cycle really sucked as someone coming from the web. You added a line of code and you would wait for the Java compiler you know, 10 minutes to change that. And so I immediately gravitated after doing some product work towards infrastructure again. The, first infrastructure project I, I joined was, uh, React Native and React Native was about bringing the web back to mobile and or web technology or web style of doing things.

And I was like, probably the third or fourth person on that team. It was the ReactJS inventor, Jordan Walke, and a couple of people. And so I joined that. It was like very, very small. I wrote a lot of the JavaScript infrastructure for React Native, meaning you know, the compiler, the build pipelines, and everything.

And then we decided to use that to build a team, to modernize the JavaScript infrastructure on facebook. com. And so I built a team, I recruited people, I recruited the maintainers of, of Babel JS. They joined, he joined Facebook, and a few other people, we built Jest, we built, Babel, by the time I left, they also released Yarn. And so that team ended up being very prolific and really built the modern JavaScript dev tool chain, and that was a really fun experience. Eventually we ended up powering facebook. com just before I left. And so that was really fun.

Todd: You were working on incredible projects and projects that are now widely used. Were you also in the back of your mind thinking still about Replit and, you know, maybe I should make my own company out of this?

Amjad: You know, I always was attracted to the entrepreneurship track, but took a, a more sort of technology view on this and like a worldview on this. So, so I had this basic beliefs, like, One: that's universal connectivity, universal computer literacy, universal computer access, and that was incredibly exciting to me. Two: there's going to be a lot more developers in the world. There's going to be a lot more flavors of developers. You don't have to go to computer science degree for four or five years to be a developer. and, and Three is just, software development kind of sucks and we could make it a lot better. And those kinds of ideas and call it a vision, sort of dictated how my career progressed and I was attracted to projects that fit within that rubric.

And so by the time, you know, my time at Facebook kind of concluded, and by the way, that was one of the most productive, infrastructure time at Facebook, where a lot of open source happened, and I thought it was sort of coming to an end, obviously they're, they're still doing amazing stuff with Lama and things like that.

And then I was like, okay, you know, whatever happened to cloud environment environments, collaborative programming. And at the time I had started thinking about AI because I've written compilers and minifiers, and I always thought in the back of my head that it's an area that's incredibly ripe for disruption by machine learning. It's a very laborious thing to write a compiler. It's actually somewhat straightforward and it's a fun thing to do, but, you know, applying machine learning on code, and actually read a paper in I think came out in 2013, it's called On The Naturalness of Software, and there were some researchers that had this bright idea that is now standard.

The idea is that. You can look at code, and treat it as natural language. You can do language modeling, you can do N-gram language modeling, you can do all sorts of language modeling, and build developer tools, that treat code as natural language. And basically what, that's what we do with GPTs today.

And that was also in the back of my head. So, in the actually in the first deck, that we presented for the seed round at Replit, I actually talk about the prospects of AI coming to coding. And so it's like, step 1: We're going to create this cloud development environment. We're going to make it collaborative. We're going to make it really fun and exciting. Step 2: Based on this network and data that we're going to be able to collect, we're going to be able to train machine learning models that make coding a lot more accessible and easier. Step 3: We're going to be this platform where people can start, uh, startups and, really democratize, access to the incredible wealth creation engine that is like making software on the internet.

Todd: Yeah. I mean, that's pretty wild ghostwriter as, you know, step two and, and getting all of these people, you know, coding in a browser based ID. That's amazing. So, but what was like the first, what was the first thing, you know, like you left Facebook 2016, 2017, what were you building at that time?

Amjad: It was reviving the open source project. It had been basically dead for a few years. you know, we still have that like sort of demo page. And so the first thing I did was I actually moved away from browser based execution. I declared it a dead end. And it was sort of controversial because, at the time, there was a bunch of competitors springing up that were starting to do a browser based execution.

I thought it was a dead end because when we tried to scale it at Codecademy, the idea of downloading tens of megabytes of JavaScript on a Chromebook in Africa. It's just a non starter. Uh, and so I thought cloud with universal connectivity and internet bandwidth improving, I thought cloud would be the way to go.

So I started rewriting the project to become cloud based execution. Docker had just come out. It became easier to do cloud based sandboxing. that's the first thing I did and then added a bunch of languages and then the project started taking off again.

Todd: So did it start getting kind of adoption immediately or did that take a little while?

Amjad: Immediately. and I think it's because, it was still getting constant traffic, although it was broken and abandoned. 

Todd: Oh, even when, even when you were at Codecademy and Facebook, the thing was still 

getting some amount of 

Amjad: traffic?

Yeah. I mean, it would cost me nothing because it was a static site. So it was just up. Every now and then browsers would break, how we did Emscripten and, and all of that, and I might go fix it here and there.

But for the most part, it was kind of abandoned and broken, but people still kept coming back because there was nothing else out there. There was Cloud9 at the time that was like the hot startup. They had, did a very ambitious thing of trying to build the full IDE, and runtime and put it in the browser.

They got acquired by AWS and now it's still part of the AWS offering. Although I think it's sort of abandoned and semi dead at this point. Tons of respect for that team and how early they are, but what I thought was missing, I didn't think it was web native. When I think about web native software, there's a few concrete things.

One, is that URLs are very important. Figma is a web native collaborative software, you can share URL with someone and they can hop into the same environment as you. You know, multiplier by default. That's something that's very important. None of these online IDEs did any of that.

So we try to build up from the ground up to be this web native software. That meant we're taking a much bigger challenge and that meant, you know, that maybe monetization was going to be harder. That meant maybe that professional developers are not going to be able to use it at work for a long time. And that, that meant you know, we were not very fundable as well because we were not pitching an enterprise SaaS thing, we were just saying " Oh, we're going to get a ton of people to use this thing." question mark, question mark, and then we're going to make money.

Todd: So this was, so this was maybe late 2017, early 2018 kind of timeframe yeah how many people were using Replit and like, who were they?

Amjad: I want to say like on the order of a hundred thousand people. They were a combination of students that found it through a teacher, there were teenagers that found it on their, on their own by just googling because, because, web native teenagers, they just assume that everything runs in the browser.

Actually, you know, a lot of them do not even have the concept of files and folders. I mean, they find it very distracting. And so they would look for it and they would, they would find it. And then, a long tail of different professional developers using it for prototyping, people on Stack Overflow using it for bug reproduction. And we built up this massive footprint on the, on the web of people sharing links to rebol's on, had this viral growth thing because you could just share a link.

But, winter 2018 is when we got into YC and we had already, significant traction.

Todd: So had you tried to raise a venture capital money prior to YC or was YC the first?

Amjad: we did raise a, pre seed. it was very difficult, but, our very first check was actually, Christina Cacioppo. She was doing scouting, I don't know for which firm, She's now the CEO of Vanta uh, who just announced that they have 100 million dollars in ARR. But then, uh, Bloomberg Beta and, or Roy Bahat, who, who I knew from my time at Codecademy, he was an investor there, ended up investing, I think it was like 600k on a 6 million, valuation. But we couldn't, you know, scale the team too much on that. It was very difficult to raise. I didn't think we're going to be able to raise again, and I was trying to go for like some kind of bootstrap profitability. And we actually did, do significant revenue from, from education customers. And we weren't able to get into YC. So we weren't really able core, like Silicon Valley's circle.

Todd: So you applied to YC once and didn't get in and then later, later you did in 2018, right?

Amjad: I applied to YC three times and never got in. 

 And then Paul Graham, finds Replit on Hacker News. Hacker News loved us from the start. Now, now they kind of hate us. Like this is always the case, once you get a little bigger, you become the enemy. But, uh, we were like on there every other week. Paul Graham reads Hacker News, still to this day kind of religiously, although he doesn't comment. he found Replit and he asked Sam Altman to reach out to us. So one day, winter 2017, it was actually like a pretty rough period, uh, it was a little, a little hard at the time.

You know, just like the massive growth we're seeing and being able to support that meant that I was on call all the time, essentially, and we're such a small team or so three people. And I go and I meet Sam Altman at the OpenAI office, this shared kind of OpenAI Neuralink office in the Mission, and he's like, "You know, actually, Paul Graham found you and he told me to reach out to you because what you're building is something that he wanted to build and he started working on before YC took off and, Look, Paul Graham is, uh, retired now, and he lives, in a mansion in England. And you can only talk to him via email, so you should, uh, I'll make an introduction, you should, like, start chatting with him, but we're very interested, in potentially funding, funding Replit." And so I started this, Sort of email relationship with Paul that spend like multiple months and he would write me, long essays and maybe one day I'll make them into a book.

But he would write me long essays about how we thought about our space, how we thought about the cloud, how we thought about cloud software development, how we thought about how software development will get democratized and really saw eye to eye in all of this. And so, we still had to interview for YC and the team running YC had to accept us, but I was kind of in my feelings about the fact that they rejected us so much. So when we went and did, did the interview, so, so Sam asked me to to apply, and I was like, "Man, I did this form like three times. I hate it. And it's like, he asked us to record a thing, so I'm not going to record a thing."

So I, put in, the Rickroll video, Rick Astley's Never Going to Give You Up. And til this day, if you go to the YC portal and you click on Replit and you click the video or it should be the, founder interview, you'll get rickrolled. And like, it, uh, it comes up on twitter, uh, Every couple of months.

So actually, Michael got really pissed that he got Rickrolled and he almost rejected us. But then they were just like impressed with the idea and the team. And so they, they let us in.

Todd: Yeah, so it was part of that, you know, you mentioned the VC conversations and the YC applications, what was part of it because, you had this big vision and you had users, but your revenue at that time was coming from education and it's sort of like, how do we get from education to as much bigger audience?

Was that part of the, the question?

Amjad: Yeah. Like I've always thought about the, like the education market as a stepping stone. I even almost like felt. dirty about monetizing it because, I just want it replica to be universally accessible. I think it's a terrible business. and it's, it's unfortunate that it's a terrible business. you would want the free market to be building, tools, and, and software, To make it so that, kids all over the world, are learning better and, and all of that. but for all sorts of structural reasons and reasons around the, you know, government, control of, of education systems and all of that. I think it, becomes a really hard business, especially for a startup with no sales force and all of that. Some startups were able To To figure it out, but you can count them on on one hand, And so there are periods where I try to convince myself that we'll be able to build somewhat of a good business and education.

But for the most part, and again, going back to my first deck and that slide, it was it was never about education. It was about look, this is going to take years to build the full feature thing and it's going to take massive investment. But for us to not lose a ton of money in the process to be able to fund it, we're just going to sell it to people who were willing to buy it at the time.

Todd: I see. Okay. So it was kind of like part one in your master plan. Effectively.

Amjad: Yeah, which, now as an angel investor, I'm always skeptical of these master plans. 

Todd: Yeah. 

Amjad: I think, 

Todd: enough to get step one to work.

Amjad: yeah, but yeah, that was, that was the intention. 

Todd: Okay. So then take us through, you know, 2018, 2019, what was kind of the big next milestone and especially in terms of like, I think Replit's interesting because you had some amount of product market fit, almost immediately. But then I imagine that built and it's strengthened over time and it reached more people over time.

So what was kind of like that the next milestone that you think back to?

Amjad: breadth as in, growth, was never a focus. It sort of happened naturally. There were like periods where we invested in like SEO. I got really good at it early on. It's just, a fun thing. I mean, it's, it's kind of awful too. It's one of those things that I feel like is not really valuable economic activity that companies engage in. And it's just sort of a gaming of the system type thing. So again, you kind of feel dirty, kind of spending any time on it because you want to build useful products. you know, we've done some things like that, but for the most part. Growth was just this viral and word of mouth thing. the thing that I always focus on in terms of a milestone, both tangible and intangible are more of like depth because the biggest question, and maybe to this day is, the question on the business is like, will this ever be a daily driver for developers and, developers that are making money. and doing economically productive activities. And so that was always the focus. That was the North star for the team. How do you increase power? How do you, market it effectively and, and challenge the dogma that you need for you to be a real developer. You need to be in Vim all the time. Challenge the dogma around like cloud development sucking. Cause it sucked for a long time. make it so that programming is more fun, more collaborative, and so that we can get a lot more people engaged in it. so, you know, I'll give you one interesting milestone is, Again, an economically productive activity. So it's seeing people charging for software, making a startup on Replit or prototyping a startup on Replit A lot of them is sort of intangible, but this is where I felt like we're to be focused in order to create something that's not a toy. I mean, now in sort of VC, literature, if you want to call it that, a culture is like toys are not bad.

I mean, Paul Graham and Peter Thiel have made these, points and now they've become mainstream is that actually toys tend to be good businesses or things that look like toys. And by the way, it's, it's a, it's like a Clay Christensen. I'm sort of like a business strategy sort of nerd just for fun. I don't know if it affects my role. I mean, it's good as an investor, but I I don't think it's, that relevant as an operator, but, but I liked reading about it. So Clay Christensen wrote the innovators dilemma and in it, he talks about how. Disruptive technology starts from this bottom aspect of the market and as it becomes more powerful through some kind of trend or some kind of tailwinds it's, it's riding on or some kind of improvement in the product or the supply chain or what have you, uh, eventually will take over.

The classical example is the x86 microcomputer PC architecture. killing the mainframe on becoming the standard way of doing computing. Uh, and so that was sort of the lens I was taking and just trying to figure out where are we in that power curve? And are we actually getting to the point where people are, feeling empowered by using replit and not limited?

Todd: And so where did it, did it start to grow? And I, cause I know the pandemic was a big time of growth for you, but prior to that, did it start to grow in certain communities? Like, or it was like, you know, the Python community was more popular than JavaScript or vice versa. What were sort of like the early pockets of growth.

Amjad: in 20. 18, at the time for us, a behemoth sort of arrived on the scene. So, the guys at Fog Creek, Spolsky Joel Spolsky who's a legend in the programming community, had, um, spun out a project and they've been incredibly successful in these spinouts, such as Trello spun out a project called Glitch. And Glitch at the time got, their seed round was 30 million, and they hired, you know, right off the bat, tens of people, we got like 80 people, we were like four or five at the time or something like that. and they, you know, put a stake in the ground, they said, like, we think that the universal, Language of the future is JavaScript.

You can build full stack apps. Now we can do node and you can even write JavaScript in your database with Mongo. And so we're just going to build a short end to end JavaScript stack. and they captured people in their imagination. the bet we should make is we should be a general purpose platform and we should not be in the business of betting on certain languages we should support as many languages as, as, as we can, now, in retrospect, turns out to be a great bet because right around the time when.

Everyone was saying like JavaScript is the lingua franca, uh, Python started growing like crazy, and we started seeing it in the numbers. And we almost had the secret, that other people did not have because we look at the data and we're like, Oh my God, Python, Python, uh, ate into the CS one on one. it was no longer Java or Lisp. Now it's moved to Python. Then Python, grew, uh, on the back of, like, early web development, with, you know, Flask and things like that. although Node was challenging it there. And then, you know, that, you know, data science and AI ML revolution, uh, started really hitting mainstream. and so we continued down the path of building general purpose systems. I think we have the best multi language runtime. We built on top of this technology called Nix. We made a huge bet on, on that to be able to create runtimes on the fly, be able to create templates and have a really rich Platform to do that. I would say there's like a couple other bets like that to be able to, for us that either lucky or some foresight that we made that, that turned out to be, to be very good. But, at some point, you know, yeah, by the time the pandemic, you know, arrived, we're really 10 to a 100x more, users and, and engagement than the next competitor. 

Todd: Well, I think this is an interesting question, Amjad, because a lot of founders wonder about this. Like, how, how, how narrow do I go versus how wide? And you guys benefited a lot from being multi language pretty much from the beginning. And did you ever, like, why did you decide that? Did you ever think, hey, maybe we should just focus on JavaScript, or we should just focus on Python?

Amjad: Oh my God, it's, it was an agonizing decision because the Silicon Valley, I want to call it dogma is, do one thing and do it right. And you know, you can be, yes, it can be, you know, zero to one, peer to teal, uh, way of thinking about things, which I think is brilliant. it is also a misinterpretation of how Steve Jobs did, product design and management. Uh, I think people viewed Apple software as okay. Being very simple, but they forget that the fact that Apple built an entire stack in order to produce those pixels that are that seem very simple. and so simplicity does not, does not mean you shouldn't do.

You shouldn't have a powerful platform. Shouldn't do it to a lot. And now we have the example of, Chinese, uh, software, you know, a company like ByteDance, people will be surprised ByteDance now is a customer of Replit They use it for a platform that they, built called LarkBase, and LarkSuite.

You can describe it as G Suite. You can describe it as Workday. You can describe it as a hundred other kind of U. S. software. But what's interesting, and you see that now with TikTok, with TikTok adding shopping and live and all this other stuff like Chinese software tend to. Grow a lot faster and tend to be a lot more ambitious. And so it's sort of a challenge to the Silicon Valley model of, narrow. and maybe there's a historical, lesson here where. In the early innings of a platform shift, you can be narrow and you can, you can find a lot of product market fit, but towards when, when the thing matures say, you know, web or cloud, what have you, you actually have to be a lot more ambitious. And maybe that's true of technology in general. I mean, now the most successful startup in the world, Took multi-billion dollar investments to get to market, which is open AI. And so we might be in a different regime now than, previous epoch of Silicon Valley. 

Todd: So how did this, um, kind of play out over the years? You're, you're going after this very ambitious vision. I think you were starting to raise quite a bit of funding, over these years. And I'm curious, what the, uh, you know, how the, how the fundraising was for you now going into 2020, 2021, and sort of what your approach was to now we've got to grow the revenue side of this business, in addition to the usage.

Amjad: fundraising was never easy for Replit again, because it's such an ambitious plan and admittedly was almost somewhat, sparse on the details of, monetization. I wanted to innovate on, on monetization. one thing that I, was interested in is the creator economy. and the idea was in the air, you know, you had the crypto folks talk about, how, Chris Dixon just came out with this, his book, Read Write Own, and the idea that, users were screwed out of the, Wealth generation opportunity that the internet has created. one way to create, new platforms is to make sure that users have a stake in the platform. And, you know, crypto is one version of that, but I think the creator economy is another version of that. I think the most successful software creator economy is Roblox. so we started experimenting with these ideas. I think we started 22. we. Build kind of an in platform currency. We have this bounties platform that went into production, I think in 23. and in the first year you know, users on the platform earned a half a million dollars from all over the world, which is something we're very excited about. I'm not entirely sure it's going to be our primary source of revenue though. you know, it's still like programming and development is still niche.

I think by the time I think AI is still. AI capabilities need to go longer. it needs to get a lot more powerful for it to be a, like a really meaningful thing. It's, it's an area of continued investment, but I think where the business model might land is actually a lot more boring in a way, which has like SaaS slash cloud. where we're seeing the most growth now, we, so we have this core plan. The core plan gives you everything that you need in order to build an application. It gives you a database. It gives you a software development environment, a collaboration, gives you a hosting platform. It gives you a set of compute credits and storage credits in order to. not worry about it for a long time. And you can start building a startup and scale it. And we see people scale startups to hundreds of thousands of dollars of revenue as Indie hackers only on the core plan. we're starting to see companies start to rack up averages, on Replit we're kicking off a more of a sort of a team's enterprise plan and we already have early customers there. and so I think that's where, the majority of the revenue over the next few years will, come. I still think the creator economy for software could be really, really big. it's just going to take time and it's more of a, like a longer time bet that we can't bet the company on. 

Todd: Let's shift gears a little bit. And I know you've been thinking about AI for a long time, it's amazing to me that that was in, you know, your initial pitch deck so many years ago. When did that, when did it actually start to get real at Replit? Like, when did you guys first start incorporating AI into the product?

Amjad: GPT 1 had come out with very little fanfare, but there was, uh, there was some early experiments of people, making, progress on a GPT based, uh, autocomplete. So it was, it was still kind of a thing that I'm watching on the, on the side. I think GPT 2. is when we started actually writing code and experimenting with it. just the machine learning stack was so hard. The, you know, the ideas around fine tuning and ideas around training were just so inaccessible and so hard to do anything with. and so we never shipped anything there. We had some experiments that were kind of exciting, but. it was hard to put on production. it wasn't until, uh, GPT-3 that we we started building things. I think the first feature we launched in partnership with Open AI was in 2021, uh, was explain code, and I think it was like one of the first, coding applications of GPT that went into, production. And then we started building, what became Ghostwriter. Now we had a we had a major roadblock So Ghostwriter is our co-pilot equivalent. it's different in ways we can, we can discuss it, but the core features, the autocomplete feature is similar to, to Copilot. and the major roadblock there was latency. To be, you know, GPT 3, even now GPT 4, GPT 3. 5, GPT 3. 5 has gotten a lot faster, but, at the time it was really slow and it was really expensive. And I had, again, I had this vision that this is a universal feature, like it's not, you'd want to think of the programming environment as AI native and not as an add on. I described the current era we're living in, in terms of AI add ons, where, you get Windows and you get Copilot for 10 bucks, you get Notion and you get Notion AI for 10 bucks or whatever. I describe this as I'm sort of a 90s kid. And so, started using DOS and if you remember, you would launch windows from DOS as an add on, you would, you know, pay Microsoft 50 bucks to get the windows add on. So GUI was an add on at some point. And I think this is where we are with AI, but I don't, I think it's a transitionary period. I think at most couple of years. where most companies are going to take the approach we're taking. So we actually deprecated the name Ghostwriter and we just call AI. We have everyone in the company building AI. We have an AI team that builds an internal platform and then everyone thinks about every feature as from the perspective of how AI could make this a lot more, streamlined and, automated. And so with that vision in mind, you know, the major roadblock was, cost and latency. You know, we were later than I would have liked because we were doing a lot of R& D internally, trying to figure out how to run smaller models. And we had a some of the breakthrough in June 22, where we were the first to put an open source model in production. It was the Salesforce code gen model. Salesforce has an AI research team and they trained a model that was pretty good. It was really slow. We did a lot of work internally, rewriting large aspect of the code base to make it faster. And we went into production with that and we made a lot of noise because again, it was, I think the first major open source powered product release. It was the first time someone challenged the idea that Copilot is a thing that only OpenAI and Microsoft can, can, can build. Uh, and the idea that you need billions of dollars to do that.

So I think we unlocked a lot of, people's imagination that this, space could actually be accessible for startups. By April 23, we had released our first model that beat Salesforce CodeGen, became a state of the art open source code model, and released an updated version of that in October last year. 

Todd: You know, I'm very curious how you, and I think you think about this a lot, the role of software engineers, the increasing number of people that can write software, how does that change do you think over 3 years, 5 years, 10 years kind of time frame?

Amjad: I think that there's going to be a bifurcation of software engineer and software creator. Engineers are not going away. we need engineers, in the same way that Instagram did not obviate the need for professional, uh, you know, journalists and photographers and all of that. Technology can make things more accessible but most of the time it creates or pushes a category that was already emerging to become a mainstream category, but it typically doesn't entirely disrupt the existing way of doing things. So from the outset, I don't think engineers are going away, at least not on the timeline that doesn't involve getting AGI. And they're going to be AI powered as well, but I think that where AI could make a massive impact on software is the people that are building products. Fred Brooks, wrote this, uh, book called, um, The Mythical Man Month, uh, it was based on his experience building, I think, OS two at IBM, uh, I might be getting the names wrong, but, it was like, the first major failure as a software project caused this huge panic in the software industry that actually like, we don't know how to make large scale software. So he wrote this entire book about how software engineering is, needs to be thought of as a new discipline. And the idea that you can throw a person at a project and reduce the number of months it takes to ship that project, the more people you add, the more complexity you add, there's no way to measure productivity. There's an essay in there about accidental complexity. Most of the things that take time in the individual engineers, individual software engineer work is accidental complexity, meaning the tools are actually working against us. And the way we think about it at Replit is, is, the reason, there isn't as many software entrepreneurs, software startups, software creators is because most of the time the tools are in your way. So if we try to abstract and solve a lot of this accidental complexity, we're gonna be able to unlock an entirely new type of software creator that can start startups, that can, do things, decreasing intricacies of coding over time. So that creates a new, and you discipline, I think. And the software engineer, on the other hand, which we also want to support, but you know, their role, if I, you know, go to the future five years in the future, they're still going to have some kind of code editor and they're going to still have AI all over the place.

They're going to have AI chat, GPT. They're going to be talking to different AIs. They're going to be debugging, but they're still going to be typing some amount of code. They're still going to be debugging some amount of amount of code. And they're still going to be generating totally novel code that GPTs have never seen.

And again, GPTs are a function of the data. So there's a lot of data locked up in enterprises that are not really going to make it into the general purpose GPTs that, you know, so there's, this problem with it. Whereas for for the product engineer or the software creator, that I'm calling them, it's, most of what they do is kind of repetitive. And so with AI agents and all of that coming down the pipeline, they're really going to be superhuman in their ability to be able to deliver customer value.

Todd: The software creator in your, I like the terminology, by the way, software engineer, software creator. They still have to be able to read the code and correct the code, right? I mean, a lot of it's generated for them, but they still have to be fluent. Is that right?

Amjad: I think yes, over the next two or three years, but I think, you know, it's a decreasing thing over time. We have now entrepreneurs on Replit generating hundreds of thousands of dollars and, from their startups, that have started on Replit, did a hundred days of code and sometimes got to day 45 and felt like they were proficient enough to start building something. And so now you can go on for the order of weeks to be able to make money, make it money equivalent to a salary. And that will just go to compress over time. And I think it compresses towards zero. When do we reach zero? I don't know, but like, take debugging. Like there's going to be a lot of research and development on AI agents that do debugging as good as people, especially again for, a specific type of developer that is building mostly features as opposed to building platforms or distributed systems. 

Todd: Okay. So because we are shrinking, compressing the time that it takes not just to get proficient as a software engineer, but actually like profitable as a software engineer, it's just going to increase the number of people that, do it and can get good at it quickly. 

Amjad: You know, when we, uh, when I first met Paul Graham, he said, there's a super linear relationship with how much easier you make software creation, and how many people we attract. And remember, Viaweb, his startup, was a way of generating, online stores. That's actually still in use today, which is fascinating.

Paul Graham's site is still hosted on it. and it's run by some like, private equity that, you know, bought it from another private equity, from another private equity firm. And, um, because they made it easier to generate a store with little coding, you could like type a little less than HTML, but, they expanded the market for how many people would want to do online commerce and be entrepreneurs on the, on the internet. And I think that realization is true that every increment of ease of use, you get a lot more double or 1. 5 that and the amount of people that, would be interested in doing that.

Todd: I mean, generative AI is moving, I think, so quickly. And you guys are really, uh, at the center of it in many ways. How do you keep up with, what's going on and the changes that are happening every day, every week? And how do you, how are you using it kind of in your own day to day? 

Amjad: 22 going on 23, I was consumed by studying it. I was reading every paper coming out and it was. It was a period when there was an explosion of, papers talking about the fundamental aspects of LLMs and how to program them and fine tune them, how to think about them.

There was a lot of mysticism around LLMs. Some people thought it was AGI at the time. Some people thought it was like, just can generalize beyond its training data and all of that. Know, a lot of this, we just understand LLMs a lot better today. LLMs are really a function of their data. They're not really that mysterious.

I mean, the most reductive way to think about an LLM, which is still true, is a compression engine. Being able to explain something and removing the mystery of it does not mean it's not powerful. Lot of people read my views on this now as, as kind of being bearish or something like that. No, I'm incredibly bullish, but like, I understand it better now. There's nothing really magical about it, like the scaling laws and all of that. I think these models will continue to get more and more powerful, but they're bottlenecked by their data and the techniques to make them useful for downstream tasks, which is, fine tuning and RAG. And I'm sure we'll discover more techniques in, in the future. This is, I think where, I sort of got a lot of education and I talked to a ton of people and I, you know, use Twitter to kind of test my ideas and, and then it just became about execution, especially after Razor last round, kind of hiring and, was a little removed from that, but honestly, I don't feel like I'm missing a lot.

And here's the reason, and it's sort of a sad reason. A lot of the labs have sort of close down how much they publish and you might have noticed that there isn't as much as many groundbreaking papers. A lot of the papers are rehashing things. A lot of the papers are fake. There's been a few, few, few of them that are kind of like, you know, not really presenting, it's like a, you know, publish or perish type, type thing that's happened in academia. There's going to be more capabilities down in the the pipeline, but there's a lot more secrecy now. How do I keep up? I think Twitter is still the major thing for me.

Some amount of Hacker News, I'll read something every now and then. I'll talk to our team, I'll talk to, through my angel investing, I learned a ton about how you can apply these, these capabilities, but it's sort of slowed down a ton as well. So in terms of, how do I use it? the way I use GPT 3 circa 21 is still my preferred way of using it, which is the old school style of prompting.

I recently was able to diagnose myself, with an issue that perplexed me and my doctors for two and three years. I'm sort of not ready to talk about that yet. Maybe I'll talk about it publicly at some point, but I had some chronic issue that was, send me down all sorts of different paths and was, you know, it was kind of annoying and wasted a lot of time and money.

And then recently, uh, I went back and actually just before OpenAI deprecated, DaVinci 3, I think, and I used the old style of prompting. And now I go to Mistral Medium or some of the, I like the completion models. Like the old style prompting is more like programming. Chat models are made for, The lowest common denominator is made for the end user, for the consumer.

I still like, you know, have some kind of LM platform kind of open next to me and try to get answers, try to ideate and brainstorm and things like that. But the one consumer product that I use is Perplexity. They've done amazing job. I'm almost always surprised how they arrive at the results that in my Google searches, I would never be able to arrive at. So one, one question could save me 10, 20 minutes. And sometimes I would not have even arrived at the answer. So Perplexity is a daily, daily usage. I'm sort of somewhat disappointed about how slow these things have been able to roll out, especially in the devices that we depend on every day.

I'm really looking forward for Apple and Google to start rolling it out and Gmail and iPhone and all of that. I'd be curious if you have used, you know, AI in a way, obviously I use AI coding with a Replit, but if you've used AI in a sort of a novel way.

Todd: I'm probably not terribly novel. I mean, I, I have used Perplexity. I actually am a pretty big fan. I used GPT 4 obviously, but I'm a pretty big fan of BARD. I find that for a lot of the like VC style research stuff I do, where it's kind of like a combination of what's in the model and stuff it's going to find on the web, you know, to compare different companies and earnings and stuff like that.

I've really gotten pretty into BARD. I'm almost, I'm probably a DAU of BARD.

Amjad: Well, I need to, use it again after the Gemini, Gemini upgrade. I haven't really checked it out, but it's really cool that the index is integrated there. Like, it's not actually going and doing a web API. It's obviously much faster than that, which I think makes it really cool. 

Todd: Zooming out, Amjad, you know, 2024, big year for Replit obviously. What's, uh, what's kind of on your mind? What are the top two or three things you're thinking about, for Replit now?

Amjad: Scaling revenue and just like making sure the core business, uh, works really well and is, is repeatable and we, we understand the growth levers and all of that. Boring a little bit, but uh, I think 2024 is going to be the commercial sort of year, year for Replit. I'm really excited about that. I actually, I think a lot of technical entrepreneurs, like, do not like that aspect of the job, but I'm, I really like sales and I really like talking to customers, I really like figuring out how to make their lives easier and all of that. We also have some really big AI goals. I can talk about all of them, but we have like a very small research component at Raplet. We built this AI platform that we're using internally that I think is really, really powerful. The reason I say I wish we would be able to move faster is that, turns out that in my opinion, training models, especially smaller models, is easier than figuring out how to leverage the large language model capabilities correctly. And I think this is where things you know, Bard and Perplexity have done a great job. So we built this really large Infrastructure that does a lot of really cool things that we haven't leveraged a lot in the product and you're going to be seeing them coming out. Can the AI be able to interact with the development environment as if it's a human? Can you start to think about the AI as a collaborator? Can it just go and execute code? Can it call APIs? Can it deploy on your behalf? Can it do background tasks? Or you can ask it for like a high level idea and it can go and like, do it in the, in the background. And so that's, that's going to be, a big area. I think towards the end of the year, we'll probably have some things to share about, fully agentic software creation.

That's going to be exciting. 

Todd: So Amjad, this has been great. You know, I wanted to wrap up with a couple of questions about you, uh, that I know people are interested in hearing.

You know, you've got such an interesting founding story, and I think it speaks volumes about perseverance and kind of what it takes to start a company. What is advice? What is some advice that you give to founders these days about how to make their company successful?

Amjad: You know, one of the things that I think we'd lost a little bit of sight over, uh, during the SaaS boom period and Silicon Valley, which I think is coming to an end, is the idea of like, tinkering and the idea of, following your passion and interest without an immediate potential for commercialization, and for VCs to be able to fund that.

And there's a lot of exciting models out there such as the, they call them the like pre idea, models such as, uh, South Park commons and, and others, I think will fund you just as a person, although it will get some option to invest in the, in the company if you, if you arrive at something. But I, I think whenever there's a new, capability that's, uh, arriving in the scene or new platform, you just have to do a lot of experimentation and a lot of this stuff looks like mad science-y stuff. You know, Replit was a sort of a mad science project. Like, can you run Python and and Ruby in the browser? Right. That's like kind of crazy idea. You know, with, with AI, I think circa again, 21, 22, we did a lot of mad science stuff. Actually, one of the first commits to LangChain was referencing something I implemented, which is how to call Python from, GPT 3. There was a lot of, it was a lot of fun. A lot of hackers on Twitter were like figuring out how to do prompting and how to do, like, we mapped out the space of, you know, tool usage and things like that that are now becoming more standardized. But I still think there's a lot more. There's a lot more experimentation and pure tinkering and hacking that could generate a lot of non obvious, uh, startups, because I think a lot of the obvious stuff is sort of mapped out, but I think there's, there's more experimentation. So my, my advice for founders entering the space is like, especially if you're a hacker. Just like you think about all the cool projects to do and give yourself enough time, especially if you're in your university or, finishing up school or something like that. Just don't enter this with a mindset of I'm a founder or entrepreneur, I'm going to start a company. It's more like what is really interesting and cool to apply this technology towards?

Todd: But how do I figure out if something is just a cool thing that I like tinkering with and that I'm interested, I follow my interests versus something that could actually be, you know, an interesting startup.

Amjad: Yeah, I mean, the short answer is you can't know until you actually try it, right? But you'll have some inkling that this could be useful. When Zuck started Facebook, Facebook was a side project. I think their main project was called Wirehog. So it's like, Oh, file sharing is a real business idea. This Facebook thing is a fun idea, right? And so you never know. I think embracing that uncertainty a little more and going back to the earlier era of Silicon Valley, I think could be, could be exciting. 

Todd: any favorite books or resources that you love or that you'd recommend for folks?

Amjad: One of my favorite books, I don't think it kind of relates to a lot of the things that we talk about, except maybe AI, is, I Am a Strange Loop by uh, Douglas Hofstadter, a lot of people know Hofstadter from GEB, Gerd Achenbach. And he's been like an AI, consciousness, theorist. The interesting thing about I Am A Strange Loop is that it weaves this personal story of like, losing his wife and with his theories about consciousness with ideas from, from math and, arrives as a sort of a novel way of thinking about the soul or consciousness and, and things like that. And I think increasingly those questions are going to become important as AI gets more and more powerful and captures people's imagination. So that's on the, non non technical non business aspect. Business strategy, I think is super interesting. I think you can like potentially take it too far and become more of a theorist. I talked about Clay Christensen's Innovator's Dilemma, which I think is still a required reading, but, you have, more recent things like the Seven Powers by Hamilton Helmer, and actually I've, I've become friends with Hamilton, and he invested in Replit, uh, but you know, we talk a lot about Moats in Silicon Valley and it's all hand wavy.

You hand wave, you say network effects and hand wave. Hamilton takes a more academic theoretical approach to like actually figuring out what moats and he calls them powers and he identified seven categories of them. I think that that book is super interesting. Maybe a programming book is a really fun way that kind of inspired how I built Replit is, The Little Schemer, uh, which is, formatted in a way where you have exercise on the left and you have empty space where you can like write out the list code. I did it on the computer, but, it sort of builds up from very simple primitives, like it goes from the null list. And then towards the end, you implement the Y combinator, the Y combinator, not this, the accelerator, the Y combinator is a list function that allows you to, do recursive calls on anonymous functions.

The reason Paul Graham picked the Y combinator as the name is because it's the function of all functions, right? So YC is the startup of all startups. Uh, and so The Little Schemer and then, it is actually multiple books, it builds up, but, uh, it's one of the most fun I've had, learning a new language.

So, so that's interesting. 

Todd: Very cool. Amjad, thank you so much. This was awesome and congrats on all the success so far with Replit and I'm sure for years to come. 

Amjad: Thank you.

Really appreciate it, Todd. Thank you. This is really fun.