“One of the most scalable organizations in human history was the Roman army. Its defining unit: The squad — eight guys. The number of guys that could fit in a tent.”
This is Twitter Engineering SVP Chris Fry on history’s greatest example of successful hyper-growth, and the tactic that made it possible: stable team building.
The moment your organization hits 40 engineers, this should be the number one thing on your mind, he says. When you’re small, everyone sits in one room and everything works. As you grow, things start to break. “That’s when you need to take all your engineering knowledge and apply it to your team, and you want to end up with a modular system where teams become the unit of scale,” Fry says. Getting bigger doesn’t have to mean getting slower.
After leading engineering at both Salesforce and Twitter during their hyper-growth stages, Fry has seen how team structure can make the difference between zero products shipped and fluid productivity. “With the team structure the Roman legion created, they were able to dominate 500 years of history and create a distributed network that ruled the Western world. If you can create a similar structure, you can rule the world too.”
He shared his tips for building this structure at First Round’s last CTO Summit.
With a PhD in cognitive science, Fry specializes in thinking through how pieces of any given system can interact for ultimate efficiency. When it comes to building a deeply efficient engineering organization, there are several things you can do to move the needle:
- Build strong teams first. Assign them problems later.
- Go modular. Remove dependencies.
- Establish a short, regular ship cycle.
- Keep teams together.
What Stable Teams Look Like
Fry is big on the importance of stability. In tech’s culture of growth and failing fast, it’s a quality that is too often sacrificed for the sake of speed. But stability isn’t just about consistency — it’s about resilience, distributed knowledge, optimal performance.
When Fry landed at Twitter, he immediately highlighted a fundamental scale issue: It’s very hard to predict events around the world. Something like the Super Bowl can spike tweets per second by millions. Last year, the company saw a massive increase out of the blue, only to realize that a popular movie had aired in Japan, urging everyone to tweet at the same time. When this is the case, it’s nearly impossible to prepare. But, increasingly, Twitter has survived even the most unexpected events —a change Fry attributes to organizational changes as it grew.
When most tech companies are just starting out — and Twitter is no exception — everyone does a little bit of everything, and almost all actions are reactive. Projects pop up, and you have to quickly rally a team around them so that things don’t fall apart. This gets you into the habit of staffing projects fast for short periods of time. This is fine at first, but it’s a habit that eventually needs to be broken.
At a certain size, two things become clear: 1) Many people are working on too many teams at once and get stretched too thin, and 2) as the leader assigning people to teams, you’ve become a huge bottleneck of information. You start accruing technical debt because there’s no time to slow down and fix all the little cracks in your products. At the same time, you lose your ability to innovate.
“The only way to survive this transition is to create stable teams,” says Fry. “In the U.S., we have this cult of the individual, but I encourage leaders to think about the value of strong units.”
When people work well together, their output is unimaginably better.
“As a manager, your focus should really be creating these high-performing squads of people who have good chemistry, who can get things done. Then you figure out how to apply them to problems.”
Fry has observed a number of managers simply plug names into a spreadsheet to staff projects as they arise. It’s faster and it beats disorganization. But this inevitably leads to a lot of reporting overhead as you shuffle people from manager to manager, and a lot of time wasted as people “re-norm” to new circumstances. There’s no time to learn how to excel together as a team. Considering this top-down approach has been the standard since the 1960s, it’s hard to blame new managers. It simply needs to change, says Fry.
The High-Performance Zone
“Imagine you need to solve a really hard problem, but every month you’re thrown into a room with a bunch of people you don’t really know, and you have to work together on this thing,” Fry says. “You would do a much better job if you knew how these people worked, if you had worked with them before, if you knew their strengths and capabilities and trusted them.”
This is one of the key reasons to keep teams together. Trust is a much bigger deal than most people think in this context. But trust breeds healthy communication, and that’s ultimately what boosts you into that high-performance zone where teams tackle problems with all cylinders firing.
“If you can get to this point, you will have reached Nirvana as a leader,” says Fry. “You’ll be able to hand off problems as they come, and your teams will come back with solutions devised and executed on their own.”
You want teams that not only work well together, but that learn from each other.
To achieve this level of flow, Fry recommends keeping teams together for at least six months, and ideally a year or more. This might be impossible at a brand new startup, but once you’ve gotten to 40 engineers, it becomes critically important that people work with others who optimize and extend their capabilities. Then, once you have this formula, you don’t want to mess with it.
A big part of building long-lasting teams is protecting your people. Too often, Fry says he sees startups failing to treat their employees as well as their hardware. “Would you run your data center at 100% utilization all the time? No. Not even 80 or 90%. But some companies expect this level of performance from their people.” This is a one-way street to burnout and attrition, and the losses that come with them.
Instead of looking at how hard teams are working, you should focus solely on how much they are able to produce, Fry says. Teams will never perform optimally if one person is doing all the work while everyone else sits around. Same thing if a hardworking team is stymied by one non-performer. Ensuring that everyone is working evenly and sustainably is the best way to get the most done.
This is one of several tactics Fry regularly deploys to build strong teams. The others include:
- Getting teams to sit together: “It’s one of the most powerful things you can do to fix issues when things aren’t working.” This is obviously harder when you have a large, global team, but when you’re just entering your growth-phase, it’s critical.
- Making someone responsible: Anoint a great, communicative leader who will be responsible for the output of any given team. Maybe it’s the engineering manager, maybe it’s someone else. But you have to have a point person who is encouraging everyone else to perform and communicating progress to the broader organization.
- Clarifying mission: Every member of every team should be able to tell you immediately what their group’s goals are and why they are important. It’s the only way to fuel momentum.
- Knowing your role: If teams at your company are performing well, your only role as a top executive should be to make sure they have all the resources they need to deliver on their mission. “This is often the hardest thing you face as a leader, but it’s also the most important,” Fry says. A lot of startup founders have a hard time letting go of the details, but to get the most out of your teams, you often have to accept a supporting role.
Establishing and respecting the sanctity of teams gives you several advantages. They become self-healing and self-correcting organisms. When a stable team encounters a glitch or misses a goal, it regroups, knows how to run a tight post-mortem and adjusts for the future. At the same time, you give people a chance to distribute important information.
“You see it all the time where one person leaves a team and takes all this institutional knowledge with them,” says Fry. “The goal should be for people to be able to move in and out without anything really changing.” This type of resilience only comes from people working closely together long enough to achieve mindshare.
Think Modular
Modularity is the key to scale.
When strong teams (instead of individuals) become the basic unit in your organization, you can build faster, distribute authority, and still do astonishingly original things, says Fry.
The first rule of thumb: Good modules are cross-functional. Fry saw this as Twitter developed its mobile strategy from its earliest days. At first, the mobile team was a small band of engineers formed simply to get the work done as fast as possible. It was split into two factions, the Android developers and the iOS developers, and every feature the company wanted to build flowed through them like a funnel.
The problem was that building high-quality features required a lot of external support: Great design, thoughtful product management, strategic marketing — not just talented engineering. With the initial system, scaling fast while maintaining this high bar would have been impossible.
To fix the problem, the first thing Fry’s team did was train its own legion of engineers how to be great at mobile across platforms. Twitter even purchased a company that specialized in mobile training to bring this expertise in-house.
“Training people so that they aren’t hyper-specialized is one of the best ways to prevent bottlenecking,” says Fry. “That way, you never have to wait on an answer from that one guy who knows how to do Android. You can do so much more on your own.”
Once the company had this highly-skilled mobile force, it distributed these engineers among feature-based teams, each comprised of a product specialist, a designer, and others needed to make that feature a success. This gave each engineer control over their own work and made them a central pillar of a strong team at the same time. As a result, Twitter was able to continually ship a number of high-quality mobile features. And because the engineers knew each other — and were able to keep lines of communication open across teams — features that made it to launch were more cohesive.
This modular structure naturally eliminates dependencies. When people have broader skill sets, they don’t have to depend on other people’s knowledge. And when designers and product managers have instant access to an engineer focused solely on their product, they don’t have to continually ask the one monolithic mobile engineering team how things are going.
“Now I can just turn to the guy who runs search and ask, ‘Hey, are you going to deliver on that one feature?’ and he can tell me right away because he knows everything that’s going on,” says Fry. “One of the biggest challenges for software organizations is keeping production high while also giving people control over their own destinies. When you remove dependency, you give people this control.”
Quick to Ship
When Fry first joined Salesforce, the company was more focused on testing product than shipping it. To shake things up, his first order of business was to restructure the engineering organization — at that time about 40 strong — into small, nimble teams. He also added more milestones and shortened testing phases. Eventually, the company was able to release data-driven products every four months while also growing to over 600 engineers.
He arrived at Twitter two years ago, eager to jump back into rapid growth. “That’s where the fun and the challenge is,” he says. “When you’re on a rapid growth curve, you’re challenged every day by problems you’ve never faced before.” Again, he was charged with scaling technology and human capital at the same time. “To do this, I really had to think about what I wanted to hold constant and what I would need to change to help the team grow.”
Soon, he instilled the same quick-ship mentality that worked at Salesforce by establishing a short but regular cycle of delivery.
You always want the shortest development cycle.
“A lot of teams do this naturally, shipping new releases every week, especially at the beginning,” says Fry. “But as you grow, it gets harder and harder to keep everyone on the same timeline and deliver something cohesive. Speed of iteration slows down when you need it to keep going up.”
This is yet another area where the team-based approach to organization wins. When teams are your basic building block, it’s much easier to enforce cycles across the board. You don’t have to worry about individuals staggering their work, and — especially when teams can manage themselves — leads can make sure everyone is hitting their marks, and deadlines are clearly communicated.
“By dictating shorter cycles, we were able to get all of Twitter shipping mobile at a high rate and at high-quality,” says Fry. “It all comes back to thinking about teams as the primary unit of scale.”
Chris Fry joined Twitter in 2012. Before that, he led development at Salesforce, and deployment of scalable web services at Weblogic. You can find him at @chfry.