When First Round redesigned its annual summit of technical leaders, it had a goal in mind: participants leaving the event with notebooks chock-full of new tactics to try. So it opted for an unconference format, one that maximizes learning touchpoints by getting very smart people together in multi-dimensional discussions, rather than an audience of many receiving one point of view from a stage.
To that end, the CTO Unconference downplayed keynotes and prized key nodes, including the CTOs, VPEs and Engineering Directors from companies like Apple, Google, Dropbox, Slack and Blue Apron.
Here’s more of a snapshot of the 160+ attendees:
- 40% of attendees were CTOs, 34% were VPEs and Directors of Engineering and 26% were rising-star engineering leads, manager, CIOs and CPOs
- 31% were from consumer companies, including Twitter, Facebook, Airbnb, TaskRabbit and Pandora
- 50% hailed from enterprise companies, including Kentik, Cloudera, Zenefits, Box, GoDaddy and Appnexus.
- Three out of 10 attendees were women.
- Participants represented a range of verticals: from healthcare (Flatiron) to hardware (eero, Nest), and from fintech (LendingHome, Coinbase) to non-profit (OpenAI, Hillary for America).
Throughout the course of the day, we overheard some wise observations and tested tactics from these impressive technical leaders. No sessions were recorded or live-streamed, so attendees really opened up to each other about their victories and vulnerabilities. What emerged is advice you'd seldom hear elsewhere — guidance that we thought could help other company builders out there.
We learned a lot from attendees and also collected some fun facts about them to break the ice. So in honor of the one person who listed Cool Runnings as a favorite movie, we hope the following advice mined from our Unconference helps you “feel the rhythm” and “feel the rhyme” in your companies.
Cause it’s bobsled time.
We hope you enjoy,
Your First Round Review Editors
*****
ON HIRING AND INTERVIEWING
When hiring candidates, ask for their operating manual. Tell candidates: “Imagine you're a robot. What does your manual say under 'ideal operating conditions.'” Once they answer, follow-up with this question: “What does the 'warning label' say?” You're likely to get insightful, unpredictable and humorous answers in this very low-lift way of gauging self-awareness and revealing personality.
Test candidates' ability to observe. Interviews index highly on engineers' capacity to execute. But there's often not an easy way to test how they absorb and apply information — especially in group sessions or activities. One technical leader offered this exercise: “We have a game that our office likes to play. It's a machine — kinda like a pinball game. After candidates go through their interview schedule, we ask them to play the game with us. They usually think it's about culture fit. But here's the catch: We have them watch the game first. And then instead of telling them the rules, we ask them to tell us the rules. We're testing to see if they can extract how to play the game from observation.”
Institute role plays in your interview process. For every engineering manager role, have the candidate sit with a member of the engineering team and play out a scenario 1:1. It can be about a technical process, an argument about prioritization or giving feedback with both criticism and praise. It's an effective way to test softer skills and replicate what you'll get in a 'real' situation. Of course, using your engineers' time like this may seem expensive, but it's more costly to bring on an engineering leader who doesn't jive with your team. Plus, after doing it for a few years, you'll find it becomes a rite of passage and engineers like participating in them.
If you're thinking about how to debug people in the interview, don't hire them. It sounds obvious. But a number of technical leaders said they regret 'debugging' people by figuring out workarounds to make a talented person fit. You'll get people who are good performers, but assholes. And you think you can get them to not disrupt the team and not be a jerk. It won't happen. You'll invest a lot of time and energy, only to fire them. Have a strict no asshole rule.
The best recruiting tool recommendations include the following, from people who have tried many solutions before any of them stuck:
- Greenhouse.io is by far the favorite pick for applicant tracking systems. It includes good APIs for reporting, integrates with many online coding tests, and offers rubrics for standardizing your hiring process. According to a few people, it's possible to negotiate good deals with the company as well.
- Asana can function as an applicant tracking system if you don't have the money for a dedicated solution in your early days. You can set it up to move candidates through a defined flow, collect feedback from everyone on the hiring loop, and follow up accordingly.
- Triplebyte has garnered a following. It's a service that helps candidates take coding quizzes that qualify them to connect with a number of jobs. As an employer, there's an advantage, because you only have to pay per hire you actually make. People also reported liking Vettery, which is a new entrant to the space and seems to yield high-quality candidates so far.
- Piazza Careers is the chosen solution for technical leaders recruiting on college campuses.
ON MANAGEMENT
Assign new employees an early win. Consider giving new recruits a small project they can accomplish during their first month on the job. Have them all spend five minutes presenting whatever it is they did at the next All Hands following their first day. It's an opportunity to make immediate impact and introduce themselves to everyone in a positive light. It'll also build camaraderie between folks who started within the same span of time.
When spinning people up, prioritize production over perfection. There's a great anecdote in the book Art & Fear: Observations On the Perils (and Rewards) of Artmaking by David Bayles. A teacher split a potter class into two. Half the class was tasked with making the single best piece of pottery. The other half of the class was tasked with producing the most pieces of pottery. Then they were judged. All the best individual pieces of pottery came from the half of the class tasked with producing the most pottery, not the best pottery.
Get out of the way of your most prolific engineers. It's better to route that energy than to squash it because it's not exactly on the goal. If you do, it'll just flow another direction — likely out of the company. Let them be prolific and you, dear technical leader, be filter. Don't ask them to be that. Prolific engineers give their peers something to aspire to. If you box them in, they'll just leave and start their own company.
Debate, decide and persuade. Try this simple goal- and expectation-setting framework: on any one topic, it's important for the team to understand if a leader is asking her team to debate (early stages of discussion), to decide (middle stages of discussion) or to persuade (later stages of discussion). Orienting a team around one of these three acts at the onset helps position the challenge and narrow the solutions to discuss.
Aim for the crest of the rainbow. With work, you have a rainbow of disgust. On one end of the rainbow is disgust and the other side is pride. You want to be somewhere in the middle where you're not proud of the code, but it gets the job done. Most of the successful projects is like chasing a 3-year old down a slope. They're a little out of control, but not horrible. They're moving faster than you're comfortable, with but still in reach.
Foster psychological safety. At the beginning of every weekly sync, have your team share something personal and professional. It can be a win or a challenge. This doesn't scale for very large teams, but if it works for smaller ones. Getting quick hits of these types of updates can make the difference between taking someone's bad mood personally to being able to rationalize and separate it from others' statements. Establishing empathy upfront saves time and energy that is better spent on the technical tasks at hand.
Make sure your post mortems don't miss the point. Too often, companies use post mortems incorrectly as a tool. First, they have them for everything as a rote part of process, even when they aren't going to be productive or yield constructive actions to take in the future. Second, they often focus far too much on what happened, and not enough on how issues can be prevented in the future. Make sure you leave every post mortem session with a couple concise ideas on how to identify the biggest issues in the future and what actions should be triggered once they're spotted.
ON MEASURING PERFORMANCE
Measure engineer happiness as a performance indicator. Job satisfaction is an important and useful leading indicator of productivity. A lot of companies use CultureAmp in particular to keep tabs on how engineers are feeling about their work. Another tactic is to run an anonymous survey (using a tool like Glint) that asks employees how willing they are to go above and beyond, whether they see themselves staying at the company for more than a couple years, and whether they'd recommend working at the company to their technical friends (a type of employee NPS score). Benchmark the results you get every six months to make sure you're maintaining or getting better.
Record how often you move the goal posts. Consider creating a dashboard of all your P1 priority projects. Make it clear how they're ranked, what their deadlines are, and who owns them. Also document how often you have to push things back due to various changes or conditions. Some team leads send out a weekly email summarizing the status of these 'anchor features.' How often are you moving deadlines back on these features due to performance-related issues? When you notice this happen more than once, you should think about the best way to intervene.
Track your mean time to repair outages. If you run a service where outages are fairly frequent and mission critical, start measuring how long it takes your engineering teams to get things back up and running. Create a metric for yourself called the MTTR (mean time to repair) and make sure that it's either staying flat or going down as your team learns more and gets better at responding.
Track how often your engineers surprise you. Most managers monitor milestones, such as if a product shipped on time. But also keep track of how often your people surprise you — in both a good and bad way. For example, when was the last time one of your engineers came up with a solution that was novel to you? Or have they missed a deadline without giving you a heads up? Both these moments are easily-tracked data points on communication — good and bad — that many managers leave on the table. Keep a tally with a few words of context. It'll be fodder to celebrate or coach your people — and indication you're paying attention.
Develop metrics to track code engagement and literacy. Ideally, you want your team to be very engaged on a regular basis with your plans and code. To keep tabs on this, measure how much test coverage you have, how many people on your team are reviewing code for others? Commenting on code or design documents? As a leader, you definitely don't want to get stuck being the only reliable code reviewer. Incentivize others to learn and do this by tracking and regularly sharing these metrics.
ON DATA
You have to make tradeoffs to get real-time data. If you want to centralize your data, it can't be real-time. And you can't maintain real-time data pipelines for everything. You have to be discerning and choose metrics you'll really benefit from tracking moment to moment. Most real-time data is too expensive and not worth it.
Don't let everyone dip into your data source. There's a dangerous trend afoot: companies will have a single, clean source of data with full instrumentation, but then different internal groups ranging from customer service to marketing will want to use their own SaaS tools to evaluate it. This can get messy. One way to solve this is to pipe all data into Redshift, and build an API layer that others can use to query it. Make it your data engineering team's prerogative to decide what data gets mined and stored. Don't let other groups determine the scope of your collection.
Favorite data tools varied depending on the needs of the company. But here are a few that people said worked well for them:
- Druid is great for storing vast amounts of real-time data, allowing you to dig into event data immediately after they take place.
- Kafka has been successful at building real-time data pipelines and is comparatively low-maintenance. As people say, it just works.
- Bigquery and Snowflake are recommended warehouses to increase capacity for analytical data. They allow you to put many machines against a problem. Snowflake in particular gets a lot of praise for being well-built and reliable without as much partitioning.
ON PLANNING
Harness the dark art of estimating project scope. Being able to estimate the time needed to build a feature or product is an extremely rare and inconsistent skill. In particular, junior engineers tend to underestimate or overestimate scope by a consistent multiple of time — either 2x or 3x. Keep tabs on what they quote to see what the pattern is, and then use that to more accurately estimate timing going forward.
Break time into small units to plan more precisely. Engineers often struggle to budget time for long-term projects. To make this easier, break them down into week-to-week plans. Similarly, engineers have a better sense of what they can accomplish in the hours before lunch and the hours after lunch — rather than what can be accomplished in a day. Consider breaking days apart to give your team a more accurate sense of how long something will take.
Make using Jira more tolerable with a few rules. Pretty much everyone hates using Jira, but if you have more than five engineers, it's pretty much the only tool that's going to get the job done. To make it more livable, make sure you're not going overboard on customizations. Keep your workflow very basic, and don't let individual people on your team make alterations that aren't absolutely necessary. If every need in your code has a ticket number, you're in good shape.
Get out of code when prototyping. Start anywhere but the code base. That may be Sketch, Invision or pen and paper. Focus on building a clickable or drawable prototype. It may be hard for engineers to stay out of the code. But it's better to iterate on Sketch or paper for early prototypes. It'll save you time and keep you from ripping out code you just wrote.
ON DIVERSITY AND INCLUSION
The CEO and leadership team have to drive the effort from the top. They have to make building diversity and inclusion a business imperative in order for it to be successful. They shouldn't hand it off to someone in HR, or task someone outside of executive leadership to spearhead the diversity effort. Otherwise, they won't get company-wide buy-in.
Make job descriptions more inclusive. You can attract more diverse applicants if you avoid using male pronouns and stereotypically masculine words or attributes when outlining desired qualifications. Tools like Textio can help identify if language in job descriptions is potentially disqualifying.
Diversify your candidate pool. Make sure that at least one woman and one underrepresented minority is considered in your hiring pool for each role. Some companies tie sourcing diverse candidates to OKRs to keep themselves accountable. This type of practice has produced measurable, near-term results for hiring managers looking to cast a wider net and increase diversity at the top of the funnel when they start a new search.
Be mindful of the public image your company projects. What do candidates see and likely assume about your company when they go to your website? How about when they read articles about you? Do you have photos of employees? What would someone glean from how your team or about pages are set up? Would they assume someone like them would thrive working for you? Make sure you're demonstrating a diverse and inclusive workplace to encourage more diverse candidates to apply.
Take intimidation out of the candidate experience. Give them more time to prepare and be flexible about the completion of take-home challenges (within reason of course, but be generous). Several successful, high-growth companies have had success giving candidates the opportunity to prepare or brush up on relevant skills by giving them an outline of what will be discussed during interviews 48 hours ahead of time. To read more about this tactic in practice, see NerdWallet Candidate Experience, Slack's Guide to finding an Engineering Job and Stripe's Interview Guide.
Ask yourself these questions to create an inclusive environment. The following list is a good place to start when you're determining whether you're on the right track with incorporating employees of varying background:
- Are they given the resources (tools and mentors) to succeed?
- Are they being fairly compensated, relative to their male peers?
- Are they being promoted at the same pace as their male counterparts?
- Do they have the same access to opportunities?
- Are they being heard?
- Are there safe environments for them to share experiences and obstacles?
- Do conversations on diversity evolve into company-wide commitments and core values?
There are gold standard D&I resources. Increasingly, there are high-quality playbooks and toolsets available for companies looking to improve diversity and inclusion. The top recommendations at CTO Unconference were Project Include, and especially its Startup Include program; Homebrew's Diversity at Startups; and Interviewing.io, a free interview tool for candidates to anonymously practice for technical interviews and submit official coding challenges to employers. Employers can also use it to request introductions to candidates, set up on-site interviews, and more.
Photography by the dexterous Brett Berson.