In 2013, Jack Krawczyk joined Pandora to formalize its first product management team dedicated to advertising. With years of experience under his belt from StumbleUpon and Google, he was charged with not only shipping clever products that would make ads palatable for listeners, but with rallying a quickly growing team around this effort.
For Krawczyk, heading a product management team began by leading by example. Given the size of his team, he jumped in immediately to product manage the reconstruction and release of Pandora’s ad insertion logic. “It not only enabled me to better understand how the product works, but also how to stay true to the development cycle at Pandora.”
Krawczyk has seen a lot of ideas that have grown to become products, and others that never saw the light of day. His team’s advertising products frequently lead to creative solutions, but also to a lot of dead-ends. With these experiences behind him, Krawczyk has steadily risen through the ranks at the company. He’s emerged with six concrete tactics product leaders can use to not just build remarkable products, but talented product managers.
Know Your Team's Dunbar Number
In the 1990s, British anthropologist Robin Dunbar proposed that humans can only comfortably maintain 150 stable social relationships, which represents the cognitive limit given the average human’s brain size. For Krawczyk and his product development team, this concept resonates in their work. The core of their jobs relies heavily on building real relationships both broadly and deeply within the company while not overextending.
“I think every good product manager inherently is a leader. It’s an interesting job function because you're a leader with absolutely zero direct reporting,” says Krawczyk. “For example, product managers must galvanize engineers that don’t report to them and have several other business stakeholders to satisfy.”
The person directing product managers then must be a leader of leaders, and understand the architecture of business relationships within a company. Here are a few key points to consider when at the helm of a fast-scaling product management team:
Build relationships with engineers so they know the ‘why’ not just the ‘what.’
“For early-stage teams, the golden ratio of engineers to product managers is usually about 5:1,” says Krawczyk. “It facilitates a close enough relationship with the developers that product managers know instinctively what makes each developer tick. That level of understanding helps product managers motivate, structure their communication and get the highest production.”
As teams grow and become more distributed, the strength of the relationship between PMs and engineers is more critical than settling on a specific ratio. Engineers should have a keen and current understanding of not only what the work is, but also the decisions behind each product feature. Context is king for cross-functional teams. “If the product manager can clearly articulate what needs to get done and why, some really amazing and beautiful things can be brought into the world.”
Small product teams need to congregate.
Krawczyk recommends that when a product team is under ten people, it should take advantage of its small size to get everybody in the same room as often as possible. Invite engineers and designers to join, and integrate them into the planning meetings. At Pandora, this has not only strengthened the relationships of those working on product, but it has also increased the likelihood that each PM leaves the meetings with a holistic understanding of the product and its status from end-to-end.
In Pandora’s early years, former CTO and EVP of Product Tom Conrad instituted a “dollar-decision-making system” for all employees who touched Pandora’s product. In each quarterly meeting, attendees from across the company discussed the Pandora’s most important products and initiatives. To ensure active participation and prioritization, each attendee would receive five “dollars” to invest in the areas they believed to be most critical. Though the amount was small, it forced people to think through their wager.
For many startups, this system works well when product teams are small enough to keep the conversation manageable. Despite its increased headcount, Pandora still uses the dollar-decision making system, but in a distributed way across various product management teams. For example, the advertising product group has a dedicated “dollar-decision” budget to inform its roadmap. Ultimately, the key is to recognize and take advantage of close conversations involving different functions. This is the fuel that will propel the company to grow.
Larger teams need to get serious about process.
“One of my most interesting challenges has been how to keep the collaborative nature of Pandora’s early decision-making process, but adapt it to serve our growing and specialized teams,” says Krawczyk. When Pandora was less than 500 people, open-invite, intimate town halls about product development worked well. But now the company is over 1,600 employees strong, including more than 25 product managers and directors. Using the same tactics would lead to chaos, circular conversation, and limited relevance for those involved.
As soon as you need an ‘overflow’ room for product meetings, it's time to really buckle down on process.
When Krawczyk started at Pandora two years ago, there were three product managers focused on advertising. During his tenure, it has grown to 12 PMs, some of whom are new to product development and others with over a decade of experience. The 12 product managers work with over 55 engineers, who to a certain extent, are also on Krawczyk’s radar.
Once the number of product managers neared double digits, Krawczyk implemented a group product manager reporting structure. This move was not only to extend his scope of leadership, but also to expand the capacity of the team. “If you have many direct reports, you’re unable to scale insights and communication on a 1:1 basis. The team will start to feel disconnected. And as a manager, you are effectively setting up roadblocks. A large part of the job is to remove obstacles for your team, not create them.”
To keep community and efficiency in his team, he has organized his group into three functional pillars, which cover advertising end-to-end: listener advertising experience, ad distribution technology and advertising buyer experience. Krawczyk assigned one group product manager to each pillar. Each group product manager manages four PMs, and each product has a designated lead. This structure helps facilitate fast, focused meetings by vertical and anticipates a growing PM team.
Put Everything in Writing — Then Tier It
“The hardest part of doing any form of work is taking the words out of your brain, putting them into writing and talking about them,” says Krawczyk. “The inclination that a lot of people seem to have about writing is that it’s repetitive work or just to hash out feelings, but that’s just not true. It’s important to overcome that misconception, especially as more people — and voices — join a team.”
This realization really hit home for Krawczyk when his PMs, engineers, sales marketers and operations team left weekly meetings with seemingly different understandings of what had been discussed and decided. He would hear these types of rifts emerge in hallway chatter and subsequent conversations. If this disconnect occurred with everyone together, what would happen when the team grew too big for one-room gatherings? In order to catch these situations, as a manager, you have to speak with enough people to make sure there's unified clarity around the work at hand.
After one fruitless meeting, one of Krawczyk’s product managers had an idea: Distilling relevant information into tiered documentation.
One day, a PM on Krawczyk’s team noticed that, on one on side of the room, there were engineers referencing their massive Product Requirement Document (PRD), chalk full of specs and user stories, and on the other side sat a contingent from the marketing team referring to their planning documents. In an effort to bridge the gap, the PM decided to extract the relevant marketing information from the PRD and put it into a separate document, but retained its format so it could map back to the PRD. That way, he’d create a useful and familiar document for both parties to use — and an easily actionable plan for the rollout team.
Krawczyk noticed how well the concept of tiering information worked in his PM’s product area, and applied the practice across all his teams. “The key to being a strong product leader isn’t coming up with the ideas yourself. It’s about listening to your team, identifying what works best and determining how to best apply it across the entire group,” he says. Now, Krawczyk’s teams operate with the help of four key documents (in order from basic to expansive):
- Executive Summary: an abstract of what the product management team is working on overall; the highest level rundown and a great place to start to get up-to-speed quickly.
- Wiki Pages: Central, intranet pages organized by product management pillar; a high-level snapshot of each focus area. It includes engineering statuses and operational milestones to hit. Wikis have a progress map for each pillar and timelines by quarter.
- Product Requirements Document (PRD): The comprehensive planning document, including standards, product specifications, user case studies, etc. This should be an end-to-end, one-stop-shop, detailed product plan.
- Launch Plan Document: A composite of all operational planning documents to coordinate every effort between all teams, including product management, sales, marketing, and development. This was adopted a little over two years ago due to Pandora’s scale.
Of the four primary documents, Krawczyk is particularly bullish on wikis as a tool to help his product managers stay visible to the company, articulate their work, set quarterly goals and root themselves in their product roadmap. To dive a little deeper, here's what an effective wiki should accomplish, and the best ways to go about it:
- Transparency and visibility. “By their nature, our wikis are collaborative and visible to anyone in the company. We have quarterly business reviews with our executives. Wikis allow us to bring visibility to our team’s work for leadership.”
- Succinct summaries and concise communication. “Each wiki links to an executive summary, which helps keep all stakeholders on the same page, even if they are spread across 38 offices.” There is also a section prompting PMs to deposit pithy pitches of what they are working on and why — a helpful exercise and reference point if talking to colleagues, customers or Pandora’s leadership.
- Quarterly goal setting. “We don’t specify exactly what’s going to happen every given quarter. When you’re in growth mode like this, there may be some quarters where we bring in five developers and others where twenty are working with us. Due to that variety and our philosophy of small experiments, I encourage product managers to phase all tests, project milestones or releases in the span of a quarter. Of course, products often require more than three months of work, but the format of the wiki pushes PMs to break bigger projects into segments and sequences.”
- Clear definitions of what is and isn’t next. “Within each group’s wiki, we have a road map, which contains a stack rank of all the product features to be developed and released. We’ve learned that it’s as important to put a section for what will be developed as what will not, in order to remove any ambiguity between what’s in the future versus not in the cards at all.”
If documentation is critical for a product manager, it’s indispensable for a leader of product managers.
Ultimately, Krawczyk identifies himself as being the product manager of the product team. The team itself is the product he's building. “Whether it’s a value proposition or why we need to put a dot here versus there, I need to explain my reasons to the team. Because as we tackle bigger challenges, it’s important that PMs understand the decisions that were made, so they can make tough calls with all the history and information in mind. It’s been insanely useful for us.”
Make Communication Your Path to Ownership
For Krawczyk, communication is intimately tied to a shared understanding not only of the topic at hand, but also an opportunity to take or give ownership.
Iterate how you communicate as much as it takes for everyone to understand the critical information.
“As a product manager you need to be constantly aware of whether or not people understand what you’ve collectively agreed to, and so that’s why communicating is such a vital part of being an effective product manager. You fully contextualize not just what needs to be done, but also why and how it maps back to the goal.”
If Krawczyk and one of his product managers disagree, he questions his underlying understanding of the issue. “I’ve found that often if I don’t agree with a solution, it’s normally because I don’t have a clear understanding of the problem. I’ll admit to the PM that maybe I don’t understand," says Krawczyk. "If I feel like I get the problem and the solution isn’t the right one, I probe deeper to unlock how the PM got to the answer and how we can mutually agree on the proposed direction. Ultimately, I keep my energy aimed at the problem, not the person.”
However, if the team doesn’t understand, you are to blame. “You are the owner of the communication flow and the common understanding that your team shares — or does not share,” Krawczyk says. “So if there is a misunderstanding, it’s your fault if they are on a different page with you or each other.”
Don't Confuse Micromanagement with Miscommunication
“It’s a very common tendency to just hand over the product requirements, and say 'I need you to do this.' The necessary next step is to contextualize why it needs to be done in the first place.”
Several months ago, Krawczyk faced a situation where he wanted to change the way he was reporting on one of his advertising product management pillars.
“I wasn’t sure that we needed to reinvent the wheel, but rather reframe how we talked about it,” he says. “I found myself making a diagram and suggesting new rules for the wiki page in this pillar and it created confusion. The pillar’s product manager looked confused. I realized I made the exact mistake that I coach my team to avoid. I stepped back and gave the full context. It was easier to explain why I wanted to make the change, rather than prescribe the solution. When I was able to clearly articulate the ask, we were able to move more quickly.”
Make Team Management Your Highest Priority Product
It happens all the time. A successful product manager is asked to lead a team of product managers. Though she’s superb at product management and is driven by creating and shipping products, she finds herself spending most of her time managing people — and becoming further and further removed from actual product management. Krawczyk felt the same until, like any savvy product manager, he reframed the challenge.
“What I love about product management was the initial idea, working with stakeholders, getting people on board with a concept, learning from missteps and taking ownership for gaffes. Of course, nothing’s better than joining my team to shout successes from the mountain tops,” he says. “As a group product manager, my job is to help PMs own their ideas, work with stakeholders, get people on board and amplify their insights. Our product managers are my products. I am using the same skills, but it’s a subtle shift of application.”
To be a good product team manager, you need to have been a successful product manager first.
So how does one build the best product managers? Krawczyk offers five ways in which thinking about team management as a product has worked:
Set up advanced listening mechanisms.
“Product managers tend to have a reputation of loving to hear the sound of their own voice,” Krawczyk says. “The best product managers are the ones that spend a majority of their time listening. I try to do as little talking in meetings as possible and just ask questions to get people to formulate and articulate ideas. It’s the most amazing feeling in the same way that, as a product manager, you see your product, you have this hypothesis for it, people use it, it kicks off, and it blossoms.”
Tactic: Start by asking your product managers open-ended questions that a customer might ask when first interacting with the product. A product management team should be designed to anticipate these questions and supply answers before anything is ever released. What questions might your team have for customers? Enumerate those as well.
Encourage your product manager to get out in front of customers — but then listen to them as consumers. To do this, Krawczyk encourages PMs to sometimes ask questions without mentioning Pandora: "What’s the one product out there that you’re most excited about right now? What are the brands that you try to emulate? What are they doing that’s really great?"
Keep your feedback loops healthy at all costs.
Krawczyk has established a series of weekly one-on-one meetings and two team check-ins — a product meeting on Mondays and a logistics meeting on Wednesday. These sessions not only give his team defined opportunities to voice concerns, but also help him stay knowledgeable since he’s more removed from day-to-day developments. He uses a dashboard to keep a pulse on what’s important, but these meetings help to confirm or adjust his understanding.
“Monday morning at 9:30 a.m. is my most important time of the week, as it’s when the full product team of over 25 product directors and managers convenes. It starts with a pre-read of our wiki pages, so we make the most of our time together identifying insights and reviewing more in-depth product releases,” says Krawczyk.
Our conversation can include more thorough updates, but also cover proposals to tackle future challenges. Several have resulted in new features put on our roadmap. “During that Monday group meeting I’m scribbling down all the notes that I would like to follow up on with individual product managers — either to share with their group product managers or directly via Slack,” Krawczyk says.
Krawczyk also holds a weekly meeting on Wednesdays for 30 minutes with his ad product team. “This session was for administrative nuts and bolts, from PTO to policy announcements. For my first six months, I neglected to carve out time for these topics or they’d always slip to the bottom of the agenda. Then I thought I could just send an email to all the PMs. But these areas are important and should be covered in-person — even if quickly.”
Tactic: Establish a consistent set of group and one-on-one meetings, ideally early in the week to help set the tone and get the team on the same page. Sequence them so their impact compounds; one can be about information gathering and the other addressing the issues that arise. Vary the meeting size so participants have a choice of venue to raise an issue and can determine when it’s most comfortable or relevant for them to do so.
Your team should have a purpose as clear and obvious as a product's purpose.
“I ask each product manager to understand the product that they’re working on by deeply articulating its purpose, its high goal and ambition. I really want them to own that,” says Krawczyk. “Laser-focus on revenue targets is very important, but for some, this results in tunnel vision. I’ve realized that taking time to understand the underlying principles and motivations will increase customer engagement rates and benefit revenue goals, too.”
As a group product manager, Krawczyk applies the same principles. Drawing from Daniel Pink’s book Drive, he’s become a student of the individual motivations of his product managers. Their drive usually draws from one of three feelings:
- Autonomy – Self direction is the innate motivation. Those driven by autonomy value trust and dislike micro-management.
- Mastery – Development in capabilities and career — not just progress with work — is key. Those driven by mastery ask, “What am I becoming better at by working on this?”
- Purpose – Connecting to common mission or cause is most critical. Those driven by purpose ask and can answer, “Why am I doing what I’m doing?”
Krawczyk has taken this framework to work more effectively throughout Pandora. As he notes, “I’ve worked with certain types of people that are just fascinated by complex math problems or throughput and scale problems. Others really want to have the most elegantly designed user interface.” One of his goals is to help his team learn how to better understand the way people across the company tick and use that information to complement their style.
Tactic: It’s common to investigate user motivations or market behavior when building and releasing a product. Do the same for your product managers to understand what will make them succeed and how this aligns with your group and company’s mission. Discern what motivates your product managers and incentivize accordingly.
Provide the right amount of runway.
Every product development team has time and resource constraints. As much as a group product manager can alleviate those issues, the better her products managers’ chance at success. Here are two strategies Krawczyk has seen work:
1. Stay firmly grounded in reality. Especially if you're building a new product from scratch, it can be easy to postulate and plan any number of pie-in-the-sky features. And indeed, a good product leader will squeeze as much creativity as possible out of the PMs they manage. But asking someone to think big without giving them the resources to execute is utterly deflating.
“You never want to delegate work to PMs when they don't have the engineers to do it,” says Krawczyk. Make sure the ideas you generate are grounded in reality. And if some end up being further afield, schedule them down the line in a way that makes sense.
2. Give your PMs quality time with a problem. “In order to be a really effective product manager you need to think about a product for more than two or three quarters straight. It takes time to get an intimate sense of the metrics that matter. After delving into the metrics, one can finally separate the signal from the noise.”
Develop your team alongside its metrics for success.
As companies scale, their product management function can change dramatically. Teams may be restructured, personnel will change, and responsibilities shift. Krawczyk has observed this evolution in his time at Pandora, StumbleUpon and Google. To rapidly adapt, he recommends two strategies:
1. Move from “fire”-based to product-based product management. It’s not uncommon in the world of startups to have three product managers across a young company. One might report to the CEO and the two others to a founder, who was the original product manager. This lack of structure and focus frequently produces PMs who are managing a lot but not for any one product in its entirety. “If you only focus on the next pressing issue, you can get stuck making local optimizations indefinitely, which leads to amorphous product development,” says Krawczyk.
2. Reframe your approach to insight-based product management. It’s as essential to own a set of insights as it is to own a product, but it can take time. As each feature launches, we ask product managers not only to show how metrics have been met, but also how each feature has impacted how people perceive Pandora. Krawczyk says, “Yes, it’s important that PMs feel ownership of products, but try to create responsibilities around insights — both quantitative and qualitative — that predominantly impact that product.”
It’s wildly important to regularly exalt and publicly amplify the good work your team is doing.
Once the approach to product management is focused on owning insights and giving PMs the autonomy to act accordingly, it’s important to create channels to share successes and lessons. Here are two tips from Krawczyk:
1. Know where the spotlight belongs. If Krawczyk’s team launches a great feature and Pandora is covered in Ad Age, he wants to be very clear that it was accomplished by the product manager, as well as her peers on the engineering, design and business teams. Additionally, if there’s an external blog post about a feature launch from the team, he doesn’t want it ghost-written for him. The product manager is the owner of his product, and so should also be the author of its story.
2. Share post-mortems on all projects. Then share them again. Post-mortems need not be long, drawn-out eulogies. For Krawczyk and his team, they're succinct and informal. After each launch or feature, they include one sentence on their internal wiki, concisely capturing the key takeaway. More detailed recommendations are built directly into the future integrations, but they can always return to the wiki to review a running log of what’s been learned along the way. “Always go back,” says Krawczyk says. “It’s worth repeating: Always go back. And once you do, repeat your findings again and again. Use them to inform your roadmap so you can continue to drive collectively forward.”
Always Be Grooming New Leaders
When it comes to the talent on his team, it’s important to Krawczyk to address ownership. “I actually really dislike calling it ‘my team.’ It sounds trite, but it’s our team. I feel that I get the opportunity of understanding what each product manager is working on and how we can bring it all together.”
There are few proactive steps he takes to identify top talent on his team and intentionally develop people into role models and managers. Often, it's a process that starts before they even come aboard.
- Articulate the mission early in candidate interviews. For Krawczyk, the most telling part of an interview is the discussion of the company’s mission. It not only reveals the candidate’s excitement and engagement for the company, but also shows if he knows why it exists and how he might contribute.
“I want every product manager that comes in to work for our team to be super excited about what they’re doing from day one, says Krawczyk. "The good ones have engaged with the product they’re going to be working on, and the best ones have already talked to customers about what they think. Doing that pre-work well means you understand the mission — and are eager to learn how to give to it.”
- Hire listeners that display intellectual curiosity. At Pandora, candidates applying to the advertising product management team will start with a phone screen or in-person meeting with Krawczyk. He likes to ask, “What have you last taught yourself?” This question gets to the heart of how applicants think through challenging themselves.
If there’s rapport, candidates have two rounds of interviews with a panel of product managers, including one from a relevant pillar. After that, applicants speak with a group product manager from another team, marketing partners and an engineer. At the end of the interview process, Krawczyk asks candidates to replay what they think the job is and measure it against how the job is structured.
- Offer high-performing PMs opportunities to onboard. “As the team scales, give product managers the chance to teach new people who come on board," says Krawczyk. "That’s where they are really crafted as directors or group product managers. It’s okay if some PMs just really love being individual contributors. Getting PMs to onboard can help you scale the team while providing more context into the day-to-day of product management at your company.”
Constantly Revisit Your Mission in New Ways
Pandora’s mission is to be the effortless and endless source of music discovery and enjoyment for billions of people. Krawczyk never lets his team forget that everything must roll-up and contribute to that greater goal. Throughout his career, he’s been asked to answer some difficult questions: "How will a new product get people to listen longer and longer? Will this resonate with people or just annoy them?"
Every day, Krawczyk returned to the company's mission to answer these questions. Even today his computer’s screen saver loops his mission, which stays with him like a battle cry: "Unleash the infinite power of music to enable brands to resonate with listeners!"
As a group product manager, here’s how to keep the mission front and center:
Test your metrics with your mission. When building a specific feature of a product, it may be challenging to easily link your immediate work to the company’s grand mission, but it can help course-correct. “In the world of advertising products the default can easily become reaching as many people as possible, when the core goal is to emotionally connect with people through advertising products," says Krawczyk. "How does this happen? Well, we’ve always had frequency metrics as proxies for connecting with listeners, so it’s easy to see how our focus might stray. We return to the mission and make tweaks to how we measure in order to get closer to the right metrics.”
A mission should be expansive; occasionally look afar for inspiration. To find ways to be an undisputed leader in radio, one can still draw inspiration from leading companies outside it. Krawczyk gives an example: “In San Francisco, we have a strong ‘foodie culture’ with Michelin-rated restaurants sprinkled all over the Bay Area. What people don’t realize is that the original Michelin Guide Books were created 100 years ago by the Michelin Corporation to encourage people to dine out more. Of course, when people eat out more, they typically drive more and will need to buy more tires. It just goes to show that some of the best advertising may not appear to be advertising.”
Working in advertising is very much like being in the CIA: When you’ve done your job well, no one notices, but if you screw up, then the world hears about it.
Most importantly, as a product leader, you must understand that your role and purpose is innately human — even if it may not seem like it immediately.
“Over my career, I’ve asked myself what is interesting about advertising,” Krawczyk says. “I genuinely believe in the power of advertising as creating resonance, calling up a lasting image, memory or emotion. We’re emotional beings. We make decisions emotionally.”
Being able to mine this deeper motivation is critical for product leaders in particular, because they need to be able to inspire the sentiment in others. This is perhaps the most critical quality to embody. It’s imperative to connect ideas until you find the human, impactful center of whatever work you have in front of you.
“At Pandora, we build products that capture an intrinsically emotional experience for people,” Krawczyk says. “That's what needs to be top of mind for us to do our best work together no matter how complex, removed or granular it gets. It's about moving people. By working in product and being a product leader, you get to think through the best way to do that.”