Product Manager Test?!?

People over the years have asked me how to hire a product manager. Product management has become more popular over the years. Within the last week, I talked to a college student who is majoring in Product Management. That’s a thing now? Amazing!

So, how do you know someone who will excel in the role. The resume looks good but can they follow through with a roadmap and plan? Maybe you’re someone who was asked to complete a case study and want to understand the ‘why’. My network is full of startup CEOs who have PM skills all the way up to FAANG employees. I’ve hired for the startup and middle-size company organizations but have taken lightweight queues from the FAANG process.

First of all, the internal team needs to post an amazing job requirement. It’s like a preview to a movie. I’m a storyteller – joining my PM team you need to be able to share a vision and distill the story for others. At a mid-size organization, I’m looking for something unique for each role. The team needs to know the candidate can have ownership over one piece of the business.

Next, we have a couple of phone calls. If it’s a mid-size org it’s with HR and the hiring manager. At a startup, expect the founders to want multiple meetings. I do video calls and coffee meetups. Nothing unique there. Here we’re looking for the basic fit. Be a good human, have experience in product management reaching past a good resume. After this phase – every PM has a take-home case study time capped at 2 hours. It’s not a test but the Internet has started calling them tests. I’m not looking for a perfect answer but I am looking that you’re able to make good choices. You are able to put together a user story in a time-efficient manner. Most importantly you’re able to synthesize a problem and understand technology, design, and data choices you need to make in the real role. PMs sit between Tech, Design, and Business and you can’t ignore how they play together to make products people love to use.

Let’s look at a couple of examples of different areas of the case study great verse what causes the team to groan when reading a product management candidates case study.

These are generalized examples. None of them are directed back to a specific person.

  • In a startup, the questions may look the same but I’m looking for ‘more’. Early hires wear many hats and are able to jump in where needed. I have a soft spot for mentoring startup founders on successful PM skills and you need to understand many aspects of the organization. As a Techstars alumnus, the Give First mantra will follow me for my entire career. On the flipside, great PMs make great Founders, CEOs, and COOs. You don’t know where your PM career may lead! Shoot for your dreams!
  • At a mid-size organization, I’m looking for specialized product management skills for a particular area of the business. The example below assesses individual contributors. We have enough director and above level staff. An overview of the take-home case study reviews the following areas:
    • Summary of the user experiences and pain points. We set up the scenario from the user’s perspective. We do some storytelling of the customer journey without telling you the customer journey (we don’t want you to go in that direction). We give you stats, other basic data, and sometimes screenshots. Now we ask you to do the following: We outline what the org has the option to build using the previous data. The case study asks for an outline of how you’d approach the problem, define the users, outline user stories, and user flow. Depending on the role this may change.
    • Outline of how you’d approach the problem: This part of the case studies we will give more detailed questions. The case study has high-level requirements and distills it down to actionable work. We’re interested in your ability to get your hands dirty. The outline always covers the following areas:
      • Users: Who are you writing the tickets for? How are you approaching the problem
        • Great: You know the users are more than our customers, you know the tech industry lingo.
        • Groan: You look at one type of user, it’s the paying customer. You state the title of a user and provide no detail.
      • User Stories: Show me your style. I know every org writes these differently. Show me your current style
        • Great: You give the titles of the user stories across an epic. The team now understands you can quickly break up the work in an organized fashion.
        • Groan: You spend 1.5 hours on crafting the perfect long PDR. Does this make sense if the team is lean kanban? If our job rec states lean kanban… don’t do this. Reading the Job Rec and don’t know what lean kanban means? Google it, read up on how it differs from agile and waterfall before you start your 2-hour time limit.
      • User Flow: You know data trends and have assumptions of its heavy desktop or mobile users. You add screenshots. As a great PM who has been to the website, you already have opinions on improvements. HR teams sometimes even provide screenshots, use them!
        • Great: PM candidate decides the provided screenshots are desktop – assumes most of the traffic is mobile and adds her own. You can draw boxes or point out in the screenshots improvements.
        • Groan: Time isn’t spent here for site optimization role. You’ve wasted 1.5 hours on the 1 user story above.
      • Objectives: PMs are great storytellers and visionaries. We don’t ask specifically but state what you are trying to solve, solve it, then tell me how you solved it. Let me know you can tell a story across the boards. In your user stories and in your presentation skills.
        • Great: Outline the objectives and needs. Wrap with this and an elevator pitch of what you did.
        • Groan: Skip to a solution.
      • Follow Up questions: Add them.

Ok, So you’ve gotten through those 2 hours. You email it back. What’s next? Often at a mid-size org you are already scheduled for the next stage. The interview can separate you from the pack. Whether it’s a startup or a mid-size organization, the team WANTS to see you succeed. You are there because someone on the team is now doing enough work for multiple people and this role is deemed needed to help move faster. Take a deep breath, come in jeans, and come prepared to draw on a whiteboard. I love whiteboards, my team loves whiteboards…. are you ‘one of us’? 🙂

The interview day, we’re going to talk a little about the case study. We’re also going to talk about the products you love and why you love them. We’re going to ask you about your experience on our site. Again – we want you to be successful.

Great:

  1. Assume we’ve read it. Also, assume team members are busy and haven’t read it.
  2. Bring your case study with you. Maybe your laptop? That’s what I would do but I haven’t seen it yet. Why? That said, you do you – be authentic.
  3. Ask follow up questions
  4. Don’t phased by grabbing a whiteboard marker.
  5. Have speculations on challenges and ideas of what you bring to the table

Groan:

  1. Case study was clearly not taken seriously. Good or bad — Assume if you spent 5 minutes on it, we know it and that’s the first impression of yourself. This is the best way to have interviews removed from your itinerary.
  2. Don’t bring the case study with you. We may not ask for it but have it.
  3. Never went through the flow and expect the team to tell you what the organization does.
  4. Answers are from a blog post about how to ‘ace a PM interview’. It’s not specific, it’s not relevant, it’s not authentic, and it doesn’t show the team you have successfully sat in a PM role prior.

To Wrap, why do I promote having case studies? After seeing my tech teams provide tests, I took serious note of why I wasn’t doing the same. Interviews were wasted on my team, sometimes a hire would speak well in an interview but day-to-day the person didn’t fit the team’s needs. I wanted a way to understand the written thought process and the ability to think quickly on their feet. None of the questions are deep or tricky. At first, I struggled that PMs would feel this was pulling for consulting work for free. It’s not the intention. I purely want to assess skills, product advocacy, and tech industry knowledge. The only way to mediate this feeling is to cap the time. I’m not sure it’s the best method. Do you have ideas to make this better? I’ve thought about providing questions outside the business focus but it doesn’t always showcase the person’s interest in the organization. What have you seen as a hiring manager or candidate?

Leave a Reply