Should product marketing decisions be data-driven or more so data-informed? Should PMMs lead with intuition and use data to back up their assumptions?

I love the data debate. A couple of my favorite quotes:

"If you torture the data long enough, it will confess to anything" - Ronald Coase.

"If I had asked people what they wanted, they would have said faster horses" - maybe Henry Ford

I dont think it's an either-or answer. Data is obviously incredibly important, as it can be the objective source of "truth" and the rise of machine learning and AI have made data more powerful and helpful than ever. But as you think about how data is used in decision making, I think it's important to decide what is "data" -- is it "hard" numbers like conversion, click thru, average deal size, people saying yes to this feature or no to that feature, etc. Or, is it experience data representing the feelings, needs, wants of customers as they engage with a brand? Or is it the insights of skilled, seasoned professionals who are processing thousands of data points at once and it shows up as "intuition" (If you've read Blink, you will know exactly what I mean)?

If you have too narrow a view of data, you run the risk of missing an opportunity (e.g. building a car instead of finding faster horses), so I like to have a more expansive view. And if you take data at face value without considering how it was gathered, analyzed or presented, you can draw inaccurate conclusions.

So, I don't think it's either-or. I love data and heavily rely on it for decisions - with an overlay of human discussion and assessment. But I also don't always have enough of it (at least traditional sources) and have to rely on other inputs. A couple of principles I like to follow:

1. If you dont have the data you wish you did, make your decisions but have them backed by stated, educated hypotheses. This will help you make sense of the outcomes of your decisions, regardless of how much data you had.
2. Get really clear on the questions you need answered to make an educated decision. Then determine the data you need to answer those questions. Often we have tons of data, but not sure what it is telling us.
3. Get alignment on what data matters, the method for gathering it, how it will factor into decisions, and where will you be ok NOT having all the data. The biggest challenge in making data-driver OR data-informed decisions is the lack of "single source of truth." Teams will spend hours debating the data, while missing opportunities to win / keep customers.

The voice of the customer should be what drives your decisions as a PMM, and it reaches you through various channels. In some instances for OkCupid, that is completely data-driven. For example, when Roe vs Wade was overturned in 2021, we saw an 18% increase in users stating they are pro-choice on their profiles, so it was clear that finding a pro-choice partner was increasingly important to our users. Therefore, we launched our I’m Pro-Choice profile badge so users could filter for other users who were pro-choice. And it worked: people with the badge are nearly 2x more likely to get a reply to a message compared to those who don’t have it.

Other times, we are driven by ideas and feedback from our users (and potential users) that may come more anecdotally. We see what our key demo is discussing on social media and what topics are trending, and implement new features. Recently, we added new matching questions written by ChatGPT because we found that millennials and Gen Z are more and more interested in AI and how it will begin to play more of a role in our everyday lives. Those questions now have hundreds of thousands of responses from our users.

Having data to back up decisions and new product features is vital. But it’s important to remember the human aspect of the product you deliver, and listen to what your customer may be saying beyond just on your platform.

• Data driven!

• You can use intuition (re: what you're hearing from prospects, customers, reps, analysts etc) to create a shortlist of assumptions, but you must validate that with data. If the data doesn't support it, it doesn't mean it's not important, but you need to clearly call out the discrepancy of what you're hearing vs seeing in the data (and perhaps dig more into why this is happening -- or give it another quarter or two of data to track any changes)

• If you lead with just data informed, you're going to be looking for specific data that validates your assumptions (and might miss other insights right in front of you). Taking a data-driven approach means you are looking at the quant data and summarizing what it's telling you -- and then combine and rationalize that with the qualitative data you're collecting from the field.

• Qualitative feedback matters and should be used in tandem with data. But any recommendations or business cases or perspective you share should be based in data (both quantitative and qualitative)

• But data alone isn't enough. You need to understand the data - what is the insight from this data? What is the data telling/showing us? And what are you going to do about it?

I'm not sure the distinction you're making between data-driven vs data-informed but I'm going to say that you should be data driven. There's some narrative about PMM that says PMMs don't need to be analytical, or comfortable using data and I don't agree with that at all. Instead of intuition, you should have strong hypotheses - grounded in a customer insight - that you want to use data to validate. Intuition is super subjective and very hard to defend, but almost no one will refute a hypothesis that you're looking to confirm or deny.

The highest quality PMM decisions are ones where the data and customer anecdotes align.

Skilled PMMs often bring a strong perspective to a conversation, but are willing to be wrong and change their recommendations in light of new or better information.

A strong POV can come from either data or your intuition, but one alone is never enough. You can state a hypothesis and then seek to validate this through both direct customer feedback and qualified data.

You should always be suspicious when the data and customer anecdotes don't match up. When this happens, trust the anecdotes and your intuition and reinspect the data.

Product marketing is an art and a science - while you want to make decisions based on data, you also have to use creativity to make bold moves. Decisions should be data-informed, with room to experiment and use instinct and knowledge of customers and the market to back up your assumptions.

I really like the term you used here: "data-informed". At SurveyMonkey, we strive to use the best data we have available to inform our decisions. But you don't always have data, so that's where a PMM's intuition comes into play.

I'll also challenge you that it's not always hard "data" that you need to inform a decision. Sometimes it's sources like:

• Conversations - with those on the frontlines like sales & success managers, or with customers themselves

• Qualitative interviews / ethnographies

• Industry reports

• Competitive websites

Sometimes you need to place a bet on a direction, lean into it, and then test performance. You can make a plan to have data to inform how you evolve once you're in market.

Marketing measurement has been, is, and will likely continue to be imperfect - attribution in particular is so tricky. So you can't always rely completely on data. Same thing with revenue pipeline metrics - the quality of pipeline depends on sales discipline. Revenue is a good measure, but it is often a lagging metric, especially when the buyer journey can span several months to a year. And often times you don't have sufficient data and have to make decisions, make bets, allocate marketing budgets based on a combination of factors.

So PMMs should be data-informed. Use data as one of the key factors that guide your decisions, but also consider your understanding of the market and customer needs, anecdotes gathered from customers or your sister teams, as well as your gut feel and intuition. Intuition comes from experience, use data to validate or refine your assumptions.

All PMM decisions should be driven by user research, available data, market knowledge, and product expertise, topped up with an understanding of the company's long-term strategy. Amen. 🙏



To add on decision-making:
avoid following the "Highest Paid Person's Opinion" (HiPPO). It's when PMM decisions & initiatives are being prioritized based on

- past HiPPO experiences in larger popular companies,
- successful frameworks used previously,
- and "strategic vision"

However, such PMM prioritization doesn't come from user insights, market analysis, and data deep dives. This can result in a recipe for disaster.