How do you measure the success of product messaging on and after product launches and iterate on it?
I use 30 day adoption as the primary KPI for measuring launch success (did we target the right audience with the right message and make it painfully obvious to do the thing?). I think success of the product or feature, over time, is another matter—where message testing and evolution is critical.
But for the former, you'll need to get very crisp on your measurement tactic, especially for features over net new products which are easier to measure as line-items in an order form:
1) Who do you actually expect to use the thing that's been launched? If adoption is 10% of your entire user base, that's not great, unless you've just launched a feature that's only applicable to 10% of your base...
2) Ensure your data team needs to know exactly what "counts" as feature adoption (clicks a thing, takes multiple steps, uses for a week, takes an action off the back of the thing, etc), and that they can reliably collect this data. There are tools out there like Heap, Amplitude, and Pendo that make this a lot easier if you don't have extensive data team support.
First, I think it’s completely acceptable to not have the perfect messaging ready in time for your launch. It’s ok, and actually, very advisable to keep iterating on your product launch messaging.
In order to help you measure the success of product messaging on and after product launches, here are a couple of tips:
- Take a qualitative + quantitative approach: For quantitative you can survey customers who have used your product. Evaluate if their value props and understanding of the product values match your messaging. For qualitative, try to run small focus-groups of customers using the product and similarly try to understand how customers describe the benefits of the product. You may also want to reach out to a couple of customers who churned off your product to understand what did not resonate well with them as far as the messaging and product experience.
- Talk to your customer-facing teams: You can reach out to trusted contacts from all levels of sales, success, advocacy and ask them if you can listen into their customer calls to see how they pitch it to their customers or prospects. Are they using the same messaging? What’s resonating? What’s not?
- Do some user testing: You can work with your UX Research team to do some user testing of the messaging, or you can do A/B test of things like in-product messages or emails to see if the messaging hits the right chord.
At launch, it is measured through qualitative sales feedback and top-level metrics for the landing page/social engagement.
Post-launch, go on shadow sales meetings to see customer reactions to pitch, and listen to sales call recordings. If you've done your research before launch well, your customer should be hearing their problems repeated to them in their own words to achieve immediate resonance to the pain you've uncovered. If they don't think the problem is a problem or if it is not big enough, then probe them to uncover the real pain and re-message.