What's an example of how you've used user research to update messaging?

A few years ago, we had a bit of long-running internal debate about whether our target audience was more interested in “customization” or “guided study.” There was a product philosophy that we should offer guided options as much as possible to reduce friction. While that premise made sense, we often heard that the students who actually converted to our subscription offering, Quizlet Plus, were motivated by the promise of being able to create, customize, and modify their content, and also the way they studied. 

Around the time, I conducted some mixed qual and quant testing that included a number of qualification questions to segment by age and persona. When you conduct this kind of unmoderated study with a wide enough sample, you can start to see a wide spread in how different individuals are in their interpretation of your message. 

We saw that a lot of students were interested in the idea of “smart guidance to make the most of your time,” but also many of those same students had skepticism about the promise involved. The “guidance” piece sounded a little heavy-handed to many of the less-serious students that we had thought it might appeal to. We heard things like, “I don’t need someone to ‘guide me,’ I trust my own study skills. It could be helpful but it’s the least interesting option.” 

Another variant we tested in the same study was “customize study sets”. This variant was expected to go over well with the serious students who were currently most likely to subscribe, and it generally did. But less serious students also resonated with it. They read it less as “something that would be hard to do,” and more as, “a way to target only what I need for a specific test.” 

This insight enabled us to shift away from the initial ask from leadership to make “guided study” our topline value prop for Quizlet Plus. The data gave us conviction around centering the main message around “customizing sets to be exactly what you need.” The insight here was that customization isn’t always about going above and beyond as a power user, sometimes it’s just about relevance and empowerment, which is something that the “guidance” message failed to communicate.