AMA: Glassdoor Product Marketing Team on Market Research

First of all, *hugs*. I've worked on both the vendor and client sides of research, so I've had the opportunity to work with many different teams with varying levels of interest in market research. When I'm working with a team that isn't "bought-in" on the value of market research, it's usually due to one of the following reasons: 1. They're in "start-up" mode and not creating space within their processes to leverage market research insights 2. They haven't made it a priority to upskill their team on how and when to use market research insights 3. They're focused on short-term growth and improvements over long-term investments and innovation Much like market research, empathizing with your target audience is the first step to solving this problem. If a team says something like "Customers don't know what they want," or "We don't have time to take a step back," consider it an invitation to talk about what's working well in their process and what could be better. Learn what's working really well for them and where there are opportunities in their process. See if you can identify a problem they want to solve and make market research the solution. For me, I usually have luck leveraging the forward-leaning strategic value of market research in the product development lifecycle in these kinds of situations. The best products stem from exposing a pain point in the market and developing an innovative solution. It's a lot more efficient to do the research and discover the pain points than it is to A/B test and hope for the best. Good luck! 

There are a few different ways to go about creating personas (which may be more closely aligned to “archetypes” or “segments” depending on the terminology your organization uses). The way you source the data largely depends on what you’re trying to accomplish. But the short answer is you can use qualitative data, quantitative data, or a mix of both. Either way, you’ll usually get the best results with primary research data - data you collect as opposed to insights you find in published research. Methodology: 1. Using qualitative data is a good choice if: 1. Your target audience is difficult to reach (e.g. HR professionals or executives of large companies) 2. You don’t have the budget or tools to conduct quantitative research 3. You already have a strong sense of who your user/buyers are and are looking for a deeper, richer understanding of these folks 4. You need to complete the research on a short timeline 2. Using quantitative data is a good choice if: 1. You want to better understand the total addressable market (TAM), which kinds of attitudes, wants and needs make each segment unique, and which segments are most aligned with your offerings 2. You want to learn more about your existing user base by leveraging behavioral data (e.g. platform usage or purchase behavior), and segment them into groups based on their behavioral patterns 3. You have the resources and time to fully develop the research methodology to accurately meet your needs Research Objectives: 1. You’ll also want to consider your primary research objectives. Are you looking for personas that will primarily impact current or future strategic initiatives? Which teams will be leveraging these personas? Will they need to be operationalized in a way that will include algorithms and formulas to track their behavior? 1. Whether quantitative or qualitative, personas that are primarily based on attitudes, wants, and needs tend to be more future-focused because they highlight unmet needs and areas of opportunity. If your goal is to grow your user base or create new, innovative products, this is a good place to start. 2. Personas that are rooted in current behavior help optimize for the present and shorter term strategies. For example, if you’re creating presonas to learn more about your most engaged users, or users of a particular product, this could be a good place to start. Protips: 1. Bonnie’s (Director of Market Insights) Pro tip: Prioritize the jobs to be done with your personas. If it only does one thing well, what should that thing be and who will leverage it the most? 2. Patti’s (Head of Consumer PMM) Pro tip: Get the most out of your personas by engaging all potential stakeholders before you finalize your research plan. 3. Sophia’s (Product Marketing Lead) Pro tip 3: Use what you have! Get scrappy and connect with whichever teams have the best access to your target audience (e.g. sales, customer service, data science, UX, market research, PMM, etc.). Take initiative to get in touch with customers, work with teams that have dedicated research resources (like UX and market research). 4. (For more, check out our answer to the question about persona framework!)

1. Ideally you would draw on both a combination of quantitative and qualitative data to shape your segmentation research. Start with a hypothesis of who your TAM (total addressable market) is. This will help you to identify new market segments you may not have thought to target. For example, for Glassdoor our TAM is comprised of all working age adults who are either currently employed or want to be employed. Segmentation can then be used to expand the sandbox (market) that you’re playing in by expanding your current definition of who your true audience is - so you can see which segments present the best opportunity, the ones who are most valuable (i.e. highly engaged, monetizable) and/or the ones with the most growth potential who you can more easily convert. 2. To obtain a more forward leaning segmentation that is more long lasting, segment on audience wants, needs and attitudes as opposed to current behaviors which can constrain you as market and social conditions change. 3. Bonnie Chiurazzi’s Pro Tip: Take the time to really identify what is the problem, pain point(s), and action to take. This is critical. As Einstein quotes: “If I had an hour to solve a problem I'd spend 55 minutes thinking about the problem and 5 minutes thinking about solutions.” Screen respondents on behaviors, attitudes, wants and needs to see who is included in your segmentation. Don’t limit yourself to very specific personas. For example, in the area of employer branding, Bonnie screened respondents on the responsibilities they held, not their job title as different roles across the company have differing responsibilities at various companies. This greatly increased the number of people she was able to gather insights from. If Bonnie had screened by job title, she would have only had a small handful of Employer Branding professionals to draw insights from. But by widening the net, she was able to include respondents both in talent acquisition and in marketing roles who had Employer Branding responsibilities.

1. Persona frameworks are unique to the organizations and teams who use them. But there are a few core elements that will add some structural integrity to your personas. 1. Context and Market - How do you define the market that this persona is pulled from? Do you need to be able to size the perona’s incidence in the marketplace? 2. Name - Choose a name that is memorable and conveys whatever it is that makes this persona unique and/or valuable. Some folks like to give their persona a name like “Steve” or maybe “Steller Steve.” This is more of a stylistic thing, but I prefer more descriptive names to help stakeholders remember the most important traits of the segment. Also, make sure you pick a name you’re comfortable saying over and over in meetings! 3. Value - Identify how you will assign value to each persona. Is it an attitude they have? A behavior they exhibit? An action they take? 4. Who - Who are they (demographics, job title, hobbies, home life, etc.)? 5. What - What are they trying to accomplish in your marketplace and why? 6. Where - Where do they currently go to meet their needs? How does your brand show up compared to competitors? 7. When - What triggers them to action within your marketplace? 8. Why - Why do they have these specific needs in the marketplace? What is it that makes them unique? Presumably, this a big part of the reason you decided to create the persona, so dedicate extra time to this. 9. Pain points - What isn’t working for them? What could be better? 10. Their journey - Put it all together. What triggers them to action? What’s their desired outcome? What actually happens? 2. Sophia’s Pro Tip: Do the work to align on semantics early on. Align on a definition of a “persona” and what will be included in the profile. Make sure to revisit how your organization has used personas in the past and leverage existing frameworks and information. If you deviate from the existing frameworks or if you’re adding a new persona while others are still being leverage, make sure you’re ready to speak to the value your new methodology will add. 3. Bonnie’s pro tip: Map out your persona profile before you conduct your research. Then map your questionnaire or discussion to the profile to ensure you’re asking all the right questions to completely fill out the profile. 4. Patti’s pro tip: Check your personas for unconscious bias and make sure to include a diverse set of voice in the research that leads up to persona development.

Before we answer this one, here’s a little context about market research at Glassdoor. Our market research team is called “Market Insights” and it sits on the PMM team. Market Insights projects almost always have a strong PMM lead. 1. Before product built, PMM/Market Insights takes it; once built, UX takes it. There are some exceptions, but the reason for this criteria is that market research is more in tune with the attitudes, wants, and needs of the total audience, which helps us identify opportunities for growth. UX is especially helpful when you’re looking for opportunities to deepen engagement with specific products and experiences that already exist. 2. When UX and PMM work together, they can paint a clearer picture of who the target audience is and how they will experience your product. Some best practices that have worked well for us as we collaborate with our UX colleagues: 1. Meeting regularly to review quarterly research agendas and recent insights 2. Inviting each other to report share-outs 3. Including each other in insights-driven workshops 3. Bonnie’s pro tip: Map out your organization’s product development lifecycle and highlight key areas for research. Then determine which research team has the expertise and resources to tackle that phase of research and how they can package it up for the perfect handoff when it’s time to tag in the other team. 4. Sophia’s pro tip: As you’re considering which team should lead the research, consider which team is best positioned to level up the product offering and activate the insights gleaned from the research.

There are so many best practices here! We’ll name the ones most top-of-mind, but please know this is not an exhaustive list. It seems simple, in theory, to conduct a research interview. It’s just asking questions, right? But moderators spend years learning and perfecting their craft. 1. Screening 1. Lean into your champions, not your haters. Find respondents who have a real need in the space and who don’t already have negative feelings towards your brand. These people will give you the feedback you need to differentiate yourself in the space. 2. Be sure to listen to a diverse set of voices when you conduct research. Ensure a good mix of age, race/ethnicity, gender, etc. 2. Create a safe space, no right or wrong answers. I like to say, “The best way to help me out today is by giving your open and honest opinion.” 3. Ask questions in order of least specific to most specific and save any concepts or ideas to evaluate for the end of the interview or survey. That way you won’t bias the respondent up front with your ideas. 4. Ask questions that don’t have a right or wrong answer. For example, don't ask “Which is the best HR platform?” because there could be an objective right or wrong answer. Instead, ask, “Which is your favorite HR platform and why?” because then you’re asking for a response based on their personal experience rather than what they think the right answer is. Anchor questions to experiences, attitudes, wants, and needs. If questions are based to factual responses, the respondent will focus on being “right” more than being honest. 5. Similarly, don’t ask respondents what they want you to build for them. That’s your job. Instead, ask them what’s going well and what could be better with their experience in the marketplace. Listen for pain points that your organization could act on. 6. Seems obvious, but don’t word questions in a really biased way, e.g. “How much do you love this product?” or “Which of the ideas is your favorite?” Instead try, “How interested are you in using this product? Why, why not?” 7. Listen. It’s tempting to give way too much context or correct respondents when you feel like they don’t understand an idea fully. But instead of jumping in to correct, see if you can instead ask them more about their perspective. 8. Bonnie’s pro tip: Go watch really good interviewers get their subjects to open up and hone in on a core source of truth. Listen to when they speak (and don’t speak). Pay attention to how they ask follow-up questions. Note their body language. Watch how the interviewee responds to different cues. Oprah is truly one of the best. Re-watch some of her greatest interviews and pay close attention. I imagine there might be a point where analyzing X number of Oprah interviews is the equivalent of a RIVA certification (I said what I said). 9. Sophia’s pro tip: Using “Why?” as your go-to probe (aka follow-up question) will start to feel like an interrogation after a while. Try to mix it up with probes like, “Tell me more about that,” “Keep going,” “Can you give me an example,” etc. 10. Patti’s pro tip: Be empathetic. It’s not enough to say there are no right or wrong answers. You must truly conduct the interview with an open mind and be curious to learn about the respondent’s perspective. You’ll know you’ve made a connection if they start sharing not just their opinions, but their experiences and emotions too.

In our experience, market research is generally better suited toward future-focused strategic decisions than behavioral data (e.g. where can we expand our product offerings and what problems should we be solving?). Behavioral data can usually offer a clearer view into how to improve experiences and deepen engagement with existing products (e.g. after they do X, Y is a natural next step so let's send them a notification to nudge them in that direction). You can even link your survey data to behavioral data to get even more benefits. The three of us just completed a huge consumer segmentation analysis of our total addressable market. Now we're seeking to activate our typing tool on our site survey and link it to behavioral data so we can see exactly how each segment behaves on our site. This will give us the advantage of the forward-leaning insights from the segmentation as well as the current behavioral patterns of each segment today so we can learn how to deepen engagement now and expand usage in the future.

Yes! Glassdoor, under the leadership of Bonnie (Head of Market Insights), established two market research online communities (MROC): one for consumers (B2C) and one for customers (B2B). We regularly field message tests to these two research communities to get quick, detailed feedback on messaging. This is what Bonnie had to say about MROCs: 1. MROCs make message testing easy, efficient, and effective! Once your online community is fully recruited, you can send them a survey and usually get a sufficient number of responses back within 48 hours. Automatic reporting dashboards make it easy to share insights with the team. Fuel Cycle is our MROC provider and our partnership enables us to conduct far more research than we could before when we hired research vendors to launch our message tests for us. 2. One of the reasons we chose Fuel Cycle is because they’re partnered with Alchemer (formally Survey Gizmo). Alchemer is a fantastic survey platform, especially for team on a budget. In addition to all the usual survey question types, they have text and image heat maps which are some of my favorite tools to use while message testing. The PMM team loves to see which words and phrases engage the audience the most as well as words and phrases that are confusing or disliked. We can also cut the data to look at it by different demographics or audience segments to see how a message may affect groups differently. 3. When we run message tests, we’re not just looking to see which message is the best performing, we want to learn: 1. Which audience is engaging the most and why? 2. Are there messages that appeal especially well to specific target audiences, even if that message did perform the best overall? 3. Who are the product/feature acceptors and what do they think about the message linked to the product/feature? 4. Which messages are most likely to lead to the desired outcome? Pro Tips: * Sophia’s pro tip: Leverage the tools you have to get feedback. If you can’t field a survey, could you conduct a few in-depth interviews with customers that represent your target audience to see how they react? * Bonnie’s pro tip: Whether you’re using qualitative or quantitative methodologies, know your target audience and make sure to test the message with them. Getting feedback from outside our target audience may lead your messaging strategy astray. * Patti’s (Head of Consumer PMM) pro tip: In addition to surveys, when available, you can A/B test through the growth team in marketing and/or product. We recommend testing every part of the funnel to understand conversion and get a more nuanced understanding of what actually drives greater appeal (is it the color, design, certain parts of the copy/specific key phrases, etc.)

In the most simpliest terms, a direct competitor is solving a simliar or same pain point as you are. One goal in conducting a competitive analysis as part of your market research is to identify points of differentiation and keeping a pulse on what's working (or not) in market. You can produce helpful reference docs such as a competitive market landscape to competitive messaging frameworks to sales battle cards! Here is an example from Patti, our Head of Consumer PMM, from her past life: think about both share of wallet and share of mind. At Leapfrog, core competitors are the ones that also make electronic learning toys; larger set of competitors is every company that makes toys.

Bonnie, Patti, and I always recommend getting in touch with your target audience/customers by any means necessary! * One way to get started is to identify who has the best direct line to customers/consumers. Make friends w/ Customer Service team and make sure you’re aligned on best practices when reaching out to and speaking with customers. The people in your organization who can help you are usually in Sales, Social Media, Customer Service, UX, and Market Research. Pro tip: You can build an informal community by leveraging your known brand ambassadors; create your own lo-fi/informal community to dip into any time! * Another way is to get comfortable moderating your own research interviews. There’s a lot of information, classes, and certifications available online to help you start building your qualitative research toolkit. If you’re not sure where to start, Bonnie recommends checking out Quirks (an online market research publication with great articles). It’s a great place to go to learn best practices. * Also, try leveraging your existing tools. Most of us already have access to Zoom and Google forms. Many UX teams also have a subscription to UserTesting and the sales team might already have a subscription to Calendly. Take inventory of what you have and then determine which methodologies you can reasonably commit to. * Finally, Bonnie recommends Alchemer as a tool that many teams could implement, even with limited resources (And both Patti and I can attest to how great it is and are SO happy with the output of this tool for product marketing research). Alchemer is a great option for small research teams looking to conduct their own online surveys. Bonnie prefers Alchemer to Survey Monkey because it has a greater variety of survey questions (love the text and image heatmaps!) and its reporting dashboards are fast and easy to use. Plus it’s inexpensive and they have the BEST customer support. According to Bonnie, they once sent her congratulatory brownies to her office after they helped her successfully troubleshoot an especially difficult issue.

1. There are a number of foundational research reports and insights you can provide to your product partners before they delve into the development process. These include: 1. A broader overview of the competitive landscape and market landscape 2. As well as a closer look at the health of your brand and how it fares against it competitors over time through brand trackers and CSAT (consumer satisfaction) surveys 3. In terms of users and target audience, they can draw on segmentation and persona research 4. And I find that my product partners greatly appreciate and rely on value proposition research to frame their design decisions and utilize messaging insights to better frame the end product to our users. 2. In terms of how we present these insights, we find it helps to give a preview to Product leaders first to clear up questions or reframe as needed given their feedback so they can become early supporters and proponents of the research. Also, when sharing out more widely to the product org, as calendars can be hard to manage, I find it easier to be added on as an agenda item on a recurring Product team meeting, as most of the team will be in attendance. Another way we are currently experimenting with having Product partners ingest and internalize insights at Glassdoor are through immersion workshops. This allows them to digest insights we currently have before developing new hypotheses, like incorporating new segmentation research. In this case, we can develop a shared understanding of the unmet user needs, break down the jobs to be done and identify user pain points of our target audience.

Market shifts cannot be ignored, otherwise you will be left behind and will no longer be relevant. In the last two and a half years, we’ve all seen significant changes in the market given global events and cultural shifts on a number of fronts especially in terms of the pandemic and for social justice. Within a month of covid shutting down the US, Glassdoor quickly created a Covid-19 Response Campaign that offered a slew of resources and tips for job seekers from directing them to companies who were surge hiring and work from home jobs to offering a collection of blog tips and a webinar on how to find a job during that challenging time. In the aftermath of George Floyd, DEI (Diversity, Equity & Inclusion) is now a focus for many companies; the top hiring priority for companies is now Diversity & Inclusion according to a Lighthouse research study in 2022. With the ongoing civil unrest during the summer of 2020 and calls for righting systemic injustice, Glassdoor immediately chose to proactively meet this critical cultural moment and the needs of those who do not have a voice. Glassdoor made a public commitment to leverage its product and resources to help achieve equity in the workplace. We spun up a DEI Product task force to create products and features that help underrepresented and traditionally marginalized job seekers learn how people like them experience the culture at companies of interest, so we can shine a light on inequities and hold companies accountable. On the other side of the coin, employers are now able to highlight their DEI efforts and make statements in how they want to reach their DEI goals (i.e. increase women representation among senior leadership to 50% by 2025, increase Black employee representation to 8% in 2025). Learn more on Glassdoor’s initial DEI efforts here and our expansion of DEI features the following year. In relation to this, pay equity has been a growing issue highlighted both in social media and the growing number of states that have enacted equal pay laws, acts or statutes. Therefore, Glassdoor has obsessively focused on delivering a much more comprehensive, accurate and easy-to-digest view of a person’s total pay potential by investing in an advanced AI driven machine learning model. Along with that, we created features where job seekers and employees can see salaries broken down by gender and race/ethnicity to see where there are inequities in pay. We then generated campaigns around Equal Pay day to highlight these products and our core belief that everyone deserves to be paid fairly, and to get there, we need salary transparency to create more equality. The brief for our fall 2021 brand campaign also took into account the forces that were affecting the market. We chose to focus on those who were most affected by covid and the job market - women of color with children, with our ads telling a compelling story of one of these moms and how Glassdoor could help them find a job they love. The campaign resulted in a significant increase in both awareness and consideration. We also have a stellar Glassdoor Economic Research team that provides timely insights and research on today’s labor market by unearthing important trends in hiring, pay and the broader economy. They take current market forces into account and have released recent studies on parents return to the workforce, hybrid work, changes for Gen Z employees returning to the workforce, and a pay equity analysis. Much of their work is picked up in the press and media, positioning Glassdoor as thought leaders in this space. Bonnie’s pro tip: Get the most out of your research by using it to advance your organization’s thought leadership within your industry. If your perspective shifted after learning something, chances are that you’re not the only one who would benefit from that knowledge. For an example of thought leadership research, check out Glassdoor’s Employer Branding Report.

From my, Patti (Head of Consumer PMM) and Bonnie’s (Head of Market Insights) point of view, a successful GTM launch requires early PMM/market research involvement in the product development lifecycle. 1. At the pre launch stage, and ideally during product development: 1. Talk to your target audience 2. Validate product/market fit based on the market research conducted; be sure to get clarity on who you are solving for and what their JTBD or Pain Points are. 3. Once validated, go into message testing/value prop testing to ensure you’re putting your best foot forward in messaging that resonates with your target audience. 4. Now it’s time to pull together a GTM launch brief. 2. Getting ready for launch: 1. Shop your brief and value props with reasons-to-believe with key stakeholders across the org (i.e., product, brand/creative, GTM leaders, marketing, comms, business operations, rev ops and/or legal as needed). Your brief should have your market research insights infused in your recommendations. From your creative assets and copywriting to which audiences we want to target on which channels, you will ideally have data to back up your suggestions. 2. Iterate 3. Go! 3. Post-launch 1. Collaborate with product on engagement metrics/adoption metrics 2. Test outcome of the launch with the target audience (this part is super hard but possible. At Glassdoor, we leverage our brand tracker and site survey to keep a pulse and report on marketing effectiveness. Bonnie’s pro tip: Identify the pain point you’re activating against (ideally before you build anything) and use it as your north star throughout the entire process. Build for it. Message to it. Measure against it. Lean into feedback from respondents who feel that particular pain point most acutely.