How AI is Set to Unlock Unprecedented Customer Sentiment Insights Using Better Open-Ended Survey Questions in Unstructured Online Interviews
Background & Problem
There are a range of tools available to customer experience (“CX”) teams. Popular tools include the use of traditional online surveys created in-house, and outsourcing more complex market research needs to specialized firms. However, both options have their faults.
Traditional online surveys are a more affordable option to gain insights. However, while more accessible, they suffer from low engagement, are limited to pre-defined answer selections, and therefore, generally offer limited insight into participant behavior.
On the other hand, market research firms provide services such as conducting in-person focus groups and in-depth interviews on behalf of their clients. Focus groups and in-depth interviews go deeper than traditional online surveys. Both focus groups and interviews provide flexibility to follow up and explore unanticipated topics, and are adaptable to ensure participant engagement. However, these services can be prohibitively expensive.
New AI Solution
Customer Experience professionals can now use generative AI-led and moderated online interviews, filling a key gap for in the industry previously not possible prior to the rise of large language models (commonly referred to as, “LLMs”). For the first time ever, high-cost in-person interviews led by a single human moderator going one-by-one can be online and led by a customizable AI prompt interviewing tens, hundreds, or even thousands simultaneously, and at a lower cost akin to traditional online surveys.
Recently Successful Pilot Using Generative AI to Lead In-Depth Interviews
Our team at Crowdlytics had a CX customer who wanted to understand more about interest-levels around a new service offering under consideration. The customer knew the topic was too complex for a traditional online survey, and they preferred not to hire a high-cost research firm to conduct the analysis. They also wanted to use a third-party audience which required screening and sourcing. Therefore, an online interview led by our AI agent was a great match for their needs.
Configuring the Campaign
Using our platform at Crowdlytics, we launched a new campaign to support the project. We started by setting the context for the discussion to be led by AI. In other words, our customer configured our prompt with what they were trying to learn more about from the audience. We instructed our AI agent (the “interviewer”), to conduct live 1:1 interviews, and further instructed the interviewer its job was to creatively pitch the idea to the participants, gather their reactions, probe why they may or may not be interested, explore pricing and comfort-level details, and spark fresh ideas surrounding the product and service concept.
Customizing the Prompt
We gave our prompt specific instructions, such as:
- Open with a dynamic, engaging pitch that clearly explains the service and highlights its benefits in a way that connects with everyday frustrations or future aspirations tied to the service.
- Lead the conversation by posing open-ended, thought-provoking questions to your participants; capture their responses in a realistic, conversational style.
- If any participant expresses disinterest, delve into the reasons behind it and see if there are factors that could change their mind.
- If participants show interest, explore things including but not limited to the following:
- How much they’d be willing to pay for such a service;
- What level of detail they’d expect in the service;
- Privacy and security related concerns; and
- Payment preferences.
- Use leftover time to lead participants through some brainstorming on related ideas, new angles, or future directions for the service.
- Stay in character as the interviewer until the session ends.
With the topic finalized, we configured additional settings for our customer, including the name of the AI-interviewer and custom welcome and exit messages. We also arranged for a third-party audience to be sourced from one of our vendors.
The Results
Within a few days we sourced and completed the campaign, and all 50 participants completed their sessions with a median interview duration of ~5 minutes and 30 seconds. We provided our customer with a summary report and access to a dashboard highlighting various metrics. Our summary report included a range of participant demographic statistics and an overall summary providing color on:
- Preferences
- Pain points
- Behaviors & patterns
- Pricing sensitivity
Our dashboard captured several unique quotes and quotes where we assigned a customer sentiment ranking, which is an indicator we developed to evaluate a user’s overall sentiment in a response from 0 to 100 (100 reflecting an extremely positive sentiment). Lastly, our dashboard generated several visualizations of common themes it captured across all interviews.
What Does This Mean?
Our technology demonstrates that AI can do more than just generate question templates or be an ancillary tool for existing researchers conducting a focus group or interview. Our AI can LEAD interviews by asking thoughtful, engaging, open-ended questions to an unprecedented audience size and synthesize that information into useful metrics. With this technology, interviews are now as accessible as traditional online surveys and will provide CX professionals with more information than ever before about their customer base in a far deeper and more nuanced way. What are some practical use cases for unstructured, in-depth interviews led by AI?
- Exploring personal experiences or narratives
- Emotionality and sentiment
- Understanding cultural or social phenomena
- Uncovering sensitive topics
- Exploring complex decision-making processes
- Capturing in-depth opinions or attitudes
- Exploring behavioral motivations
Stop using traditional online surveys with one-dimensional takeaways. Start having dynamic, open-ended, unstructured conversations to gain real insights.
Want to learn more? Reach out here.