Learnings from over 400 hours of market research with AI and without AI

How we researched our target market manually with traditional secondary market research

When conducting our research on Reddit and social media, we manually collected it, put it into a spreadsheet, and analyzed it. 

We use data collected on Reddit and social media to help us understand the market’s problems, desires, and needs and determine how we could best serve them, both with content and products and services.

Although we initially conducted this research manually, directly on Reddit, once we completed the first round of research, we did additional market research every month using a tool to fast-track the research process. Ongoing research keeps us up with market trends and helps us uncover new content ideas that will resonate with our audience.

How we used AI tool for target market research

We used an AI-powered market research tool to assist with our research. This tool research scours the web to help gain insights for a comprehensive audience profile. The tool then uses this audience profile to create content that hits on the audience’s problems, pain needs, and desired outcomes.

Although in the past, we’ve always conducted our research using traditional, manual methods, after experimenting with different tools and processes, we found that AI made our research far more efficient and effective. 

After determining a broad audience and Problem using manual research methods, we found that the optimal next step involves using AI to help determine the Desired Outcome of the audience and the audience’s needs.

An AI writing tool that uses the audience persona and audience profile

We then used the insights from the manual and AI research to create a data-driven audience profile and persona. The research and writing tool we input these insights into is used as a filter when creating content. 

Because of this, semantics were incredibly important when creating these profiles. The language we input into the tool influences the language in the content generated by the AI writing tool.

Following this, we used AI to refine our messaging. By this stage, we had enough data to determine our niche confidently. From here, the next step involved using AI with the Sell On Autopilot Process Framework to determine the steps our target audience would take to solve their Problem. It’s these steps that form the foundation of a content plan for the business.

After using AI, we manually research “the people behind the comments” online to learn more about specific people in our audience. Our goal here was to paint a picture of “the average Joe” in our audience. What are their interests, what books and websites do they visit, and which influencers do they follow?

We also manually research our solution (product and service) competitors and our content competitors to understand their Unique Value Propositions. Using all of this data, we can gain a clear picture of the landscape so we can “niche down” even further and create a Unique Sell Proposition for our solution that is irresistible to our audience and sets us apart from competitors. 

Finally, we use AI to fill gaps in our data to create a complete demographic profile and a semantic and psychographic avatar representing our ideal audience.

These insights we gained from our study

The “pros” of traditional market research

• When conducting secondary research online manually, we gained a more in-depth look at customer sentiment and behavior due to the nature of the data. 
• By collecting and analyzing conversations from Reddit and social media, we could gain intimate insights about our audience. Accessing conversations where people ask questions and talk about their problems, needs, hopes, and desires enabled us to “get inside their heads.” 
• While manually collecting audience data takes longer than producing insights with an AI research tool, it provides valuable emotional insights into customer behavior that we would otherwise miss.

The “cons” of traditional market research

• Because the traditional method of collecting and analyzing data relies on human judgment, there is always the potential for missing important details, failing to gain insights and bias in results. 
• Collecting and analyzing data can be very time-consuming and sometimes tedious.

On the whole, manual research provided higher-level and more granular insights than AI research and was more emotive. These insights helped us better understand our target market, audience, and niche so that we can serve them in the best way possible.

The “pros” of AI research

• AI can save considerable research time as you don’t need to review data manually. AI can analyze large data sets quickly and provide detailed insights that you may not be able to achieve through other methods. Doing this can help you to make faster decisions about your product, pricing, and marketing strategy.
• Using AI for market research provides access to a larger pool of data. The software is designed specifically for analyzing unstructured data sources such as social media and review platforms like Reddit – where people tend to leave much richer information about themselves than surveys allow.
• The insights we gained using AI were abstract and very concise. They helped us cut to the audience’s core, providing clarity without any anecdotal or personal stories “getting in the way.” 
• The AI insights helped us find a starting point for our messaging, which can be difficult when looking at a sea of manually collected data containing many emotive stories, personal opinions, and lengthy verbiage.
• Using AI to crawl Google, Reddit, Quora, and Social Media provide deep insights quickly and easily, which we cannot gain through face-to-face interviews and online surveys. Getting AI to act as a “fly-on-the-wall” and creating content that taps into this data enables us to develop content and campaigns that are truly audience-centric.

We determined the best time to use AI after the initial manual “seed” research to cut through the confusion and form succinct brand messaging. 

Cons of using Artificial Intelligence for market research

There are several weaknesses when conducting market research with AI:

1. As mentioned, if we only used AI for research, we could miss the emotional insights we can garner during traditional market research.
2. Because it relies on algorithms to analyze this data, there is always the potential for bias in results.

How and where do we now use traditional and AI market research now?

1. Manual research is ideal for the first stage of the seed research process: finding an audience and pinpointing problems. 

2. AI research is ideal for the second stage of research, to create data-backed messaging that is concise and abstract and cuts through the noise of anecdotal experiences by individuals.

3. We switch back to manual research to learn about competitors and specific “ideal” members of our target audience.

4. AI research is then used again to fill any gaps in the data.

Key takeaways

Both manual and AI market research have their merits and complement each other to give marketers the most comprehensive understanding of their target audience. After more than 400 hours of research and experimentation with and without AI, we developed a mixed-method system that draws upon the strengths of each research approach.

AI can provide faster results with larger amounts of data and an abstract overview of a market or niche, as machines can process this information more quickly than humans. 

As AI research often lacks emotional insights or anecdotal descriptions because it relies on algorithm-based analysis rather than human interpretation, we use manual data collection and analysis to bridge this gap. 

When used together, AI and manual research can give marketers the most comprehensive understanding of a target audience.