Back
Good, fast or cheap – you can pick two. When it comes to marketing research, there is a requisite tradeoff between depth of insight obtained, time and cost. With the advent of AI, researchers naturally wondered whether this technology could break that paradigm. Can AI help us be more efficient and insightful when it comes to things like social listening and audience profiling? Is there a chance for AI to make easier work of drafting survey questionnaires or analyzing data? Can AI finally provide the key to delivering strategy-shaping insights in a fraction of the time, for a fraction of the cost? I believe it’s possible – but after doing some research of my own (see what I did there?), I have identified three important caveats to consider before saying “yes.”
#1: Garbage In, Garbage Out
The biggest reason to say “yes” “is that algorithms developed with data from reputable sources can enable businesses to assess and analyze vast volumes of data in far less time than traditional analytical methods. Speed alone could enable research efforts to achieve scale never before imaginable. “Sign me up!” you might say. However, “data from reputable sources” is often hard to come by.
Unfortunately, AI algorithms are trained on existing available information, and there are no accuracy controls or standards applied to information posted online. As a result, using generative AI tools like ChatGPT that are trained on this information delivers results that are often not trustworthy or complete.
In fact, ChatGPT includes disclaimers in its responses to this effect: Please note that my information is not current, and the [answer] may have changed since then. You many want to check the latest statistics and updates to confirm [the answer]. Firms who want to use this type of AI need to be aware that, unless they have proprietary data from their own customers or other pristine data sources, AI analyses will always need to be vetted for accuracy and often re-researched for validation, which can offset the speediness of AI and its potential cost-savings from reduced analysis time.
#2: Generalized vs. Specific Information
When leading research studies for clients, I firmly believe that the value we provide is wholly dependent on our confidence in the data we collect (e.g., the “good” part of the equation). In applying AI to marketing research, another consideration is that AI tools are not yet able to accurately grasp the context of a conversation or interpret ambiguity in information. Further, by identifying the most common data points related to a query, AI results represent a most common or generalized scenario, which doesn’t enable researchers and marketers to identify those less obvious but more specific and differentiating insights related to our objectives.
As an example, the ChatGPT question “What birthday present should I get for my sister?” resulted in a laundry list of the most common gift categories. I’m imagining my sister’s response when I say: “Happy birthday! I got you a personalized gift. Hugs!” The true struggle, of course, is identifying what kind of artwork, jewelry or other personalized item my sister would like, rendering these results worthless.
The reality is that, to date, there is no way for these models to provide tailored outcomes because unique, personalized inputs are unavailable to them. Meanwhile, research insights are rarely “good” if they’re not specific. Specificity of insight could not be more critical for marketers when devising content and communications plans; without it, marketers risk going to market as an undifferentiated player.
#3: Shiny New Objects
The term “generative AI” nods to the creation of something new – i.e., new text, new images, new data. That all sounds exciting – but when applied to marketing research, it raises the question: What about best practices?
Research professionals adore best practices because they represent the tried-and-true approaches that have been supported and validated by reams of usage and data. Even better, data and approaches that can be “standardized” as benchmarks – like the Net Promoter Score metric (NPS) or price elasticity models (I see you van Westendorp) – are relied on with added confidence based on their industry validation, understanding and acceptance.
So, excuse us strategists if we get a little fidgety considering adopting new technologies like AI as part of our research plans. For me, I realized that I’d just have to see how well AI actually works to have an informed point of view.
AI in Action
Taking a page from Jim Carrey’s character in the movie “Yes Man,” I forced myself to say “yes” to analytical tools offering an AI-version or option in Beta. This included a social listening tool, audience profiling tool, and a resource to draft survey questions. Within seconds, I received results! And although the outputs all needed editing and additional thought, it did save me time and provided a starting point.
My first opportunity was while using a social listening tool that offered the opportunity to craft a query using AI instead of Boolean search terms (in Beta). In four seconds, it turned my keywords into a thorough Boolean search query. Verdict: OMG, sign me up!
The next AI opportunity occurred while using my audience profiling tool, which has added an AI search engine (also in Beta). This search yielded a response in under six seconds – way faster than it takes to manually build an audience. Despite the fact that the results weren’t perfect, it was usable and a great start. Verdict: Again, sign me up!
The last opportunity was to use AI to create a survey questionnaire to answer a survey goal (again, in Beta). In less than 14 seconds, the tool produced an eight-question survey that addressed my objective and added some additional questions that hadn’t occurred to me. However, it also created a “question of questions” that made absolutely no sense (e.g., the answers to the question were also questions themselves). Additionally, the answer options for some questions were not in accordance with best practices, so the survey would require significant editing. Verdict: Not yet. But I admit it was a big ask!
Bottom Line
Despite the amazing things AI-enabled technology can do, I’m not ready to conclude that it has broken the “good-fast-cheap/pick two” paradigm. However, I’m a firm believer of using AI to support research efforts by speeding up certain processes if there is oversight from human research experts. An article in Forbes summed it up well: “Although algorithms can process information and reveal patterns at scale, they lack the real-world business context, industry knowledge and discernment of experienced analysts.”
For our thoughts on communication and brand strategy, industry trends and more, subscribe to Padilla Insights here.