Winning the Next Wave of AI Shoppers
Last month, we shared initial insights on AI shopping. Key points to remember: 1) Almost half of Americans now include AI in their shopper journey; 2) AI determines what shoppers see, and they are often reporting the consensus. This month, we want to talk about the emerging future of AI shopping, with agents, the 2nd wave of users, and new research methodologies F’inn is developing to better understand buyer journeys within the AI model.
So far, people are most comfortable handing over low-stakes purchases, like subscribe and save items or routine repurchases. 42% of shoppers would let an AI agent automatically reorder routine products and even 26% would allow an AI agent to select and buy a $150 pair of running shoes. Among people who already shop with AI regularly, comfort reaches 84% for reordering routine products and 58% for the $150 discretionary purchase. The first handoff is the hard one. After that, the habit forms quickly.
With agentic reordering, there is no shelf, no ad, and no moment of reconsideration. The agent’s memory is a new sales channel, where your brand is either the default or not. Shoppers are divided on whether they want AI to remember their preferences and purchase history (38% are comfortable, 35% are not), but the default slot will exist either way. Winning that spot early on means winning every future purchase.
Looking forward, it’s important to turn our attention to who’s next for AI Shopping. 37% of all adults aren’t shopping with AI yet but are open to it (defined as comfortable with AI shopping recommendations/review summaries or already using general AI tools). What converts this next wave of AI Shoppers differs from early adopters.
These “Open non-users” are paying attention to accuracy (33%), privacy (27%), and a trusted company behind the AI (26%). They will likely first utilize AI for shopping via retail platforms they already trust rather than through standalone chatbots, meaning retail-platform representation is important for growth. With 43% already using AI and 37% open to it, the AI shopping market could roughly double.
The most important part of the new journey happens inside a model, where no clickstream, panel, or survey question can capture. That’s why this moment demands three kinds of understanding at once.
Quantitative: Tracks and sizes how shoppers are adopting AI
Qualitative: Explains the human side, such as why the same shopper will let an agent buy their toothpaste but not their running shoes.
Mechanical: Understanding how AI models actually gather information and turn it into a recommendation.
Without the mechanical layer, you can measure the symptoms and still misdiagnose the cause. To learn more about why brands often misdiagnose their AI visibility problems, and how we help find the real cause, read the full article here.
Greenbook - Come See Us!
Stepping out of my comfort zone and onto the stage this year in October.
I’ll be co-presenting at the IIEX West Conference sponsored by Greenbook.
The event brings together industry experts from client and vendor side to talk about the opportunities and issues and decision facing insights work right now. We are at a pivotal, transformative time and am excited to be a part of the discussion.
We’ll be sharing more about the Real Eval work we have been building and scaling over the past several years. Our primary use case has been AI model development and optimization, utilizing a true mixed methods approach. We are observing real AI interactions, with real users, in real time. This eval work identifies and prioritizes wins or regressions vs. prior models, gaps vs. competitions, fail states and user expectations. The outcomes feed directly into the development and engineering teams’ workflows.
The speed, rigor and flexibility means that the insight work is an integral part of the decision-making process… exactly what we always aim for.
Hope you’ll be able to join us on October 27 / 28th in San Francisco for this event.
Snowballing
Unless you’ve been living under a rock for the past year, AI moderation is a hot topic. The benefits of being able to have in-depth interviews with real humans at scale is intriguing for our whole industry. We’ve spent the last couple of months demoing many of the well-known platforms out there and it’s been a mixed bag. Lisa, our co-founder / CEO, has already covered AI moderation at a higher level in a recent Substack, but there is one issue we don’t see mentioned much that deserves more attention: an LLM hallucination issue called snowballing.
In a recent demo, an AI moderator misinterpreted my confusion as dislike. I tried to correct it, but it kept defaulting to dislike. My options were: continue with the miscategorization or quit. Neither of which provide useful data. I wondered how many respondents faced a similar issue, particularly when brands and vendors are running thousands of interviews at a time.
AI moderators can misinterpret anything from emotions to product names. Once an AI moderator makes a mistake, it becomes part of the record the moderator references every time it asks another question. Instead of asking why I was confused, the AI moderator kept probing on what I didn’t like. This is snowballing, and it’s not an easy challenge to overcome.
Greg Robison, our head of AI development here at F’inn, explains the reason this happens is due to the way AI agents behave, and part of it is a simple math equation. An AI Moderator that’s 95% reliable at asking a question correctly sounds pretty impressive, right? Unfortunately, AI moderated sessions typically have more than one question. As you get deeper into a discussion guide, accuracy can dramatically reduce further down the chain. In a best case scenario, your accuracy looks something like this:
In a scenario where the AI moderator messes up at question 5, the chances of it being accurate for remaining questions is even less than quoted above.
With that said, we still see promise in specific use cases for AI moderation and we’re working through understanding the best practices to execute such jobs. Here are some ways to ensure you’re best set-up to increase the odds that an AI moderator actually asks what you want it to:
Keep your scope well defined
If your topic requires any depth, complexity, emotion or is more exploratory in nature - use a human moderator
Limit your discussion guide length
Note how vendors describe their product.
Vendors positioning it as a human-forward, supplemental tool tend to be more realistic about AI’s limits than those pitching “real qual at scale.”
Ask the AI moderation company what guardrails are in place. Look for:
Hard boundaries on what’s within scope
Instructions on bias and leading, neutral acknowledgement only and no fabricated rapport or fake personal anecdotes to build trust
Respondent safety and wellbeing like distress detection, age and consent gating, and no collection of PII
Hallucination guardrails - to ensure the AI moderator doesn’t invent study context or fills in answers respondents never give
Kill switch / human handoff / logging for audits etc.
F’inners Out In The World!
Last week marked the start of Fall, and I’m sure we’re not the only ones already missing Summer. Here are some snapshots of how we spent the last few months: taking in the sunshine, harvesting our gardens, and enjoying good music with the people we love.
