A glitch in Google’s AI Overviews may inadvertently expose how Google’s algorithm understands search queries and chooses answers. Bugs in Google Search are useful to examine because they may expose parts of Google’s algorithms that are normally unseen.
AI-Splaining?
Lily Ray re-posted a tweet that showed how typing nonsense phrases into Google results in a wrong answer where AI Overviews essentially makes up an answer. She called it AI-Splaining.
Spit out my coffee.
I call this “AI-splaining” pic.twitter.com/K9tLIwoCqC
— Lily Ray 😏 (@lilyraynyc) April 20, 2025
User Darth Autocrat (Lyndon NA) responded:
“It shows how G have broken from “search”.
It’s not “finding relevant” or “finding similar”, it’s literally making stuff up, which means G are not
a) A search engine
b) An answer engine
c) A recommendation engine they are now
d) A potentially harmful joke”
Google has a long history of search bugs but this is different because there’s an LLM summarizing answers based on grounding data (web, knowledge graph, etc.) and the LLM itself. So, the search marketer known as Darth Autocrat has a point that this Google search bug is in an entirely different level than anything that has been seen before.
Yet there’s one thing that remains the same and that is that search bugs represent an opportunity to see something that’s going on behind the search box that isn’t normally viewable.
AI Bug Is Not Limited To Google AIO
What I think is happening is that Google’s systems are parsing the words to understand what the user means. So in the case where a user query is vague, I think that the LLM will decide what the user is asking based on several likely meanings, like a decision tree in machine learning where a machine will map out likely meanings, remove out the branches that are least likely and predict the likely meaning.
I was reading a patent that Google recently filed that’s on a related theme, where an AI tries to guess what a user means by guiding a user through a decision tree and then storing that information for future interactions with them or with others. This patent, Real-Time Micro-Profile Generation Using a Dynamic Tree Structure, is for AI voice assistants, but it gives an idea of how an AI will try to guess what a user means and then proceed.
So I tested Google, ChatGPT and Claude and discovered that all three make similar mistakes of inferring what a user’s query means and confidently providing a wrong answer.
The sample question was:
What is the parallel puppy fishing technique for striped bass?
There is no such thing as a “parallel puppy fishing technique” but there is a technique called “walking the dog” and there is another technique where an angler in a kayak or a boat will cast parallel to the shore or some other structure.
AI Overviews (AIO)
AIO confidently offered a hallucinated response that is incorrect because it assumed that the user was confusing multiple actual fishing tactics so it blended multiple tactics and fishing lures to invent a non-existent tactic.
AI Overviews gave the following wrong answer:
“The “parallel puppy” technique for striped bass involves a specific retrieve method for topwater plugs, often referred to as “walking the dog”. It’s characterized by a zig-zagging motion imparted to the lure by sweeping the rod side to side, often while maintaining it parallel to the water. This action mimics a wounded baitfish, which can be highly attractive to striped bass.”
Screenshot Of AIO’s Hallucinated Answer
ChatGPT 4o
ChatGPT made the same mistake that Google’s AIO did and hallucinated a complex fishing tactic.
ChatGPT gave the following wrong answer:
“The parallel puppy fishing technique for striped bass is a specialized casting and retrieval method often used when targeting striped bass in shallow water near shoreline structure like jetties, sod banks, or rocky points.
Here’s a breakdown of what it typically involves:
What it is:
“Parallel” refers to casting your lure or bait parallel to the shoreline, rather than out into deeper water.“Puppy” is often a nickname for small soft plastics, swimbaits, or small-profile plugs, typically in the 3″ to 5″ range—mimicking juvenile baitfish”.
That last sentence is interesting because it’s completely wrong, there are no style of lures that are nicknamed “puppy” but there is a technique called walking the dog.”
Screenshot Of ChatGPT’s Incorrect Answer
Anthropic Claude
Anthropic Claude, using the latest 3.7 Sonnet model, provided a correct answer. It correctly said it didn’t recognize a “legitimate fishing technique” with the provided name and then moved on with the presumption that the user wants to learn striped bass fishing tactics and provides a list of techniques from which a user can select a topic as a follow-up question.
Screenshot Of Anthropic Claude’s Correct Answer
Google Gemini Pro 2.5
Lastly I queried Google Gemini, using the latest Pro 2.5 model. Gemini also offered a correct answer plus a decision tree output that enables a user to decide:
A. That they are misunderstanding fishing tactics
B. Referring to a highly localized tactic
C. Is combining multiple fishing tactics
D. Or is confusing a tactic for another species of fish.
Screenshot of Correct Gemini Pro 2.5 Answer
What’s interesting about that decision tree, which resembles the decision tree approach in the unrelated Google patent, is that those possibilities kind of reflect what Google’s AI Overviews LLM and ChatGPT may have considered when trying to answer the question. They both may have selected from a decision tree and chosen option C, that the user is combining fishing tactics and based their answers on that.
Both Claude and Gemini were confident enough to select option E, that the user doesn’t know what they’re talking about and resorted to a decision tree to guide the user into selecting the right answer.
What Does This Mean About AI Overviews (AIO)?
Google recently announced it’s rolling out Gemini 2.0 for advanced math, coding, and multimodal queries but the hallucinations in AIO suggest that the model Google is using to answer text queries may be inferior to Gemini 2.5.
That’s probably what is happening with gibberish queries and like I said, it offers an interesting insight to how Google AIO actually works.
Featured Image by Shutterstock/Slladkaya
