Google admits its AI Overviews can generate “some odd, inaccurate” results

Must read

Google goes all in on AI at developer event

Google goes all in on AI at annual developer conference 04:47

Google on Thursday admitted that its AI Overviews tool, which uses artificial intelligence to respond to search queries, needs improvement.

While the internet search giant said it tested the new feature extensively before launching it two weeks ago, Google acknowledged that the technology produces “some odd and erroneous overviews.” Examples include suggesting using glue to get cheese to stick to pizza or drinking urine to pass kidney stones quickly. 

The rollback is the latest instance of a tech company prematurely rushing out an AI product to position itself as a leader in the closely watched space.

Because Google’s AI Overviews sometimes generated unhelpful responses to queries, the company is scaling it back while continuing to make improvements, Google’s head of search, Liz Reid, said in a company blog post Thursday. 

“[S]ome odd, inaccurate or unhelpful AI Overviews certainly did show up. And while these were generally for queries that people don’t commonly do, it highlighted some specific areas that we needed to improve,” Reid said.

Nonsensical questions such as, “How many rocks should I eat?” generated questionable content from AI Overviews, Reid said, because of the lack of useful, related advice on the internet. She added that the AI Overviews feature is also prone to taking sarcastic content from discussion forums at face value, and potentially misinterpreting webpage language to present inaccurate information in response to Google searches. 

How to use AI as a tool 06:23

“In a small number of cases, we have seen AI Overviews misinterpret language on webpages and present inaccurate information. We worked quickly to address these issues, either through improvements to our algorithms or through established processes to remove responses that don’t comply with our policies,” Reid wrote. 

For now, the company is scaling back on AI-generated overviews by adding “triggering restrictions for queries where AI Overviews were not proving to be as helpful.” Google also says it tries not to show AI Overviews for hard news topics “where freshness and factuality are important.”

The company said it has also made updates “to limit the use of user-generated content in responses that could offer misleading advice.”

Megan Cerullo

Megan Cerullo is a New York-based reporter for CBS MoneyWatch covering small business, workplace, health care, consumer spending and personal finance topics. She regularly appears on CBS News 24/7 to discuss her reporting.

More articles

Latest article