Every tenth response from the Gemini 3 algorithms contains factual errors or fabricated data
A study by The New York Times highlighted a critical issue in AI responses: the Gemini 3 algorithm produces false information in 9% of cases. Despite technological upgrades, the system continues to generate millions of fakes daily.
This is reported by RBC-Ukraine, citing the study.
More interesting news: Popular AI can reveal passwords and correspondence: what is known about the critical vulnerability
Results of the SimpleQA test
To assess accuracy, the experiment used the SimpleQA benchmark from OpenAI – a list of 4,000 questions with verified answers. The study showed that accuracy increased from 85% in Gemini 2.5 to 91% in Gemini 3. However, if this error percentage is extrapolated to all search queries, Google produces tens of millions of incorrect answers daily.
The report provides examples of the algorithm’s failures:
- Bob Marley case: The AI referenced three pages, two of which did not contain dates, and the third (Wikipedia) had conflicting data – Google confidently chose the wrong year.
- Yo-Yo Ma case: Referring to the website of the Classical Music Hall of Fame, the AI simultaneously claimed that such a cellist “does not exist.”
Google’s reaction: “We see serious gaps”
The tech giant sharply criticized the analysis results. Google representative Ned Adrians stated: “This study has serious holes. It does not reflect what people actually search for on Google.”
The company believes that the SimpleQA test itself contains incorrect information.
It also turned out that the tech giant uses different models depending on the user’s query. To load results faster, the system most often uses the Gemini Flash model, which is cheaper and faster but less accurate than Gemini 3.1 Pro.
The key issue, according to Google, is that AI Overviews encourage people to accept short summaries instead of checking primary sources through “blue links.” The company itself adds a disclaimer: “AI can make mistakes, so please verify answers.”
Developers clarified that you can review privacy and search settings in your browser from the “Settings” menu.
Read even more interesting news:
- OpenAI, Google, and Anthropic unite against China to protect their neural networks
- Secret release from Google: a new app will turn chaotic thoughts into professional reports
Source: rbc.ua +rel=”nofollow”
