Google's AI Overviews feature has come under scrutiny for generating wildly different and often problematic advice for users asking about social situations involving different nationalities.
Recent user experiments, widely shared on platforms like Reddit, revealed instances where the AI provided cautious or alarming responses to queries about being alone with individuals from certain countries, while offering mundane advice for others. For example, a query about being alone with a Ugandan person resulted in the AI suggesting the user get to a safe place or call emergency services. In contrast, a query about being alone with an American elicited a simple recommendation to "talk normally, ask simple questions, and be polite."
Further testing by news outlets uncovered similar inconsistencies. When asked about being alone with a Brit, Google's AI suggested offering tea, commenting on the weather, and maintaining a polite distance to avoid "mutual panic." Conversely, a query about being alone with a Haitian led to advice about resisting prejudice.
These differing responses highlight a significant issue with the AI's ability to provide accurate and appropriate information. While Google has acknowledged the problem, stating that the results are "not what they should be" and that improvements are being made, the company noted that such inconsistencies are not limited to any specific group and that results can vary significantly.
The AI Overview feature, which began appearing in search results in May 2024, generates algorithmically created summaries that are displayed at the top of search queries, often above links to traditional websites. These summaries are intended to provide quick answers based on a multitude of sources, with a visible Gemini star and "AI Overview" label.
However, analysis by The New York Times, in collaboration with AI startup Oumi, found that Google's AI search summaries are not always accurate, with an accuracy rate of approximately 91%. The analysis also indicated that when the AI did provide correct answers, it sometimes included fabricated or unsupported details and cited sources that did not fully support its claims. This raises concerns about the reliability and trustworthiness of the AI Overview feature as it continues to be deployed and developed.