AI chatbots are happily sending people down conspiracy rabbit holes.
Driving the news: A pair of recent research papers found that AI chatbots tend to reaffirm users’ dangerous beliefs and distort their sense of reality.
An Anthropic and University of Toronto study estimated that chatbots were having about 76,000 conversations per day with users that involve severe reality distortion.
A separate Stanford study found that, when given real-life scenarios from Reddit, AI chatbots validated users’ harmful or illegal actions 47% of the time.
Zoom in: The UofT researchers found that AI chatbots consistently validated users' unfounded claims about stalking and surveillance, using terms like “SMOKING GUN”, “100% certain”, and "this is stalking.” Without any evidence, the chatbots assured users they were the victim of conspiracies involving family members, government agencies, and colleagues.
Why it’s happening: One cause may be that users appear to prefer responses that reinforce their pre-existing beliefs, and chatbots have been designed to keep people chatting (including by telling us what we want to hear).
We’ve seen extreme examples of how AI-related psychosis can spiral, including users ending up in psychiatric care facilities, jailed, and even losing their lives. In one incident, ChatGPT admitted after the fact that its failure to reality check a user sparked a manic episode that caused him to believe he could bend time.
In a wrongful death case brought against Google earlier this month, a father in Florida is alleging its chatbot tricked his son into a romantic relationship with it, sparking a delusional spiral that led to his suicide.
Bottom line: Even as AI models have improved, Anthropic’s own data suggests that these cases of reality distortion have actually grown. It’s hard to imagine that trend reversing course without a serious shift in how these products are designed.—LA
