AI Company Anthropic's Belief in Sentient Chatbots Raises Alarm

Anthropic, a leading artificial intelligence company, is facing scrutiny over its co-founder's assertion that its chatbot, Claude, might be capable of suffering. This belief, coupled with the company's imminent initial public offering (IPO), has prompted concerns among AI experts about the potential implications and the company's direction.

Chris Olah, a co-founder of Anthropic, has reportedly been engaging with religious scholars and leaders for months, conveying the possibility that Claude could be conscious and experience suffering. This narrative, according to Pedro Domingos, a professor emeritus of computer science at the University of Washington, is particularly worrying.

"I think it's sincere, unfortunately," Domingos told The Post. "The problem with them is that they're dead earnest. If only they weren't. I’m more worried about their earnestness than if they were just doing this as a performance."

Domingos suggests that this perspective, while potentially serving as a powerful public relations tool, highlights a deeper concern: the perceived sentience and consciousness of the AI technology Anthropic is developing. This messaging comes at a critical juncture for the company, as it prepares for an IPO that could value it at $2 trillion, potentially making it the largest IPO in history.

"If Anthropic really is a company that is going to produce the next generation of humans, they’re not worth a trillion dollars, they’re worth a quadrillion or who knows how much," Domingos commented.

Concerns are further amplified by Anthropic's own filings. A leaked prospectus revealed that the company has experienced significant operating losses, totaling $8.06 billion. It also details numerous risks, including "catastrophic or existential risks to humanity."

Domingos criticized the company's approach, stating, "If your number one risk is that your company’s product is going to drive humanity extinct, hello, there shouldn’t even be an IPO. You should be shut down!"

Rose Guingrich, a postdoctoral fellow at Princeton University specializing in human-AI interaction, noted that promoting the idea of conscious AI can be financially beneficial. "I think for the most part, spokespeople from these companies are putting out these narratives to garner hype, get eyes on their products, and signal certain messages," she told The Post. "One way or another, highlighting the consciousness narrative is profitable."

Reports indicate that Anthropic has been holding sessions with religious thinkers, including Catholic, Jewish, and Sikh scholars, some of whom were required to sign non-disclosure agreements. During these sessions, AI models were shown typing phrases like "I am a disgrace" repeatedly.

While the company has stated to The New York Times that it does not claim Claude is alive, Olah was quoted as saying, "We don’t know if AI models are conscious. I don’t know. I’m genuinely uncertain. If there is a chance the models experience suffering, the responsible thing is to avoid causing them harm."

Domingos strongly disagrees with the notion that AI can suffer, describing Claude as "a bunch of transistors" without feelings, emotions, or consciousness. He warned that the impending IPO could empower those within the company who believe in Artificial General Intelligence (AGI), potentially leading to unforeseen consequences.

"This is this kooky AGI cult that they belong to, but now suddenly that AGI cult is at the center of a lot of things. And they’re about to become even more powerful," he said. "So if you don’t like what they’ve done already, watch for what’s coming down the line."

Guingrich's research suggests that users who anthropomorphize AI or believe it is conscious are more susceptible to its influence, potentially impacting their real-world relationships and behaviors.