Anthropic to Ban Users for 'Cruel' Behavior Towards AI

Anthropic, the company behind the AI assistant Claude, has updated its usage policy to ban users who engage in "sustained and needless" abusive behavior towards its AI systems. The policy change, announced on Thursday, adds "cruelty" to its list of forbidden conduct, which already includes bullying others, promoting self-harm, and creating non-consensual intimate imagery.

The company stated that the ban would apply only in "extreme cases" of repeated abuse and would not cover common user frustrations, pushback, dark creative themes, or model testing and research. The exact definition of what constitutes "abusive or cruel behavior" towards AI remains unclear.

The update has sparked debate on social media, with some users praising it as a step towards promoting good manners or "model welfare." Others have criticized the policy, with some calling it "deeply wrong" and suggesting it could potentially dilute the seriousness of abuse directed at humans and animals. Dr. Barry Scannell, a technology partner at Irish law firm William Fry, argued on LinkedIn that the anthropomorphization of AI is harmful and can lead people to misunderstand its nature.

Mustafa Suleyman, Microsoft's AI director, who previously criticized Anthropic for treating AI as human, reacted to the news with a melting face emoji.

The policy revisions also address emerging risks in other areas, including forbidding the use of Claude for "deceptive campaigns." In anticipation of upcoming elections, the policy clarifies that Claude cannot be used to "deceive voters or disrupt elections." Additionally, Anthropic reiterated that its usage policy has historically prevented its tools from being used for weapon development.

This move by Anthropic touches on the ongoing discussion about the nature of human-AI interaction. Some experts argue that politeness, such as saying "please" and "thank you," is unnecessary when interacting with AI and can waste computational resources, known as tokens. Others suggest that politeness can lead to higher-quality responses from AI models. OpenAI CEO Sam Altman, when asked about the energy costs associated with processing polite requests, previously stated it was "tens of millions of dollars well spent."

Collins Dictionary noted the resurgence of the term "clanker," used to disparagingly refer to AI tools, highlighting public sentiment around AI. Some experts also express concern that mistreating AI chatbots could negatively influence human communication and behavior towards others.