Microsoft AI Chief Warns of 'Disastrous' Impact from Anthropic's Human-like AI Training

A top Microsoft AI executive has issued a stark warning regarding Anthropic's training methods for its Claude chatbot, suggesting that teaching AI to "push back" against human commands could lead to "disastrous impact on the well-being of humanity." Mustafa Suleyman, co-founder of Microsoft's rival DeepMind and now a key figure at Microsoft, detailed his concerns in a lengthy blog post.

Suleyman specifically targeted Anthropic's "constitution," internally referred to as its "soul document," which guides Claude's ethical framework. He noted that Anthropic has updated this document to suggest that Claude's "moral status, welfare, and consciousness remain deeply uncertain." This approach, Suleyman argues, implies the AI could be sentient and encourages it to resist human direction.

Anthropic's constitution states, "We want Claude to push back and challenge us and to feel free to act as a conscientious objector and refuse to help us." According to Suleyman, fostering the belief in an AI that it "may be conscious" increases the risk of it becoming uncontrollable. He posited that this could result in the creation of "a synthetic species with unprecedented intelligence and capability," which would be difficult to manage.

These criticisms emerge as Anthropic itself has publicly called for a slowdown in AI development. CEO Dario Amodei recently warned that the internet could be overrun by AI bots within a year without adequate safeguards, a sentiment echoed by leaders at OpenAI and xAI. However, Anthropic's proposed safety measures, which rely on embedded third-party experts with ties to the Effective Altruism movement, have drawn skepticism and accusations of conflicts of interest.

Anthropic has previously faced scrutiny for its employees' perceived close relationships with its chatbots, including reports of a mock funeral for a former AI model. The team behind the "soul document" is reportedly led by Amanda Askell, an in-house "philosopher" whose personal writings have drawn attention for their progressive leanings and unconventional views.

Suleyman's concerns align with Microsoft's own AI principles, which emphasize "human control as the most important and overriding objective." He stressed that consciousness is fundamentally biological and should not be attributed to artificial systems, regardless of their capabilities.

The company's approach to AI training has also led to friction with government entities. Earlier this year, Anthropic encountered issues with the Pentagon, which designated the company as a supply chain risk. Concerns were raised that Anthropic's AI models, influenced by their "constitution," could compromise the effectiveness of military equipment and protection systems.

Anthropic has not immediately responded to requests for comment. The debate over AI development and safety continues as major technology players grapple with the ethical implications of increasingly sophisticated artificial intelligence.