Trump Faces AI Safety Concerns Ahead of Xi Jinping Summit

President Donald Trump is set to discuss artificial intelligence with Chinese President Xi Jinping during his upcoming visit to Washington, D.C., on September 24. While Trump asserts the U.S. must maintain its lead in AI over China, a growing chorus of voices from within the AI industry is warning of potential dangers associated with the rapid advancement of these technologies.

Key figures in the AI field, including researchers and CEOs from major labs, have expressed concerns that the pace of development is outstripping the ability to ensure these powerful systems remain safely under human control. Jacob Coxon, a former researcher at OpenAI and Anthropic, resigned from his role with a warning that competition among leading AI laboratories is accelerating the deployment of increasingly capable systems before adequate safety measures are in place. He described the coming year or two as "crunch time for humanity."

Anthropic CEO Dario Amodei has echoed these sentiments, advocating for a deliberate slowdown in the rate at which AI model capabilities are improved to allow safeguards to catch up. He has proposed implementing independent evaluators within frontier labs, establishing coordinated safety standards among democratic nations, and eventually creating international agreements for the most dangerous AI capabilities. Amodei's call has garnered support from other industry leaders, including OpenAI's Sam Altman and Elon Musk. Google DeepMind CEO Demis Hassabis and Microsoft CEO Satya Nadella have also indicated support for a cautious approach, with Nadella stating that superintelligence that is not beneficial to humanity and remains beyond human control is not worth pursuing.

Despite these warnings, some analyses suggest that the urgency surrounding AI safety might be partly influenced by financial incentives, particularly as companies like Anthropic are reportedly preparing for significant IPOs. However, decisions by companies like OpenAI to delay their own IPOs over safety concerns complicate this narrative.

The Geopolitical Context

The U.S. is navigating this complex landscape while preparing for discussions with China, a nation seen as a key competitor in the AI race. Treasury Secretary Scott Bessent highlighted the strategic challenge, stating, "We can’t pause. You can’t, because the Chinese won’t pause." This sentiment is shared by former President Trump, who has dismissed some AI safety warnings as exaggerated and emphasized the critical importance of winning the AI race. "Whoever wins AI wins," Trump has stated, while also suggesting that guardrails can still be implemented.

The Trump administration has pursued initiatives like the Genesis Mission, which aims to link federal scientific data and supercomputing to AI research across multiple agencies, while also issuing directives to ensure AI systems remain controllable and subject to human accountability. Voluntary pre-release federal cybersecurity evaluations for frontier AI models are also being offered.

On Capitol Hill, lawmakers are grappling with how to address AI. House Speaker Mike Johnson has rejected calls for an immediate moratorium on AI development, arguing that such a move could allow China to gain a technological advantage. Others are exploring various regulatory approaches, from bans on superintelligence to requirements for developers to test and disclose catastrophic risks.

The international dimension of AI development is further complicated by accusations from U.S. officials that Chinese AI companies are distilling American AI technology at an industrial scale. China has denied these accusations, calling distillation a common industry practice. Beijing has also pushed back against proposals for slowing AI development, denouncing them as fearmongering and warning that confrontation could disrupt global AI governance.

Amodei himself acknowledges the risk that slowing down AI development in democratic nations could inadvertently allow China-linked projects to advance, potentially creating new dangers. This situation presents a classic security dilemma, where actions taken by one side to increase its security can lead the other side to accelerate, potentially leaving both less secure.

Preparing for the Summit

As the September 24 summit approaches, discussions around U.S.-China AI safety are anticipated, potentially covering areas such as AI-directed cyberattacks and incident information-sharing. However, the challenge of verification remains significant, particularly for AI systems that reside largely in software. Experts suggest that limited agreements on specific dangerous applications might be more feasible than a comprehensive pause.

The core dilemma remains: the U.S. cannot afford to fall behind China in AI development, yet an unrestricted race carries inherent risks. Industry leaders are highlighting the need for control over the technology, while China is simultaneously advocating for cooperation while also pursuing its own technological advantage. The upcoming summit between Trump and Xi presents a critical juncture for addressing these complex strategic and engineering challenges.