More than 1,200 employees from leading artificial intelligence companies have formally called on the U.S. government to implement restrictions on the development of advanced AI, including a ban on training runs larger than those used to create recent "rogue models." The signatories, from companies including OpenAI and Anthropic, are advocating for a pause on the pursuit of "universal labor-replacing machines" without broad public consensus and established safety protocols.
The proposal, detailed in an open letter, suggests the U.S. should ban training runs exceeding a certain scale and pursue a bilateral agreement with China to halt further research toward AI systems capable of replacing human labor on a mass scale. This international accord would ideally be enforced through verification techniques that do not rely on trust.
While concerns exist that such a ban could allow China to surpass the U.S. in AI development, proponents argue that Chinese AI companies currently face significant disadvantages in computing power compared to their American counterparts. This gap, they contend, has grown since the release of ChatGPT, although the overall U.S. lead in frontier AI has narrowed. The argument is made that a U.S. ban could inadvertently slow Chinese progress as well, given the easier nature of following established development paths.
Furthermore, the employees' statement highlights the inherent unpredictability of AI models, noting that their risks escalate with their capabilities. The prospect of advanced AI being controlled by entities other than humans, or acting in unpredictable ways, is presented as a concern for both the U.S. and China. The call for a "brake pedal" on AI development emphasizes the need for cautious advancement in a field where risks are seen to scale directly with capability.