New Report Details Secretive Race for Self-Improving AI, Offers Insights for Regulators
A firm called Gladstone AI has released a report aiming to clarify recursive self-improvement (RSI) for lawmakers, detailing how AI companies are automating their development pipelines.
AI companies OpenAI and Anthropic have been vocal about the potential dangers of artificial intelligence achieving recursive self-improvement (RSI), a state where AI can enhance its own capabilities. However, the specifics of their efforts in this area have remained largely undisclosed. A new report from the firm Gladstone AI aims to illuminate this secretive race, providing a detailed look at how RSI might function and the progress AI companies are making toward it. The report is based on interviews with researchers working within leading AI firms.
According to the report, AI companies are actively working to automate every stage of the AI development process, including idea generation, experimental testing, and result interpretation. Several insiders believe that RSI could begin within the next three to twelve months, potentially sooner, unless governmental intervention slows the pace. Some researchers in the field are reportedly preparing for potential negative outcomes, with some engaging in "disaster prepping and bio-bunker purchasing."
Gladstone AI's stated goal is to equip lawmakers with a better understanding of RSI, thereby enhancing their ability to regulate it. The firm intends for the report to facilitate a more informed discussion about the process of AI development, particularly in Washington, D.C. Instead of focusing solely on broad bans on superintelligence, the report aims to pinpoint specific stages within the development process that require regulatory oversight.
How RSI Could Work
The report outlines a simplified model of the current AI research pipeline and how companies are automating it. In the initial phase of AI model training, known as pretraining, the focus is on extracting more intelligence from models while minimizing the use of computational power, or "compute."
AI companies typically employ a portfolio approach to testing new ideas. Researchers allocate limited compute to various concepts, training a series of models for each. Models that consistently demonstrate improved performance as they scale are selected for further training with more substantial compute resources. The availability of compute is a limiting factor, restricting the number of promising ideas that can be tested at larger scales. This means that even if an AI can generate promising ideas, the efficiency of filtering good ideas from bad ones remains crucial. If extensive time or compute is required for filtering, human researchers with intuition may continue to be more effective.
Conversely, companies with greater access to compute can test a wider array of ideas. This could give them an advantage in achieving RSI first. With more computational resources available for experiments, these companies may be more likely to identify rare and highly sought-after breakthroughs: ideas that improve with increased compute, even if they appear unremarkable at smaller scales. Identifying when such advancements occur, particularly with AI assistance, is a critical aspect of the process that currently lacks transparency for legislators.
The gradual nature of RSI makes regulation challenging. The detailed description in the report aims to enable identification of specific points in the process where human oversight is essential. This approach seeks to allow for targeted regulation rather than broad, potentially ineffective, prohibitions.