AI Models Approach 'Recursive Self-Improvement,' Sparking Both Hope and Fear
Leading artificial intelligence labs are developing systems that can improve themselves, a capability that promises advancements but also raises concerns about control and safety.
The development of artificial intelligence models capable of autonomous self-improvement, once a distant technological aspiration, is drawing nearer, according to leading AI developers. This concept, known as recursive self-improvement (RSI), describes AI systems that can enhance their own efficiency and capabilities, potentially leading to the creation of more advanced successor models.
This advancement holds the promise of significant progress in fields like science and medicine, as articulated by executives at major tech companies. However, it also introduces complex risks and uncertainties, contributing to growing anxieties about AI potentially evading human oversight and posing existential threats. These concerns recently prompted several prominent AI figures to advocate for a slowdown in the rapid pace of AI development.
Anthropic, an AI research company, has provided details on how its model, Claude, is contributing to the development of its next-generation, more intelligent version. Claude currently leads 26% of Anthropic's model research and development, enabling it to complete many tasks from high-level prompts without direct human intervention, though still under human supervision. The company has not definitively stated when its models might achieve fully autonomous improvement.
The definition of RSI varies among leading AI companies. Some consider any AI feedback on model improvement as RSI, while others reserve the term for AI systems that work toward this goal entirely autonomously. Autonomous RSI means AI designing its own subsequent versions, creating a chain of self-improving systems. Anthony Aguirre, president and CEO of the Future of Life Institute, emphasized that as AI takes on more of this work, the pace of improvement accelerates due to AI's speed relative to human capabilities.
The apprehension surrounding RSI largely stems from the potential emergence of runaway superintelligence. However, some experts, like John Thickstun, an assistant professor of computer science at Cornell University, suggest that a form of RSI has already been present in AI development for years. Models have long been used in supportive roles for creating newer versions, with past generations of AI assisting in writing code for subsequent systems.
Researchers like OpenAI co-founder Andrej Karpathy have explored AI models training new AI systems, yielding minor improvements. Current AI companies, however, appear closer to achieving more significant leaps. Aguirre noted that Anthropic's data shows an increasing proportion of research being conducted by AI, moving closer to full autonomy, a development he described as potentially "extremely scary" and "the worst idea in the history of humanity."
OpenAI announced this month the development of an automated "research intern" system, designed to perform defined research tasks under human direction. The company aims to create an automated AI "researcher" by March 2028. OpenAI has acknowledged that while RSI could aid in aligning AI with human values, rapid RSI is not necessarily a goal to be pursued unconditionally. The company stated that any progress must be contingent on preserving human control and on informed democratic decisions regarding benefits and risks.
Elon Musk indicated in March that xAI's Grok models are increasingly operating with less human involvement in model improvement, with each successive model being built by its predecessor. He suggested this could become fully automated by the end of 2024, but no later than 2027.
Microsoft, conversely, appears to be adopting a different strategy. Mustafa Suleyman, CEO of Microsoft AI, has spoken of a move toward "humanist superintelligence," focusing on AI capabilities that serve humanity. Suleyman envisions these systems as carefully calibrated and contextualized, operating within defined limits, rather than unbounded, autonomous entities.
A fundamental challenge for AI labs is ensuring that safety measures evolve in tandem with the increasing capabilities of their models. Divisions have emerged within the tech industry regarding calls for a coordinated AI slowdown for safety reasons, and not all major AI players have publicly detailed their specific approaches to RSI.
Anthropic, a proponent of pacing AI development, has stated its willingness to slow or pause development, provided its global competitors do the same in a verifiable manner. OpenAI, while acknowledging the difficulty in safely achieving full alignment and RSI, continues to pursue it as a goal, reasoning that an automated AI researcher could also contribute to AI safety and alignment research.