AI Researchers Warn of Superintelligent Machines, Urge Caution on China

Artificial intelligence researchers are expressing growing concern about the rapid advancement of AI, with some warning that humanity is on track to create a "new species" of superintelligent machines. The debate, featuring prominent AI optimist Oren Etzioni and AI "doomer" Max Tegmark, highlighted the potential for AI to surpass human control and the critical need for regulatory oversight.

Tegmark, founder of the Future of Life Institute, voiced surprise that authorities have not intervened more forcefully as companies pursue the creation of superintelligence. "Companies are just openly saying, well, we’re going to build super intelligence. [It] might be a new species," Tegmark stated, emphasizing the potential turning point when AI surpasses human capabilities in AI research.

Both Etzioni, a professor at the University of Washington, and Tegmark agreed that a key strategy for managing AI risks is to limit machine access and power. Etzioni advocated for designing systems with carefully defined, narrow intelligence, rather than broad, general agents. "By default, machines should be given the least amount of access and power that they need to do their job," Tegmark added, drawing an analogy to not giving sensitive information to individuals without a need-to-know.

The discussion also touched upon the geopolitical implications, particularly concerning China. Etzioni expressed concern that the United States rushing to implement stringent AI regulations could inadvertently allow China to gain a technological advantage. "I don’t trust the Chinese at all," Etzioni said. "Nothing would make them happier than us hamstringing ourselves while [they take] the front seat… We need to be mindful of the adversarial race with China, not delude ourselves that our interests are aligned."

Tegmark, however, suggested that China might also recognize the risks of uncontrolled superintelligence. "We don’t have to trust China," he argued. "They will want to do it anyway."

Calls for regulation have grown, with AI companies like Anthropic and OpenAI advocating for oversight. However, Etzioni dismissed the idea of self-regulation by major tech firms as "a joke," comparing the situation to the historical struggles with Big Tobacco and the need for external discipline.

Etzioni also voiced anxieties about the political process potentially co-opting AI regulation for populist gains. He expressed concern that fear-driven, short-sighted solutions could emerge from the democratic process, warning, "I am terrified of what the democratic process does, particularly when it sets its mind to regulate something that we don’t understand, particularly when it does so out of fear."

Tegmark, conversely, has become more optimistic as AI safety concerns gain mainstream attention. He noted that a recent "Pro-Human Assembly" in Washington D.C. saw bipartisan agreement on the need for AI regulation, bringing together figures across the political spectrum.

The debate between the two researchers underscored the rapid and disorienting pace of AI development, a speed that challenges even the experts in the field.