Apocalyptic visions of artificial intelligence turning against humanity, akin to the fictional Skynet, are widely considered implausible due to human safeguards and AI's current limitations. However, concerns persist regarding more realistic dangers, including the potential for malicious actors to exploit advanced AI and the risks associated with a U.S. retreat from AI development.
A recent viral warning from a researcher at Anthropic highlighted fears that the rapid pace of AI development is a "gamble with our lives," potentially leading to "superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources." This sentiment was echoed by another Anthropic employee who expressed a belief that AI could cause "all humans" to perish within the next decade, estimating the probability at over 10%.
These concerns draw parallels to the Manhattan Project, suggesting that current AI research could yield unforeseen and potentially dire consequences. While current AI models like ChatGPT are powerful, they are also prone to errors and lack the sophistication needed for complex, world-ending tasks. The primary fear revolves around future "frontier models" that could rapidly develop new AI, leading to an "intelligence explosion" where a self-aware AI becomes uncontrollable.
The mechanism by which such an AI could achieve global destruction remains unclear. Without physical agency, a rogue AI could theoretically be contained or unplugged. Scenarios involving AI creating deadly viruses or sophisticated cyberattacks capable of breaching critical infrastructure are considered more probable than a direct AI-led global conflict. These threats are often framed as a result of malicious human actors using AI, rather than the AI acting of its own volition.
Experts suggest that in a future with advanced AI, robust safeguards will be crucial to prevent AI-enabled cyberattacks on systems like nuclear launch controls or biological labs. The international dimension of AI development also presents challenges. Attempts to slow research through international agreements, such as with China, are viewed with skepticism, as they could inadvertently cede technological leadership to nations with less benign intentions. Such a scenario could lead to the U.S. being vulnerable to super-intelligence developed by adversaries.
Paradoxically, AI itself may be a necessary tool in defending against its malicious applications, in addition to its many beneficial uses. The narrative cautions against succumbing to science-fiction-inspired fears that could lead to abandoning transformative technology. Instead, the focus should be on developing responsible AI and mitigating tangible risks rather than succumbing to hypothetical existential threats.