New York City Council Questions Tech Giants on AI Risks
Lawmakers grilled executives from major AI firms about potential existential threats posed by the technology, as the city considers new regulations.
New York City lawmakers pressed executives from leading artificial intelligence companies on Monday regarding the potential dangers of the technology, including existential risks to humanity. The City Council held a public hearing where representatives from Google, Meta, Anthropic, and OpenAI were questioned about how they quantify the risks associated with AI.
Council Speaker Julie Menin directly asked the executives to describe the worst-case catastrophic scenarios posed by AI. Morgan Dwyer, OpenAI's head of policy development and operations, responded that he didn't know or that it didn't matter, a statement that drew sharp criticism from Menin. "To say you don’t know or [it] doesn’t matter is flippant at best," Menin retorted, questioning how companies can ensure safety without quantifying risk, drawing an analogy to a pharmaceutical company releasing an untested drug.
The hearing featured testimony from AI whistleblowers who expressed concerns about the rapid advancement of the technology. Jacob Coxon, a former researcher at Anthropic and OpenAI, warned the council that "On the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction." He described the companies as being "extremely reckless given the stakes."
Representatives from the tech companies highlighted the potential benefits of AI, such as advancements in medical research and tools for small businesses. Dwyer stated, "AI is making it possible to solve economic, scientific and social problems faster than ever." However, they also acknowledged the inherent risks. "No technology is without risks and there are a wide spectrum of risks. Clearly some risks are unacceptable," Dwyer added.
Logan Graham, who leads Anthropic’s Frontier Red Team, agreed that "if not properly safeguarded, the risks from AI can be extremely high." The executives largely supported the council's proposals for increased protections, including potential cash rewards for whistleblowers.
The council members questioned the executives on third-party validation, potential job losses, and cooperation with immigration officials. However, they did not specifically address the council's proposal for a mandatory "kill switch" or human override for AI systems operating within the city.
Recognizing the limitations of local regulation for globally operating companies, the tech executives emphasized the need for robust federal oversight. Alice Friends, Google's global director of AI policy, stated, "The most effective safeguards are anchored in national technical standards and harmonized across state and federal levels."