Whistleblower to Testify at NYC Hearing on AI Regulation

Jacob Coxon, a former researcher at Anthropic, is expected to testify at a New York City Council hearing on Monday regarding artificial intelligence regulation. Coxon previously issued a stark warning about the rapid advancement of AI, stating it could pose existential risks. His testimony comes at the request of Council Speaker Julie Menin, who is encouraging AI whistleblowers to share their insights as the city considers new legislation.

Joining Coxon at the hearing are Alex Turner, a former researcher from Google DeepMind, and Daniel Kokotajlo, an ex-OpenAI researcher who now leads the AI Futures Project. Representatives from major AI companies including Anthropic, OpenAI, Google, and Meta are also slated to attend. These companies will be sending policy leads and other executives, rather than their chief executive officers.

The tech giants reportedly agreed to participate after the City Council indicated it might use subpoena power to compel their attendance. The hearing will address a series of proposed bills aimed at regulating AI within New York City. These proposals include mandating "kill switches" for AI systems, defined as a human override mechanism capable of shutting down the technology. The city is also considering a first-of-its-kind cash reward program for whistleblowers, funded by fines collected from non-compliant companies.

Coxon's departure from Anthropic last month and his subsequent warning about AI's potential to cause widespread harm have intensified discussions surrounding AI safety and regulation. In response to such concerns, leaders from major AI firms, including Anthropic CEO Dario Amodei, have called for more stringent regulations and international cooperation to manage the development of increasingly sophisticated AI models. Concerns about "recursive self-improvement," where AI systems become capable of self-enhancement, have been voiced by figures like Sam Altman of OpenAI and Bill Gates. However, some, like former President Trump, have expressed concern that slowing AI development in the U.S. could cede a strategic advantage to other nations, such as China.

New York City's legislative efforts are part of a broader trend of governmental bodies seeking to regulate AI. New York State recently announced that AI companies will be required to register with the state and report critical safety incidents. The proposed city laws would impose a $25,000 fine for violations of the vetting or kill switch mandates. Officials from the Mayor's administration, including representatives from the Department of Consumer and Worker Protection and the tech office, are also scheduled to testify.