Former Anthropic Researcher Warns of AI Extinction Risks

Jacob Coxon, a former researcher at Anthropic and OpenAI, has stated that AI staff are genuinely frightened for the future of humanity. Speaking to the BBC, Coxon warned that there is a strong chance people could die in the immediate future if the current rate of AI progress does not slow down. He suggested that a coordinated slowdown with China is necessary to avoid an international race. Coxon noted that a scenario involving a swarm of bots acting as a supercomputer to take over the internet could be realistic within six months to a year.

These warnings align with an essay titled We Must Pace the Frontier published by Anthropic head Dario Amodei. Amodei proposed a three point plan involving independent monitoring of AI models, industry wide regulation, and global regulation. An Anthropic spokesperson stated that the company continues to build models with strong safeguards and supports a lawful, verifiable way to pace the release of powerful models.

Other industry figures have dismissed these concerns. Nvidia CEO Jensen Huang has previously called the idea that AI will end humanity complete nonsense and reportedly dismissed Coxon's comments as untrue. Clement Delangue, CEO of Hugging Face, questioned the perspective of asking Coxon about extinction risks, though he later offered to help with Amodei's proposed solutions. Some critics suggest that warnings about AI perils may be designed to generate hype or to trigger regulations that would create a duopoly for OpenAI and Anthropic.

While a center leaning source provided a detailed account of the warnings and the subsequent industry debate, a center left leaning source framed the event as the latest warning about the technology's potential to annihilate humanity.