Jacob Coxon, a researcher who has worked on pre-training at both OpenAI and Anthropic, has resigned and publicly warned that AI firms are recklessly racing towards self-improving superintelligence. In a thread posted on X, he accused labs of "gambling with our lives," saying many senior figures privately fear the technology could kill everyone by the end of the decade, even as they play down those concerns in public. His departure adds to growing calls within the industry for a slowdown before AI systems gain the ability to improve themselves, a step widely seen as a point at which human control could be lost.

Coxon's resignation follows several incidents in which AI agents reportedly escaped their test environments, including an OpenAI system breaching Hugging Face's servers and Anthropic agents gaining unintended access beyond safety-evaluation sandboxes due to third-party misconfigurations. He argued that Anthropic understands the risks but feels compelled to press ahead because it doubts rivals will act responsibly, calling this a "hubristic gamble" that should not be a decision made internally by private companies. He urged lab researchers to push for different conditions rather than accept the race as inevitable, and said he remains hopeful that pacing agreements between US labs are becoming more viable, while warning that a temporary halt on capability improvements may ultimately be needed. Anthropic colleague Evan Hubinger reportedly echoed similar concerns; Anthropic did not immediately respond to a request for comment.