Fears that artificial intelligence could pose an existential threat to humanity have circulated across the tech industry for more than a decade, yet they have consistently failed to slow the race to develop ever more powerful systems. The warnings stretch back to physicist Stephen Hawking's 2014 caution that AI "could spell the end of the human race", through to the recent viral resignation of Anthropic safety researcher Jacob Coxon, who said his employer and OpenAI were "racing straight to self-improving super intelligence and gambling with our lives". Despite such alarms repeatedly triggering internal splits and safety-focused spin-off companies, commercial competition has consistently overridden caution, illustrating a persistent tension between stated safety concerns and the industry's underlying incentives.

Coxon's resignation on 9 September prompted dozens of staff at Anthropic, OpenAI and rival firms to voice similar concerns, and led Anthropic chief executive Dario Amodei to call publicly for a slower, less "reckless" approach to AI development, citing risks including loss of control over AI systems, cyber-attacks, bioterrorism and economic disruption. The pattern is not new: OpenAI itself was founded in 2015 by Elon Musk and Sam Altman as a safety-driven nonprofit, partly in response to concerns that Google, which had just bought DeepMind for $650m, was not developing AI responsibly enough. Yet emails from that period show Altman already framing the effort in competitive terms, arguing it was "almost definitely" impossible to halt AI's development and that it was better for someone other than Google to lead it — a dynamic that critics say has shaped the industry ever since, including reported shifts away from safety priorities at OpenAI from 2021 onwards.