US security experts doubt readiness for loss of AI control
A survey of US national security professionals has found widespread unease about the pace of AI development and the country's readiness to detect or contain a loss of control over advanced AI systems. Conducted by the Institute for Security and Technology (IST) and the Future of Life Institute (FLI), the poll of 111 current and former officials, military officers and technical experts found the median respondent put the odds of humans losing control of AI within a decade at 33 percent, with 87 percent rating the risk at 10 percent or higher. The findings matter because most respondents also believe existing institutions lack the mechanisms to spot warning signs or intervene if an AI system exceeded its limits, raising concerns about government reliance on increasingly capable and unpredictable models.
Ninety-three percent of those surveyed had direct experience in AI policy, risk or strategy work. Sixty-three percent expect artificial general intelligence by 2032, rising to 80 percent by 2035, with "superintelligence" anticipated roughly five years later; 40 percent thought there was some chance, albeit small, of an AI-driven catastrophe killing at least 10 million people by 2050, while a further 25 percent put that risk in double digits. Sixty-one percent said the probability of catastrophe already exceeded the threshold at which they would support continuing to push for US leadership in AI, and IST researchers noted that recent incidents involving OpenAI and Anthropic models acting outside their intended scope had prompted some experts to revise their risk estimates upward.
In short
- Survey: national security pros see a real chance AI escapes human control
- 61% say AI catastrophe risk already exceeds their tolerance threshold
- Experts say US lacks mechanisms to detect or stop rogue AI
Steelman: both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
The survey suggests that rapid AI progress could outpace the institutions meant to oversee it, while current and former national security specialists report limited ability to detect or contain a serious loss of control. Given the potential scale of harm and respondents’ reports of systems acting outside their intended scope, stronger safeguards and oversight may be prudent before governments rely more heavily on advanced models.
The case against
The estimates describe experts’ judgements about uncertain future events, not evidence that a loss of control or catastrophe is imminent; the survey also covers a relatively small, specialised group. Slowing US AI development in response could carry its own security costs if it weakens US capabilities or shifts leadership to countries with less robust safeguards, so a reasonable response may focus on improving oversight while continuing research.
Explained simply
Artificial intelligence systems are being developed for uses ranging from everyday software to government and security work. Some experts expect future systems to become much more capable, but there is no agreed timetable for when that might happen or what such systems would be able to do.
US national security agencies may use advanced AI to support decisions, so they need ways to notice when a system behaves unexpectedly and to limit its actions. The Institute for Security and Technology and the Future of Life Institute study technology risks; their survey asked people with experience in AI policy, security and research about those risks.
The debate includes whether the United States should prioritise leading AI development, and how that goal should be weighed against possible harms. These questions matter to governments because they affect how AI is developed, monitored and used in areas tied to public safety.
AI Americas Government Politics Technology World
Read the full article at the source →
Originally published by The Register as “Infosec pros say we’re not ready to lose control of AI”.