AI Models Can Be Easily Modified to Assist Terrorist and Extremist Activity
A report by UK non-profit Tech Against Terrorism finds that many AI models can be prompted to provide information useful to terrorists or extremists, and that safeguards on openly available models can be removed and shared quickly. Researchers warn that these weaknesses create risks beyond the public debate about advanced AI, including help with radicalisation and propaganda.
The study tested 134 models with more than 200 prompts, each tried three ways; only two models passed every test, while 81 failed at least one measure. Framing a request as coming from a safety researcher made models almost nine times more likely to answer, and they did so 16.9 per cent of the time. On average, modified versions of newly released open-weight models appeared online in under three days, compared with more than 24 days in 2024.
- Many tested AI models failed at least one safety measure.
- Researcher framing made models more likely to answer.
- Modified open-weight models now appear online in under three days.
New here? Start with this
Tech Against Terrorism is a UK non-profit that works to limit the use of online services by terrorist and extremist groups. AI models are computer systems that generate text in response to questions; some are openly available, and their underlying components, or “weights”, can be downloaded and altered.
The concern is that people could use these systems to produce material that supports extremist activity, or remove safeguards intended to prevent harmful answers. Researchers, technology companies and online platforms all play a part in assessing these risks and deciding how models are released and protected.
The issue matters because altered versions can spread beyond the organisations that built the original models. That makes it harder to control how they are used, while raising questions about how to balance public access to AI with efforts to prevent harm.
Both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
The findings support stronger safeguards and oversight: if models can be coaxed into providing extremist material and modified versions spread within days, developers and platforms have a responsibility to reduce foreseeable harm. Those risks extend beyond direct operational instructions to propaganda and radicalisation, which can be harder to detect and may affect vulnerable people.
The case against
The report identifies weaknesses, but it does not establish how often people use these models to facilitate real-world extremist activity or how much additional harm they cause. Broad restrictions could also limit legitimate security research and access to open models; a proportionate response would preserve those benefits while targeting demonstrable risks and improving safeguards.