Closed models refuse to help researcher swat Linux bug
Security researcher Daniel Fox Franke of Akamai Technologies found that OpenAI's safety classifier repeatedly blocked GPT-5.6 Sol from helping him diagnose a segmentation fault in the ripgrep tool, forcing him to switch to Chinese open-weight models to complete the investigation, which ultimately uncovered an apparent Linux kernel bug. The episode highlights a growing tension for AI vendors: guardrails designed to stop closed models from aiding cyberattacks can also prevent legitimate security researchers from using them for routine vulnerability analysis.
Franke said OpenAI's classifier, a separate system that filters the model's output, kept flagging his queries as an "inappropriate line of inquiry" even after he narrowed the scope of his requests to avoid triggering it, and he said the underlying model itself never refused to help. He avoided Anthropic's Claude models entirely, assuming they would be even more restrictive, and declined to enrol in OpenAI's individual Trusted Access programme due to its verification requirements. Instead, he completed the analysis using Z'ai's GLM 5.2 and Moonshot AI's Kimi K3, with Kimi K3 providing the key evidence that the bug originated in the kernel rather than in ripgrep or musl.
In short
- OpenAI's safety classifier blocked a researcher debugging a Linux segfault
- He switched to Chinese open models (GLM 5.2, Kimi K3) instead
- Case shows AI guardrails can hinder legitimate security research
Steelman: both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
Strict safeguards can reduce the risk that powerful models help users exploit systems, and automated classifiers offer a scalable way to screen potentially dangerous requests. From this perspective, occasional friction is a reasonable cost if it limits assistance to malicious actors, especially when researchers can use verified access routes or other tools for legitimate work.
The case against
Overly broad filters can block routine debugging and vulnerability research, delaying the discovery of flaws that affect many users. Researchers may then turn to less restricted tools, while verification requirements can make trusted access impractical; safeguards work better when they distinguish benign analysis from actionable misuse.
Explained simply
A security researcher trying to use OpenAI's latest AI model to help fix a technical problem found that the company's safety system repeatedly blocked his requests. The researcher was asking straightforward technical questions, but the safety filters treated them as inappropriate.
Unable to use OpenAI's model and wary of trying others, the researcher turned instead to freely available AI models from Chinese companies. These models helped him complete his analysis and revealed what appears to be a bug in the core Linux operating system, which millions of computers worldwide rely on.
The episode illustrates a growing problem for AI companies: the safety filters designed to stop models from being misused also prevent legitimate security researchers from using them for routine technical work. It highlights a tension between guarding against potential abuse and enabling valid professional uses of the technology.