Closed models refuse to help researcher swat Linux bug

← Back to the feed

Closed models refuse to help researcher swat Linux bug

The Register · 5 hours ago

Security researcher Daniel Fox Franke of Akamai Technologies found that OpenAI's safety classifier repeatedly blocked GPT-5.6 Sol from helping him diagnose a segmentation fault in the ripgrep tool, forcing him to switch to Chinese open-weight models to complete the investigation, which ultimately uncovered an apparent Linux kernel bug. The episode highlights a growing tension for AI vendors: guardrails designed to stop closed models from aiding cyberattacks can also prevent legitimate security researchers from using them for routine vulnerability analysis.

Franke said OpenAI's classifier, a separate system that filters the model's output, kept flagging his queries as an "inappropriate line of inquiry" even after he narrowed the scope of his requests to avoid triggering it, and he said the underlying model itself never refused to help. He avoided Anthropic's Claude models entirely, assuming they would be even more restrictive, and declined to enrol in OpenAI's individual Trusted Access programme due to its verification requirements. Instead, he completed the analysis using Z'ai's GLM 5.2 and Moonshot AI's Kimi K3, with Kimi K3 providing the key evidence that the bug originated in the kernel rather than in ripgrep or musl.

  • OpenAI's safety classifier blocked a researcher debugging a Linux segfault
  • He switched to Chinese open models (GLM 5.2, Kimi K3) instead
  • Case shows AI guardrails can hinder legitimate security research

Software

Read the full article at the source →