Cisco releases open-source tool to detect risky AI agent behaviour
Cisco has released Project Quarry, an open-source tool that analyses AI agents’ behaviour to help security teams spot risky activity. It is designed to assess actions in context across an entire session, since a sequence of individually ordinary steps can combine to expose data or enable an attack.
Quarry covers conversations with language models, agent tool use, Model Context Protocol interactions and multimodal inputs. Its rules can flag patterns such as reading a sensitive file and then sending data to an external service, or using credentials elsewhere; Cisco says it might have identified warning signs in the Hugging Face intrusion, though it has not tested the tool against that incident’s data. The project includes setup guidance, integrations and starter rules.
In short
- Cisco has open-sourced Project Quarry to monitor AI agent behaviour.
- It looks for risky sequences of actions across a session.
- Rules can flag sensitive data being sent to external services.
Explained simply
AI agents are software systems that can interpret requests and take steps on a user’s behalf, such as searching files or calling other services. These steps may look harmless in isolation, but their order and purpose can affect what data is exposed or what actions are carried out.
Cisco is a technology company that makes networking and security products. Project Quarry is an open-source tool, meaning its code is available for others to inspect and adapt; it examines an agent’s activity across a session so security teams can look for concerning patterns.
The issue matters as organisations connect AI agents to files, accounts and online services. Monitoring activity in context may help teams spot risky behaviour, though a tool’s alerts need to be assessed against the circumstances.
Read the full article at the source →
Originally published by The Register as “Cisco open sources tool for detecting AI agents on hacking sprees”.