Guardrails: what happens when a security consultant asks an AI for help

Security consultant Kit Barker argues that AI safety guardrails are made up of several separate systems, rather than one filter, and that this matters for legitimate penetration testing. Security professionals may need AI help with requests that resemble malicious activity, so the systems can block authorised work even when it is properly scoped.

Barker describes three layers: safety behaviour trained into the model, hidden system instructions that set priorities, and provider-side classifiers and account monitoring. The first two involve model judgement and can vary between products; external classifiers can block requests without considering a tester’s contract or scope. Barker says requests for working exploit code are a common source of friction, and that logs can distinguish a model’s written refusal from a classifier’s API block.