Guardrail Classifier
A lightweight AI model that runs before or after the main agent response to detect policy violations—toxicity, PII leakage, off-topic responses, prompt injection attempts, or unauthorized actions. Guardrail classifiers add 20–50ms of latency but prevent harmful outputs from reaching users. They operate independently of the main model, providing defense-in-depth: even if the primary model is jailbroken, the classifier catches the violation.
Written by Max Zeshut
Founder at Agentmelt · Last updated Sep 9, 2026
Example
A support agent generates a response that accidentally includes a customer's credit card number from the conversation history. The guardrail classifier detects the PII pattern, redacts the number, and logs the incident—before the response is sent to the customer.
Frequently asked questions
- Can't the main model just be prompted to follow guardrails?
- Prompting helps but isn't sufficient alone. Models can be jailbroken, make mistakes under pressure, or hallucinate policy violations. A separate classifier provides an independent safety layer—like having a code reviewer who catches bugs the original developer missed.