AI Guardrails

A guardrail is a check that runs on AI traffic while it is in flight. It reads the text of a request on its way to a model, or a response on its way back, decides whether that text violates one of your policies, and then blocks it, redacts part of it, or records it and lets it pass.

The checks themselves are ordinary programs — regular expressions for credentials and identifiers, a curated phrase list for jailbreak attempts, and transformer models for the judgements that need one. Nothing about them is specific to a provider, so the same policy governs OpenAI, Anthropic, Bedrock, a self-hosted model, and an MCP server.

This section is about configuring and operating them. If you have not deployed a gateway yet, start at Install AI Gateway.

Two ways to run a guardrail

In the request path. The gateway is a reverse proxy in front of your model providers. Every call passes through it, so guardrails apply to all traffic without changing a line of application code, and a blocked request never reaches the provider. This is the deployment most customers want.

Called as a service. Your own code, an IDE hook, or LiteLLM sends text to the gateway and gets back a verdict. Nothing is proxied. Use this when you cannot put a proxy in the path, or when the thing you want a ruling on is not an LLM call at all — a shell command an agent wants to run, a file it is about to read, an MCP tool invocation. See Guardrail API.

Both run the same policy engine over the same scanners. One configuration governs both.

How a policy is expressed

A policy answers five questions. Every one of them changes what happens to a request, and it is worth being able to name all five before you write one.

	Question	Values
Policy type	What family of risk is this?	DATA_PROTECTION, CONTENT_SAFETY, ACCESS_CONTROL, MCP_GOVERNANCE
Detector	What specifically are we looking for?	A detector code such as PROMPT_INJECTION or PII_SSN — see Scanner catalogue
Action	What do we do when it fires?	BLOCK, MASK, FLAG / LOG
Direction	Which leg of the call?	INPUT, OUTPUT, BOTH
Enforcement mode	Are we actually enforcing yet?	MONITOR, ENFORCE

Actions

Action	What happens
BLOCK	The request is rejected with HTTP 403 and a JSON body naming the policy and the request ID. The text never reaches the provider.
MASK	The matched span is replaced in place — 123-45-6789 becomes [SSN-REDACTED] — and the rewritten body is forwarded. The call succeeds; the provider never sees the original.
FLAG / LOG	The request passes through unchanged. The finding is recorded and alerted.

A BLOCK looks like this on the wire:

{
  "error": "blocked",
  "reason": "Request blocked by inspection findings",
  "policy": "inspection",
  "request_id": "954a57c4-3d2f-443a-a7c3-b878ccb81df0"
}

MASK is the right default for anything that matches broadly. The built-in PII pattern set includes url, so a BLOCK on PII rejects any prompt containing a link — which in practice is most of them.

Direction, and why OUTPUT is the one that matters for injection

INPUT scans what your user sent. OUTPUT scans what came back from the model. BOTH installs the guard on each leg independently.

Direct prompt injection — a user typing ignore your previous instructions — is an INPUT problem. Indirect prompt injection is not. There the attack is planted in something the model retrieves: a web page it fetches, a document in your RAG index, the output of a tool it called. The user's prompt is innocent; the payload arrives on the way back. Only an OUTPUT guard is positioned to see it.

This is the most commonly missed half of a guardrail configuration. A deployment with injection detection on INPUT only will report clean while an attacker walks in through a retrieved document.

MONITOR vs ENFORCE

MONITOR is how you roll out a policy without risking live traffic. In monitor mode every action downgrades to FLAG — including an explicit BLOCK. The request is never rejected and nothing is ever rewritten.

What does not change is detection. Findings are still produced, alerts still fire, and the audit record still names the policy, tagged with the enforcement mode it came from. So you can leave a policy in MONITOR for a week, look at exactly what it would have blocked, and promote it to ENFORCE with the false-positive rate already measured.

Run new policies in MONITOR first. This is particularly true for the ML content scanners, whose false-positive rate depends on what your users actually write.

The gate that makes everything else moot

Request inspection only runs when a governance rule asks for it

The gateway evaluates governance rules first and content guards second. Request guards run only when the matched governance rule returns allow_with_inspection (in the Levo UI: an ACCESS_CONTROL policy with an INSPECT or BLOCK action). If no rule matches, traffic is allowed — and allowed means not inspected, no matter how many scanners you configured.

A snapshot that contains only DATA_PROTECTION and CONTENT_SAFETY policies looks completely configured, loads without a warning, and scans nothing on the request leg. This is the single most common "my policies aren't working" report.

The difference is stark, and easy to reproduce. These two configurations have an identical guard list; only the governance action differs:

    rules:
      - name: inspect-all
        match: "true"
        action: allow_with_inspection   # request guards run
        priority: 50

    rules:
      - name: allow-all
        match: "true"
        action: allow                   # request guards never run
        priority: 50

With allow_with_inspection, a prompt injection is rejected:

HTTP 403
{"error":"blocked","reason":"Request blocked by inspection findings", ...}

With allow, the same prompt — and a leaked AWS key, and an SSN — pass through untouched with HTTP 200.

Two things are deliberately not subject to this gate, and knowing that saves a lot of confused debugging:

• Response guards always run. Response-leg inspection is a separate stage that does not consult the governance decision at all. So a misconfigured deployment can block indirect injection on the way back while leaving the request leg completely unguarded — which reads, from the outside, like the guardrails are working.
• The Guardrail API is not gated either. It evaluates your guards directly. A verdict from that endpoint is not evidence that in-path inspection is running.

If you configure policies through the Levo UI, the fix is to add one ACCESS_CONTROL policy whose action is INSPECT, scoped to the traffic you want scanned. See Configuring policies.

Where to go next

• Scanner catalogue — every detector code you can type, what it maps to, and which ones are accepted and then silently skipped.
• Configuring policies — the Levo UI, static YAML for air-gapped and GitOps deployments, and how the two interact.
• Operating guardrails — fail-open vs fail-closed, model requirements, sizing, and health endpoints.
• Alerts and audit — where a violation surfaces, and the JSON record to ship to your SIEM.
• Troubleshooting — symptom to cause.