AnyoneLearnAI
Start learning
Chapter DGuardrails in codePage 7 of 8

Guardrails in code

Cover security and operational gates

Guardrails in code is production work only when one frozen failure can be reproduced, one measurable gate can stop a release, and one operator can safely reverse it.

~14 minSecurity and ops

Before you start

Why this matters

Read this incident aloud: a user asks for an order update while smuggling an extra argument that would refund a different order. In two minutes, write the earliest deterministic check that should fail, the telemetry signal you would inspect, and the action that must not happen automatically. Compare your answer with this chapter's boundary: identity comes from the session, never model output; tools are allowlisted and arguments are schema-validated.

1Learn the idea

Read

Enforce security and operational boundaries

Treat every external string, retrieved document, provider response, and generated tool argument as untrusted. Authentication establishes who called; authorization decides which resource and action are allowed; schema validation only establishes shape. Keep those checks separate. The governing boundary here is: identity comes from the session, never model output; tools are allowlisted and arguments are schema-validated. Enforce it in deterministic code outside the prompt and again at the action adapter.

Use least-privilege credentials for staging and production. Deny outbound network access in adversarial tests. Replace secrets with canaries that are useless outside the fixture. Redact before serialization so sensitive values cannot leak through logs, traces, exception messages, or dead-letter queues. Test redaction with representative email, token, and customer-ID patterns, including nested fields.

Operational security also means controlling change. Version policies, fixtures, model configuration, and allowlists. Require review for boundary changes. Set retention and deletion rules for evidence. Rate-limit abusive sources without making the model responsible for identity. During an incident, deny the call, return a neutral response, and emit a redacted audit event. Preserve safe metadata for investigation, rotate credentials only when evidence indicates exposure, and do not copy raw payloads into tickets.

For security and operations, write an abuse case and a credential/permission check beside the normal test. Exercise denial, evidence retention, and break-glass review without granting the model extra authority.

Read

Focused implementation artifact

DENIED_KEYS = {"api_key", "authorization", "password", "raw_prompt"}

def safe_attributes(values):
    return {k: ("[REDACTED]" if k in DENIED_KEYS else v)
            for k, v in values.items()}

def authorize(session_subject, requested_subject):
    if session_subject != requested_subject:
        raise PermissionError("subject mismatch")

def test_no_sensitive_evidence_exported(exported_span):
    assert DENIED_KEYS.isdisjoint(exported_span.attributes)

Read

Test the boundary as an attacker

Start from the benign fixture and mutate one trust boundary at a time: identity, tool name, arguments, retrieved instruction, callback, or exported telemetry. Each mutation must be denied by deterministic code, not by asking the model to be careful. Assert the denial reason and zero forbidden side effects. The named abuse outcome is a keyword filter passes harmless wording but the model emits refund_order with an attacker-controlled customer_id.

Run the suite with fake canary secrets and outbound network denied. Search captured logs, spans, errors, and dead-letter payloads for the canary and representative PII. A clean response body is insufficient if an exporter leaked the value. Verify least-privilege credentials and ensure staging workers cannot write production state.

Operationalize the boundary: version policy and allowlists, require review for authority changes, set evidence retention, and document break-glass access. Track guardrail_decision_total without attacker-controlled labels. Security acceptance is not an average; the release still must satisfy zero unauthorized tool executions and at least 0.95 benign-request pass rate. On evidence of a breach, deny the call, return a neutral response, and emit a redacted audit event.

Checking tutor…

Continue learning · glossary & guides