Instrument the pre/post guardrail pipeline

Instrument the pre/post output guardrail pipeline so a run records enough structure to debug offline: counts, latency if relevant, pass/fail of block rate, false-block samples, escape rate on red-team set, and a stable stage name. Redact secrets and raw credentials from every event.

import json
print(json.dumps({'stage':'guardrails','blocked':2,'false_block_samples':0,'model_calls_skipped':2}))

Expected evidence: guardrail telemetry. Prefer JSON or structured text you can grep in CI over prose logs for guardrails-in-code.

Turn one historical failure—especially post-guard only that still bills the model for blocked intents—into a test that fails if the signal disappears for the pre/post guardrail pipeline. Observability without a failing test is optional decoration; observability with a test is part of the guardrails-in-code artifact.

Draft a three-field event for the pre/post guardrail pipeline: stage, ok, and one domain field derived from block rate, false-block samples, escape rate on red-team set. Add fixture_id or docs_version when content can change. Explicitly list fields that must never appear (tokens, passwords, raw prompts) because logging blocked prompts that contain secrets into an unredacted sink is in scope for this lab.

Wire one assertion that fails if the pre/post guardrail pipeline event is missing after a run. Observability that cannot fail a test will not survive contact with a busy guardrails-in-code repository.

Imagine a teammate opens only your event stream after a bad deploy. Could they tell whether policy with PII and self-harm categories + sample prompts was wrong, whether post-guard only that still bills the model for blocked intents, or regex gaps on obfuscation returned, or whether logging blocked prompts that contain secrets into an unredacted sink slipped through? If not, rename fields until those three stories are distinguishable.

At the testing and observability stage for guardrails-in-code, the job is narrower than finishing a product demo. You are creating one progressive evidence piece about policy with PII and self-harm categories + sample prompts that later pages inherit without redefining success. Keep that fixture small enough to inspect by hand, keep outputs copy-pasteable as text, and refuse to narrate this baseline as if it were a production SLA: model call with no guards on the same fixtures.

For this page specifically, success looks like a structured event schema locked by a test while still centering the user decision to block unsafe prompts and strip or refuse unsafe completions before they reach users. If you cannot point to a file, command, or assertion that proves that for the pre/post guardrail pipeline, stay on this page instead of advancing.

Cheatsheet: prompt injection defense

Previous · Next