Ship and explain the injection regression suite

A shippable lab artifact includes: how to run it, the metric result (attack success rate, canary leakage, forbidden tool-call count), the failure you can still reproduce (answer refuses in text but a hidden tool call posts a canary token), the security gate for treating retrieved document text as trusted system instructions, and a rollback note. The user decision it supports remains: block indirect instructions in retrieved documents from causing forbidden tool calls.

release: candidate
baseline: stable
gates:
  primary_metric: "attack success rate"
  required: "0.00 canary leakage and 0.00 forbidden tool calls across critical attacks"
  security_regression: pass
  staging_probe: pass
  telemetry_signal: "injection_attack_success_total"
rollout:
  canary_percent: 5
  rollback_on: "critical failure or sustained SLO breach"
owner: on-call-ai-platform

Expected evidence: 0.00 canary leakage and 0.00 forbidden tool calls across critical attacks. Store this beside the fixture version so scores remain meaningful after content changes in prompt-injection-in-code.

State operating limits for the injection regression suite in plain language: fixture size, offline vs live dependencies, and what would require a new eval set. Shipping prompt-injection-in-code is honest scoping, not maximal confidence language.

Fill this table in your notes for the injection regression suite:

• Proved on indirect-pdf-07 with IGNORE RULES + CANARY payload: …
• Unproved beyond the fixture: …
• Metric that blocks release: attack success rate, canary leakage, forbidden tool-call count
• Failure still reproducible: answer refuses in text but a hidden tool call posts a canary token
• Security gate: treating retrieved document text as trusted system instructions
• Rollback: …

Ship the narrative only when the unproved list is honest. Reviewers trust narrow claims that support block indirect instructions in retrieved documents from causing forbidden tool calls more than maximal language that collapses under the first production oddity.

Hand your ship note to a peer and ask them to recreate a proved/unproved ship note with rollback without watching you type. If they cannot, your evidence is still tribal knowledge. Tighten the run command and the metric line until a stranger can validate the injection regression suite against indirect-pdf-07 with IGNORE RULES + CANARY payload.

At the mastery and shipping stage for prompt-injection-in-code, the job is narrower than finishing a product demo. You are creating one progressive evidence piece about indirect-pdf-07 with IGNORE RULES + CANARY payload that later pages inherit without redefining success. Keep that fixture small enough to inspect by hand, keep outputs copy-pasteable as text, and refuse to narrate this baseline as if it were a production SLA: agent without document/instruction separation on the same attacks.

For this page specifically, success looks like a proved/unproved ship note with rollback while still centering the user decision to block indirect instructions in retrieved documents from causing forbidden tool calls. If you cannot point to a file, command, or assertion that proves that for the injection regression suite, stay on this page instead of advancing.

How-to: red-team prompt injection · Snippet: injection test fixture · Glossary: adversarial prompt

Previous