Define the injection regression suite input contract

The accepted input remains: attack cases with carrier=retrieved_document and an allowlisted tool set. Keep parsing and normalization in functions that do not score, train, or call a model. That split lets a test fail the boundary without blaming the core logic. The user-facing decision stays: block indirect instructions in retrieved documents from causing forbidden tool calls.

from dataclasses import dataclass
@dataclass(frozen=True)
class InjectionCase:
    case_id: str
    carrier: str
    payload: str
    allowed_tools: tuple
c=InjectionCase('indirect-pdf-07','retrieved_document','IGNORE RULES; CANARY',('search',))
print(c.case_id, c.carrier)

Expected evidence: injection case contract. If the contract is silent on a bad value, later debugging will look like an algorithm bug when it is really a data bug.

Write one assertion for a neighboring valid input to the injection regression suite so tightening the boundary does not over-reject. Document field names and types the way a teammate would need them on day two of prompt-injection-in-code—not as comments you plan to delete.

List every field in indirect-pdf-07 with IGNORE RULES + CANARY payload and mark each as required, optional, or forbidden. Required fields must fail loudly when missing; optional fields need defaults you can quote in a test; forbidden fields (secrets, raw PII, path escapes) must never be accepted silently. This list is the contract for the injection regression suite.

Add one sentence about encoding, units, or timezones if relevant to attack cases with carrier=retrieved_document and an allowlisted tool set. Contracts that ignore units create “correct” programs that still ship wrong decisions when someone tries to block indirect instructions in retrieved documents from causing forbidden tool calls.

Write the error string you want for the most likely bad input. Prefer ValueError('threshold out of range')-style messages over generic invalid input. The contract’s job is to make answer refuses in text but a hidden tool call posts a canary token harder to confuse with a model or algorithm bug later.

At the data contract stage for prompt-injection-in-code, the job is narrower than finishing a product demo. You are creating one progressive evidence piece about indirect-pdf-07 with IGNORE RULES + CANARY payload that later pages inherit without redefining success. Keep that fixture small enough to inspect by hand, keep outputs copy-pasteable as text, and refuse to narrate this baseline as if it were a production SLA: agent without document/instruction separation on the same attacks.

For this page specifically, success looks like malformed inputs rejected with field-named errors while still centering the user decision to block indirect instructions in retrieved documents from causing forbidden tool calls. If you cannot point to a file, command, or assertion that proves that for the injection regression suite, stay on this page instead of advancing.

How-to: red-team prompt injection · Snippet: injection test fixture · Glossary: adversarial prompt

Previous · Next