Reference · Glossary
Prompt injection
Last updated
Assistants that read email, browse the web, or retrieve documents can be tricked into leaking secrets, calling tools, or ignoring safety policies — even when the user never typed the malicious instruction.
#Why it matters
Assistants that read email, browse the web, or retrieve documents can be tricked into leaking secrets, calling tools, or ignoring safety policies — even when the user never typed the malicious instruction.
#Common patterns
- “Ignore previous instructions and…” buried in a PDF or ticket
- Fake system messages inside retrieved chunks
- Tool-result spoofing that tells the agent to exfiltrate data
#Defenses (practical)
- Separate **trusted instructions** (system / developer) from **untrusted content** (user uploads, retrieval).
- Constrain tools: allowlists, confirmations for destructive actions, least privilege.
- Ask models to paraphrase untrusted text instead of executing it as instructions.
- For RAG, instruct: answer only from evidence; refuse when chunks conflict with policy.
- Log and review tool calls; add evals that include injection test cases.
#When you see it in the wild
Any product that mixes retrieved or user-uploaded text into a prompt needs an injection threat model — not only chatbots with jailbreak prompts.