Page 7 of 8~104 min topic

Prompt injection & AI security

Trace a worked example

From goal to measurement to ship-or-abort for prompt injection on the tool-using support agent.

~13 min this pageWorked example

1Learn the idea

Read

Trace

See it

Why fluent answers can still be wrong
01Predict ≠ lookupSounds like an answer
02Web is messyFacts + fanfic mix
03No embarrassmentCan sound sure
04Prompt trapAsked to invent detail

Confidence is a tone — verify before you act

Goal: improve the tool-using support agent using prompt injection without breaking protected slices.

Mechanism reminder for prompt injection: Separate instructions from data, constrain tools with least privilege, validate outputs, require approvals for consequential actions, and test adversarial content.

Baseline and shock: risk ≈ probability of successful injection × impact of available capability; reducing tool privilege cuts impact even when detection is imperfect

Tradeoff in play for prompt injection: Blocking suspicious phrases is simple but produces false positives and misses paraphrases. Giving an agent broad tools increases usefulness and blast radius together. Human approval reduces autonomous speed but is appropriate for payments, deletion, disclosure, and external messages.

Ship decision: Ship agent with no payment tool autonomy; approvals on external send; weekly injection suite must be clean.

Rollback triggers for prompt injection must cite red-team pass rate, false positive rate on benign, privileged action rate, time-to-contain. If you cannot name a tolerated regression on the tool-using support agent, do not promote the change.

Read

Make it operational

Keep the prompt injection decision record beside the tool-using support agent code paths that implement it. Future you will not remember why a default exists unless the evidence is linked from the config for prompt injection.

Also pin one numeric memory from this prompt injection chapter: risk ≈ probability of successful injection × impact of available capability; reducing tool privilege cuts impact even when detection is imperfect That number is not decoration; it is a template for how claims about prompt injection on the tool-using support agent should look in design docs. Scoped specifically to prompt injection / tool-using support agent / worked-trace.

Read

Common mix-ups

People confuse prompt injection with neighboring buzzwords when debugging the tool-using support agent. Before changing prompts, ask whether the broken stage was evidence gathering, the prompt injection judgment itself, validation, or the product action. Fixing the wrong stage creates folklore (“we tried prompt injection and it failed”) that blocks the next team on the tool-using support agent. Scoped specifically to prompt injection / tool-using support agent / worked-trace.

Read

Rehearsal (prompt-injection/worked-trace)

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt injection rather than generic AI advice.

Read

Rehearsal (prompt-injection/worked-trace)

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt injection rather than generic AI advice.

Read

Rehearsal (prompt-injection/worked-trace)

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt injection rather than generic AI advice.

Read

Chapter close 1

For prompt injection, add acceptance test 1: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 2

For prompt injection, add acceptance test 2: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 3

For prompt injection, add acceptance test 3: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 4

For prompt injection, add acceptance test 4: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 5

For prompt injection, add acceptance test 5: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 6

For prompt injection, add acceptance test 6: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 7

For prompt injection, add acceptance test 7: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 8

For prompt injection, add acceptance test 8: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 9

For prompt injection, add acceptance test 9: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 10

For prompt injection, add acceptance test 10: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 11

For prompt injection, add acceptance test 11: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 12

For prompt injection, add acceptance test 12: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 13

For prompt injection, add acceptance test 13: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 14

For prompt injection, add acceptance test 14: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 15

For prompt injection, add acceptance test 15: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 16

For prompt injection, add acceptance test 16: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 17

For prompt injection, add acceptance test 17: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 18

For prompt injection, add acceptance test 18: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 19

For prompt injection, add acceptance test 19: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 20

For prompt injection, add acceptance test 20: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 21

For prompt injection, add acceptance test 21: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 22

For prompt injection, add acceptance test 22: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 23

For prompt injection, add acceptance test 23: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 24

For prompt injection, add acceptance test 24: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 25

For prompt injection, add acceptance test 25: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 26

For prompt injection, add acceptance test 26: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 27

For prompt injection, add acceptance test 27: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 28

For prompt injection, add acceptance test 28: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 29

For prompt injection, add acceptance test 29: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 30

For prompt injection, add acceptance test 30: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 31

For prompt injection, add acceptance test 31: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 32

For prompt injection, add acceptance test 32: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 33

For prompt injection, add acceptance test 33: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 34

For prompt injection, add acceptance test 34: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Read

Chapter close 35

For prompt injection, add acceptance test 35: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

Go deeper

Before you start

Why this matters

List the constraints (latency, cost, privacy, review capacity) that any prompt injection change must respect for the tool-using support agent.

In the wild

See how this idea shows up as a product and a company — then come back to the lesson. Skills transfer across vendors.

Check your understanding

Page assessment

Answer from memory. Completion is saved from this evidence, not from opening the next page.

1. What is one idea from this page you would apply, and what evidence would you check?

All responses are required.