Page 4 of 8~104 min topic

Prompt injection & AI security

Weigh the tradeoffs

Blocking suspicious phrases is simple but produces false positives and misses paraphrases. Giving an agent broad tools increases usefulness and blast radius together. Human approval reduces autonomous speed but is appropriate for payments, deletion, disclosure, and external messages.

~13 min this pageTradeoffs

1Learn the idea

Read

The live tension

See it

Why fluent answers can still be wrong
01Predict ≠ lookupSounds like an answer
02Web is messyFacts + fanfic mix
03No embarrassmentCan sound sure
04Prompt trapAsked to invent detail

Confidence is a tone — verify before you act

Blocking suspicious phrases is simple but produces false positives and misses paraphrases. Giving an agent broad tools increases usefulness and blast radius together. Human approval reduces autonomous speed but is appropriate for payments, deletion, disclosure, and external messages.

Translate into user impact on the tool-using support agent when tuning prompt injection. Which error class costs more—missed catches, slower answers, higher spend, or privacy exposure? That ranking picks the default more honestly than a blog’s recommended settings for prompt injection.

Read

Numbers that force honesty

risk ≈ probability of successful injection × impact of available capability; reducing tool privilege cuts impact even when detection is imperfect Scoped specifically to prompt injection / tool-using support agent / tradeoffs.

If the aggressive prompt injection setting wins the headline metric while breaking a protected slice or blowing the latency budget on the tool-using support agent, it is not a win. Record intended gain and tolerated regression together for prompt injection.

Read

Make it operational

Revisit the prompt injection tradeoff when traffic shape changes on the tool-using support agent. A setting that was right at low volume can fail when a new language segment or document length appears. Tradeoffs expire; re-measure on a calendar, not only on incidents.

Also pin one numeric memory from this prompt injection chapter: risk ≈ probability of successful injection × impact of available capability; reducing tool privilege cuts impact even when detection is imperfect That number is not decoration; it is a template for how claims about prompt injection on the tool-using support agent should look in design docs. Scoped specifically to prompt injection / tool-using support agent / tradeoffs.

Read

Common mix-ups

People confuse prompt injection with neighboring buzzwords when debugging the tool-using support agent. Before changing prompts, ask whether the broken stage was evidence gathering, the prompt injection judgment itself, validation, or the product action. Fixing the wrong stage creates folklore (“we tried prompt injection and it failed”) that blocks the next team on the tool-using support agent. Scoped specifically to prompt injection / tool-using support agent / tradeoffs.

Read

Rehearsal (prompt-injection/tradeoffs)

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt injection rather than generic AI advice.

Read

Rehearsal (prompt-injection/tradeoffs)

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt injection rather than generic AI advice.

Go deeper

Before you start

Why this matters

For the tool-using support agent, name one regression you will tolerate when pursuing the main benefit of prompt injection, and one regression that is stop-ship.

In the wild

See how this idea shows up as a product and a company — then come back to the lesson. Skills transfer across vendors.

Check your understanding

Page assessment

Answer from memory. Completion is saved from this evidence, not from opening the next page.

1. What is one idea from this page you would apply, and what evidence would you check?

All responses are required.