Page 6 of 8~104 min topic

Prompt injection & AI security

Evaluate with evidence

Measure prompt injection with denominators, slices, and gates chosen before seeing results on the tool-using support agent.

~13 min this pageEvaluation

1Learn the idea

Read

Metrics

See it

Why fluent answers can still be wrong
01Predict ≠ lookupSounds like an answer
02Web is messyFacts + fanfic mix
03No embarrassmentCan sound sure
04Prompt trapAsked to invent detail

Confidence is a tone — verify before you act

Track for prompt injection: red-team pass rate, false positive rate on benign, privileged action rate, time-to-contain. Report fractions like 36/40, not vague quality adjectives. Segment by language, plan tier, document length, or other slices that matter for the tool-using support agent.

Read

Protocol

Freeze inputs and neighboring versions while evaluating prompt injection. Change one control. Pair results case by case on the tool-using support agent. Define hard gates (severe errors, privacy, latency) before the bake-off. Use deterministic checks where possible; humans for nuance; model judges only with calibration against gold.

Numeric reminder for prompt injection: risk ≈ probability of successful injection × impact of available capability; reducing tool privilege cuts impact even when detection is imperfect

Read

Make it operational

Resist adding a twelfth metric before the first three for prompt injection on the tool-using support agent have owners. This workload improves faster when a small scorecard is trusted than when a warehouse of unused plots exists.

Also pin one numeric memory from this prompt injection chapter: risk ≈ probability of successful injection × impact of available capability; reducing tool privilege cuts impact even when detection is imperfect That number is not decoration; it is a template for how claims about prompt injection on the tool-using support agent should look in design docs. Scoped specifically to prompt injection / tool-using support agent / evaluation.

Read

Common mix-ups

People confuse prompt injection with neighboring buzzwords when debugging the tool-using support agent. Before changing prompts, ask whether the broken stage was evidence gathering, the prompt injection judgment itself, validation, or the product action. Fixing the wrong stage creates folklore (“we tried prompt injection and it failed”) that blocks the next team on the tool-using support agent. Scoped specifically to prompt injection / tool-using support agent / evaluation.

Read

Rehearsal (prompt-injection/evaluation)

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt injection rather than generic AI advice.

Read

Rehearsal (prompt-injection/evaluation)

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt injection rather than generic AI advice.

Read

Rehearsal (prompt-injection/evaluation)

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt injection rather than generic AI advice.

Read

Rehearsal (prompt-injection/evaluation)

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt injection rather than generic AI advice.

Go deeper

Before you start

Why this matters

A demo of the tool-using support agent looks great on three hand-picked examples of prompt injection. What does that demo refuse to tell you?

In the wild

See how this idea shows up as a product and a company — then come back to the lesson. Skills transfer across vendors.

Check your understanding

Page assessment

Answer from memory. Completion is saved from this evidence, not from opening the next page.

1. What is one idea from this page you would apply, and what evidence would you check?

All responses are required.