Reference · Glossary
Red teaming
Deliberately **attacking your own AI system** to find failures — unsafe outputs, leaks, bad tool calls — before users do.
When to use
Before launching customer-facing bots, agents with tools, or apps handling sensitive data.
When not to
A solo weekend hackathon with no users — basic prompt testing may be enough initially.
Example
A red team tries 200 adversarial prompts (injection, bias bait, PII extraction) and files bugs ranked by severity.