Examples and few-shot prompting

Good examples demonstrate:

• the exact output schema or heading order;
• how to handle missing fields;
• the difference between similar categories;
• the level of brevity you want;
• when to refuse, escalate, or ask a question.

Bad examples accidentally teach invention, overconfidence, or irrelevant style flourishes. If every example invents a confident root cause, the model will imitate that habit. If every example is perfectly filled, the model may avoid nulls even when the source is incomplete.

Prefer a small set that covers the decision space:

1. a clean, typical case;
2. a borderline case;
3. a messy or incomplete case;
4. optionally, a case that must be refused or escalated.

Three to five examples usually beat ten near-duplicates. Duplicates waste context and hide the edge cases that cause production pain.

Keep examples short. Long essays as shots consume tokens and bury the label. For extraction, show the source snippet and the structured result. For classification, show the item and the category with a one-line reason if reasons are part of the contract.

Never put confidential or outdated policy into casual examples that people copy. Prefer synthetic or redacted samples for teaching format. Keep live policy in a clearly labeled context block that can be updated independently.

If an example includes a date, amount, or rule, ask whether that value is still true. Stale examples become silent policy bugs.

Structure the prompt so examples cannot be confused with the live task:

You classify support messages into billing, bug, how-to, or escalate.
Use escalate when identity, payment execution, or legal risk appears.
If unsure, choose escalate and state the uncertainty.

Examples:
Input: "My invoice shows tax twice."
Output: {"label":"billing","confidence":"medium","note":"duplicate tax line"}

Input: "The export button spins forever on Chrome."
Output: {"label":"bug","confidence":"high","note":"repro in Chrome export"}

Input: "Can you refund me now and change the legal entity name?"
Output: {"label":"escalate","confidence":"high","note":"payment + legal change"}

Now classify this message:
[INPUT]

The live input should be last. Models often continue the most recent pattern.

Examples guide behavior; they do not prove the prompt works. Test with held-out cases that were not in the prompt. Score label accuracy, missing-field handling, and escalation quality. When a failure appears, decide whether to fix instructions, replace an example, or add a clearer edge-case shot.

For high-volume workflows, move from chat examples to a maintained template plus an evaluation set. The pattern stays; the packaging becomes operational.

Few-shot pairs well with role-context-task and structured output. It pairs poorly with contradictory examples or with chain-of-thought that invents hidden facts. If you need reasoning, ask for brief evidence tied to the input rather than dramatic inner monologue.

Examples rot. Products change names, policies change thresholds, and teammates change house style. Put a review date on any few-shot template you reuse weekly. When an example no longer matches reality, replace it even if the prompt still “feels fine.” A library of outdated shots quietly trains the wrong behavior at scale.

Also watch for accidental leakage: screenshots, customer names, and internal identifiers should not live in shared example banks. Redact first, then teach the format.