Chapter BIterate: bad to betterPage 7 of 8

Iterate: bad to better

Failure modes and stopping rules

Iteration helps only while it reduces important gaps without creating larger costs or risks.

~14 minEdge cases and tradeoffs

Before you start

Why this matters

Prompt improvement can become an avoidance loop. A user requests ten rewrites of a two-sentence note, changes tone back and forth, and still does not verify the date. Another user keeps asking the same assistant whether a citation is real. More rounds create confidence but no new evidence.

A mature workflow recognizes failure patterns and sets a stopping rule before polish becomes endless.

1Learn the idea

Read

Failure mode: changing everything

When goal, context, format, and tone all change together, comparison becomes impossible. Return to the stable core, choose the highest-impact defect, and make one controlled revision. If the original specification was fundamentally wrong, write a clean new prompt and mark it as a reset rather than pretending it was a small edit.

Read

Failure mode: contradictory history

Long conversations may contain “make it detailed,” later “keep it brief,” then “restore all examples.” The model may blend obsolete and current instructions. Summarize the final specification, remove superseded rules, and begin a fresh chat when necessary. Include only the source material needed for the current task.

Read

Failure mode: polishing unsupported claims

Repeated editing can make an invented claim more persuasive. Stop when a claim depends on information outside the supplied evidence. Verify it using an authoritative source, remove it, or label the uncertainty. Asking the model to reconsider itself is not independent verification.

Read

Failure mode: prompt overfitting

A prompt may become perfectly tuned to one example and fail on ordinary variation. It contains special instructions for every quirk until no one can understand it. Test on several representative inputs. Replace case-specific patches with general rules, and delete rules that no longer improve performance.

Read

Failure mode: conflicting criteria

You can demand exhaustive detail, extreme brevity, a friendly explanation, and rigid formatting, but the requirements may not coexist. Rank them. State that correctness and safety outrank length, or that a machine-readable schema outranks conversational style. When a tradeoff is unavoidable, choose deliberately.

Read

Failure mode: format obsession

Valid JSON or a beautiful table can still contain wrong values. Treat format validation and content validation as separate checks. Parsing proves syntax, not truth. A clean email layout proves neither authorization nor appropriateness.

Read

Failure mode: lost provenance

After many rounds, it becomes unclear which sentences came from the source and which were model-generated bridges. Preserve original notes, citations, and version history. For consequential claims, require source labels and check them. Do not let a generated summary become the only surviving record.

Read

Set stopping rules

Stop and use the output when:

  • all hard gates pass;
  • required criteria meet an agreed threshold;
  • remaining issues are low-impact preferences;
  • another round costs more than its likely benefit;
  • the responsible human has reviewed the action.

Stop and gather information when a missing fact blocks correctness. Stop and choose another tool when the task requires live search, exact calculation, deterministic transformation, specialist judgment, or an approved system. Stop entirely when the requested action is unsafe or unauthorized.

For routine work, use a round limit such as two revisions before reassessing. For reusable team prompts, use a test-set threshold rather than a round count. For high-stakes work, no number of prompt rounds replaces professional or accountable review.

Read

Define “good enough”

Good enough is task-dependent. A private brainstorming list can tolerate roughness. A customer commitment requires source checks and approval. Write the minimum quality bar before drafting:

Ship when every date matches the project record, the decision and owner are visible, no confidential details appear, and the manager can act without asking what is requested.

This prevents perfectionism while protecting important outcomes.

Read

Keep a human decision point

The assistant can propose, compare, and reformat. A person should own the final meaning, evidence, audience, and action. Read the final output as if you had not participated in the chat. Check whether it could mislead a reader who lacks your background context.

Checking tutor…

Continue learning · glossary & guides