Page 5 of 8~128 min topic

Capstone: ship a tiny AI app

Debug the capstone that cannot be trusted or demoed

When demo relies on personal .env and unreproducible notebook state, the system must degrade on purpose without widening blast radius.

~16 min this pageDebugging

1Learn the idea

Read

Classify and bound retries

Map failure classes for CLI faq_ask: retryable vs fatal vs needs-human. Retries need budgets, jitter, and idempotency rules aligned to unsupported questions abstain; every answer cites a note id; two-minute clean-room demo. The chapter’s signature failure — demo relies on personal .env and unreproducible notebook state — must take a deliberate branch, not a generic catch-all.

Read

Containment path

Implement the degrade/rollback/refuse behavior café ops lead answering hours/wifi/pets questions from a kiosk needs when CAP-CAFE-DEMO-01 repeats. Prefer scoped controls (one flag, one weight, one tenant, one secret version) over fleet-wide restarts. Preserve evidence; do not delete logs to “clean the demo.”

Read

Implementation artifact

if not citation:
    return {"abstain": True, "reason": "no_overlap"}

Read

Verify harm reduction

After containment, check gold_pass and demo_seconds moves in the safe direction and watch for retry amplification. Write the stop condition that ends the incident response for this lab.

Read

Stage depth

Chaos note: inject only one fault class at a time and restore fixtures after. Watch for dual failures — dependency down and retry amplifier — which is how demo relies on personal .env and unreproducible notebook state becomes an outage. Customer communication templates (even if only for the drill) beat silence. If you queue deferred work, define poison-message handling. Budget documents should state the maximum extra spend allowed during retries. Close the loop by linking the containment action to a dashboard panel for gold_pass and demo_seconds.

Read

Field notes for `capstone-ship-it` / `failure-handling`

Draw a state diagram for degrade modes and put it in the repo as ASCII if needed. Cap concurrent retries across the process, not only per request. Ensure cancellation propagates to downstream HTTP clients. When failing closed, choose a user-visible message that does not leak internals. Practice the single command that flips the kill switch or weight to zero. After recovery, drain or inspect deferred work before declaring green. In this chapter the product is grounded café FAQ app with validation, five-note retrieval, citations, tests, demo, the human stakeholder is café ops lead answering hours/wifi/pets questions from a kiosk, and the incident id you design against is CAP-CAFE-DEMO-01. Re-state the oracle in your notes — gold 5/5 cited + 3/3 abstain; demo under 120s from clean checkout — and keep the invariant visible: unsupported questions abstain; every answer cites a note id; two-minute clean-room demo. Track gold_pass and demo_seconds as the scoreboard. Surface under change control: CLI faq_ask. If you only have forty minutes, finish the fixture for demo relies on personal .env and unreproducible notebook state before polishing UI. Promotion language stays ternary: promote, hold, or roll back based on evidence, not hope.

Go deeper

Before you start

Why this matters

Assume demo relies on personal .env and unreproducible notebook state is happening right now. Write the first safe action, the signal that confirms containment, and the action you will not take (infinite retry, broad restart, deleting evidence). Tie the plan to invariant: unsupported questions abstain; every answer cites a note id; two-minute clean-room demo.

Capstone failure drills reuse CAP-CAFE-DEMO-01 as the story you tell in the ship review.

In the wild

See how this idea shows up as a product and a company — then come back to the lesson. Skills transfer across vendors.

Check your understanding

Page assessment

Answer from memory. Completion is saved from this evidence, not from opening the next page.

1. Are retry budgets explicit?
2. Is containment scoped?
3. Do you preserve evidence for CAP-CAFE-DEMO-01?

All responses are required.