Capstone: ship a tiny AI app
Instrument the grounded café FAQ application
Metrics for gold_pass and demo_seconds must distinguish bad input from component failure for café ops lead answering hours/wifi/pets questions from a kiosk.
1Learn the idea
Read
Golden signals for this system
Instrument grounded café FAQ app with validation, five-note retrieval, citations, tests, demo so café ops lead answering hours/wifi/pets questions from a kiosk can answer: demand, errors, latency/age, saturation. Emit fields needed by gold_pass and demo_seconds with bounded labels. Sample successful high-volume traces; keep errors and rollout transitions denser within policy.
Read
Alert path worth paging
Define at least one alert that would fire for CAP-CAFE-DEMO-01, with a for/pending window that survives deploy blips. Missing scrape or missing revision labels must not look like health. Include a trace/log example id format you will actually search.
Read
Implementation artifact
print({"event": "faq_ask", "citation": citation, "abstain": abstain})
Read
Tests for telemetry
Add a unit/integration check that metrics increment on the happy path and on the demo relies on personal .env and unreproducible notebook state branch. Store machine-readable output in CI artifacts when practical.
Read
Stage depth
Cardinality discipline: tenant and revision are usually enough; raw question text is not a label. Exemplars or trace links beat screenshots alone when debugging CAP-CAFE-DEMO-01. Define who owns alert fatigue review. If you export to a vendor, record retention and access. Synthetic probes should use non-sensitive fixtures and still exercise CLI faq_ask. Practice the query you will type at 2am once, while calm.
Read
Field notes for `capstone-ship-it` / `observability`
Document the exact PromQL or log query in the runbook stub for this service. Verify histograms have buckets around your SLO target. Add a canary synthetic check that exercises the oracle path every few minutes in staging. Confirm that PII redaction happens before export. Track build/version as a label on the golden signals. Delete noisy debug logs before they become accidental product dependencies. In this chapter the product is grounded café FAQ app with validation, five-note retrieval, citations, tests, demo, the human stakeholder is café ops lead answering hours/wifi/pets questions from a kiosk, and the incident id you design against is CAP-CAFE-DEMO-01. Re-state the oracle in your notes — gold 5/5 cited + 3/3 abstain; demo under 120s from clean checkout — and keep the invariant visible: unsupported questions abstain; every answer cites a note id; two-minute clean-room demo. Track gold_pass and demo_seconds as the scoreboard. Surface under change control: CLI faq_ask. If you only have forty minutes, finish the fixture for demo relies on personal .env and unreproducible notebook state before polishing UI. Promotion language stays ternary: promote, hold, or roll back based on evidence, not hope.
Go deeper
Before you start
Why this matters
Name the dashboard row or log line café ops lead answering hours/wifi/pets questions from a kiosk opens first during CAP-CAFE-DEMO-01. It must include a correlation id and a bounded label from gold_pass and demo_seconds. If telemetry is missing, write whether you promote, hold, or roll back — and why hold is the default.
Capstone metrics (gold_pass and demo_seconds) appear in the final evidence pack.
In the wild
See how this idea shows up as a product and a company — then come back to the lesson. Skills transfer across vendors.
Related lessons
Check your understanding
Page assessment
Answer from memory. Completion is saved from this evidence, not from opening the next page.
All responses are required.