Run the runbooks baseline

Code the narrow path that serves secondary on-call clearing ingest.lag_minutes > 45: accept → authorize/normalize → call dependency → validate → record. Keep stages named so a trace can show which boundary passed. Success must emit evidence useful to drill_completion_minutes and lag_minutes, not only a 200 with prose. Predict the observable for ingest worker + queue metrics before running: drill INC-INGEST-9021: lag 60→5 minutes in ≤ 25m following runbook v1.4.

Drive the path with recording fakes or local stubs. Assert call order and arguments. Idempotency keys or stable ids should keep retries from duplicating costly work where the product requires it. Product under test remains executable queue-backlog runbook for embedding ingestion pipeline — resist adding unrelated features mid-path.

For Runbook lab, paste the CLI/HTTP transcript beside your prediction for ingest worker + queue metrics. If the oracle is unmet (drill INC-INGEST-9021: lag 60→5 minutes in ≤ 25m following runbook v1.4), stop and debug this page; do not compensate with prompt folktales. Re-run once after a clean process start to catch hidden global state that would invalidate INC-INGEST-9021.

Performance sketch: measure local p95 for the fake-backed path so later regressions are obvious. Keep concurrency modest until failure-handling proves limits. Log a single structured event per success with request id, revision, and the evidence field behind drill_completion_minutes and lag_minutes. Avoid hidden global caches in the happy path unless the lab is about caching — and even then key by tenant. If the path calls a model, pin model id in config and echo it in the response for auditability. Remember secondary on-call clearing ingest.lag_minutes > 45 experiences wall-clock time, not your debugger’s single-step comfort.

Prefer explicit function names over a single god-object handleRequest. Thread a correlation id from ingress to the last log line. When streaming, define what partial failure means before coding. Snapshot one successful response body in fixtures after redaction. If the path writes to a queue, assert message attributes in the fake. Stop adding retries on this page; that is the next concern. In this chapter the product is executable queue-backlog runbook for embedding ingestion pipeline, the human stakeholder is secondary on-call clearing ingest.lag_minutes > 45, and the incident id you design against is INC-INGEST-9021. Re-state the oracle in your notes — drill INC-INGEST-9021: lag 60→5 minutes in ≤ 25m following runbook v1.4 — and keep the invariant visible: every command is copy-pasteable; mutating steps need change ticket; verify after each step. Track drill_completion_minutes and lag_minutes as the scoreboard. Surface under change control: ingest worker + queue metrics. If you only have forty minutes, finish the fixture for runbook says 'restart something' without naming deployment or verifying lag before polishing UI. Promotion language stays ternary: promote, hold, or roll back based on evidence, not hope.