Set a quality and verification bar

For plan and ship a personal knowledge assistant in staged, testable layers, define quality across accuracy, completeness, usefulness, safety, and reproducibility. Weight dimensions according to harm. A cosmetic miss can be revised; an unsupported claim, broken calculation, privacy leak, or rights violation blocks release.

Translate each dimension into observable checks. Accuracy means a claim, value, behavior, or frame agrees with an authoritative source. Completeness means every required field or stage appears. Usefulness means a mentor or hiring manager reviewing engineering judgment can take the intended action. Safety includes the boundary that you must use synthetic or personally owned documents initially; separate secrets, user content, telemetry, and model prompts with least-privilege access. Reproducibility means the prompt, input version, settings, and review evidence are saved.

Use checks from cheapest to strongest:

1. Contract check: required sections, schema, length, and prohibited content.
2. Source check: trace claims and values to supplied evidence.
3. Edge check: run normal, boundary, missing, and adversarial cases.
4. Independent check: calculate, test, rehearse, listen, inspect, or open the original.
5. Human gate: a responsible reviewer approves consequential use.

In this chapter, the concrete verification is to demo each milestone against golden tasks, inspect citations, run eval gates, test hostile inputs, review traces, exercise retries, and rehearse rollback. The expected candidate is A dependency-aware roadmap that ships a narrow cited-answer slice early, then adds quality and operational layers behind explicit gates. Record actual evidence, not a checkbox copied from the prompt.

Score each criterion 0 (fails), 1 (partly), or 2 (passes). Any zero for factual correctness, permission, privacy, or required disclosure is an automatic stop. A total score is useful for comparing iterations, but it must never average away a blocking defect.

Turn this capstone goal into six two-week milestones: an assistant answers questions from my approved notes with citations. Sequence API, structured output, retrieval, citations, tools, memory, evals, injection defense, tracing, retries, deploy, and monitoring. For each milestone define a demo, test, dependency, risk, and stop rule. Do not pretend all layers fit at once.

After generation, sample beyond the happy path. Failures such as building twelve disconnected demos; adding memory before correctness; no eval baseline; hidden infrastructure work; deploying without tracing or rollback often survive a superficial review because the output has the right shape. Use a counterexample designed to expose the riskiest assumption.

Store the rubric result, reviewer, date, input version, failed cases, and unresolved limitations. If the artifact changes, rerun affected checks. One evolving codebase tells a stronger story than twelve unrelated labs. Every layer must improve a user outcome and leave evidence that the improvement is real. Quality assurance is part of the work, not an apology added at the end.