Define the lab goal and success criteria

Begin with one vertical slice and measurable acceptance criteria. Separate model quality from software correctness: transport, parsing, authorization, and termination must be deterministic even when model content varies. The lab is complete only when a fixture can demonstrate the expected behavior and a negative fixture demonstrates a safe rejection.

Write a README-level contract before installing packages. Record the input, output, owner, latency ceiling, cost ceiling, and stop condition. This prevents an attractive demo from silently becoming an unbounded production promise.

Success has two layers. Software success means tool descriptions, validators, and implementations agree on names and shapes. Product success means a user receives a useful result without hidden authority or an endless wait. For this lab, the concrete oracle is: calculate({expression:"6*7"}) returns 42; an unknown tool is rejected without execution. Measure fixture pass rate, safe-rejection rate, p95 duration, and external calls per successful task. Do not use “the response looked good” as an acceptance test.

Create src/, test/, .env.example, and package.json. The first deliverable is not a framework; it is a fixture containing representative input, expected structural properties, and prohibited behavior. A useful negative fixture targets duplicate names, stale schemas, timeouts, oversized results, and unsafe side effects. That negative case forces the boundary to exist before provider output can surprise it.

export const acceptance = {
  project: "a small tool registry containing a calculator and a read-only knowledge lookup",
  invariant: "tool descriptions, validators, and implementations agree on names and shapes",
  expected: "calculate({expression:\"6*7\"}) returns 42; an unknown tool is rejected without execution.",
} as const;

The code is intentionally provider-neutral at the adapter boundary. AI_BASE_URL and model names are environment placeholders, not invented services. If the topic uses MCP, install the current official SDK and verify its exported paths against its release documentation. If a provider offers an official SDK, it can replace fetch inside execute without changing the domain contract. Always inspect the real provider’s documented request and response fields before connecting credentials.

Debug from deterministic code outward. First print the parsed configuration with secret values replaced by [set]. Next run the parser against a local fixture. Then run the orchestration with a fake dependency. Only after those pass should you inspect the real boundary. Capture status, content type, request ID, elapsed time, and a redacted response shape. Do not “fix” malformed data with a permissive cast; save it as a failing fixture and decide whether the contract or adapter is wrong.

For this chapter, the fastest diagnostic is to assert that tool descriptions, validators, and implementations agree on names and shapes. When that assertion fails, stop before downstream work. Expected behavior is concrete: calculate({expression:"6*7"}) returns 42; an unknown tool is rejected without execution. A different but plausible sentence is not enough if the structural and policy checks fail.

• One positive fixture and one adversarial fixture are written.
• Success, cost, latency, and stop conditions have numeric targets.
• Provider variability is separated from deterministic software behavior.
• Run npm run typecheck and npm test without provider credentials.
• Record expected terminal behavior beside the fixture so future changes remain reviewable.

A passing test is necessary but does not establish production quality. Review the fixture set for realistic distributions, stale assumptions, tenant boundaries, and expensive edge cases. When outputs are probabilistic, assert schemas, citations, chosen capabilities, stop reasons, and task rubric scores. Reserve exact string assertions for deterministic adapters and protocol framing.