Handle failures and retries

Not every failure deserves a retry. Retry bounded transient transport failures only when the operation is safe to repeat. Authentication, authorization, invalid input, unknown capabilities, and schema violations normally fail immediately. Use exponential backoff with jitter and honor a valid server retry hint within your total deadline.

Propagate AbortSignal through every layer. A timeout that stops waiting but leaves upstream work running still consumes money and capacity. Preserve a typed cause and request ID, but never include secrets or raw sensitive payloads in user-facing messages.

Build a failure table for duplicate names, stale schemas, timeouts, oversized results, and unsafe side effects. Give each row an owner, user message, retryability, maximum attempts, and terminal state. A retry is appropriate only when a fresh attempt could succeed and replay is safe. Use full jitter, cap delays, and include all attempts inside one deadline. Provider “retry-after” hints are advisory input: parse and cap them.

After exhaustion, return a stable degraded result or typed failure. Never spin forever. Preserve partial work only if the domain contract defines how callers distinguish partial from complete. The expected behavior remains calculate({expression:"6*7"}) returns 42; an unknown tool is rejected without execution. on recovery, while permanent failures must complete quickly and without accidental side effects.

for (let attempt = 0; attempt <= maxRetries; attempt++) {
  try { return await operation(signal); }
  catch (error) {
    const failure = classify(error);
    if (!failure.retryable || attempt === maxRetries) throw failure;
    await sleep(jitteredBackoff(attempt), signal);
  }
}
throw new Error("unreachable");

The code is intentionally provider-neutral at the adapter boundary. AI_BASE_URL and model names are environment placeholders, not invented services. If the topic uses MCP, install the current official SDK and verify its exported paths against its release documentation. If a provider offers an official SDK, it can replace fetch inside execute without changing the domain contract. Always inspect the real provider’s documented request and response fields before connecting credentials.

Debug from deterministic code outward. First print the parsed configuration with secret values replaced by [set]. Next run the parser against a local fixture. Then run the orchestration with a fake dependency. Only after those pass should you inspect the real boundary. Capture status, content type, request ID, elapsed time, and a redacted response shape. Do not “fix” malformed data with a permissive cast; save it as a failing fixture and decide whether the contract or adapter is wrong.

For this chapter, the fastest diagnostic is to assert that tool descriptions, validators, and implementations agree on names and shapes. When that assertion fails, stop before downstream work. Expected behavior is concrete: calculate({expression:"6*7"}) returns 42; an unknown tool is rejected without execution. A different but plausible sentence is not enough if the structural and policy checks fail.

• Retryable and permanent failures are separated.
• Attempts, deadline, and backoff are bounded.
• Cancellation aborts upstream work.
• Run npm run typecheck and npm test without provider credentials.
• Record expected terminal behavior beside the fixture so future changes remain reviewable.

A passing test is necessary but does not establish production quality. Review the fixture set for realistic distributions, stale assumptions, tenant boundaries, and expensive edge cases. When outputs are probabilistic, assert schemas, citations, chosen capabilities, stop reasons, and task rubric scores. Reserve exact string assertions for deterministic adapters and protocol framing.