Debug string-eval calculator in the tool registry runtime

Do not start with a speculative fix for the tool registry runtime. Force the failure on purpose, save the before output, then change one cause at a time. Retries are allowed only for transient conditions—not for bad input that will fail forever on tools-in-code.

for (let attempt = 0; attempt <= maxRetries; attempt++) {
  try { return await operation(signal); }
  catch (error) {
    const failure = classify(error);
    if (!failure.retryable || attempt === maxRetries) throw failure;
    await sleep(jitteredBackoff(attempt), signal);
  }
}
throw new Error("unreachable");

Expected evidence: duplicate names, stale schemas, timeouts, oversized results, and unsafe side effects. If you cannot reproduce on demand, you do not yet control the failure mode for tools-in-code.

After repair, rerun the exact reproduction command. Keep the failing fixture as a regression seed for the observability page. For the calculator + knowledge lookup tool registry, remember the claim you are restoring: unknown tools are rejected; calculator and lookup return typed results.

Store a one-command reproduction for: string-eval calculator, or lookup that returns entire DB on missing id. The command should use registry with calc and kb.lookup or a minimal mutant of it. Paste the failing output into notes/failure-before.txt (or your shell scrollback as copied text). After the fix, paste notes/failure-after.txt and keep both.

Retries belong only on transient faults. If the failure is bad input, a bad allowlist, or a logic bug in the tool registry runtime, retrying will amplify cost without repairing trust around route model tool requests through a registry that validates names and args.

Classify the failure as prevent, detect, contain, or recover—using this lab’s language, not a generic poster. For tools-in-code, the first fix should usually be detect+prevent at the boundary, because string-eval calculator, or lookup that returns entire DB on missing id is cheaper to stop early than to explain in production prose.

At the debugging stage for tools-in-code, the job is narrower than finishing a product demo. You are creating one progressive evidence piece about registry with calc and kb.lookup that later pages inherit without redefining success. Keep that fixture small enough to inspect by hand, keep outputs copy-pasteable as text, and refuse to narrate this baseline as if it were a production SLA: inline if/else tool routing without schemas.

For this page specifically, success looks like before/after evidence for the characteristic failure while still centering the user decision to route model tool requests through a registry that validates names and args. If you cannot point to a file, command, or assertion that proves that for the tool registry runtime, stay on this page instead of advancing.

Glossary: tool · Glossary: structured output · Cheatsheet: production ops signals

Previous · Next