Debug executing a hallucinated tool name in the order-status tool caller

Do not start with a speculative fix for the order-status tool caller. Force the failure on purpose, save the before output, then change one cause at a time. Retries are allowed only for transient conditions—not for bad input that will fail forever on function-calling-code.

for (let attempt = 0; attempt <= maxRetries; attempt++) {
  try { return await operation(signal); }
  catch (error) {
    const failure = classify(error);
    if (!failure.retryable || attempt === maxRetries) throw failure;
    await sleep(jitteredBackoff(attempt), signal);
  }
}
throw new Error("unreachable");

Expected evidence: A request for ord_123 produces one validated lookup and a reply based only on its result.. If you cannot reproduce on demand, you do not yet control the failure mode for function-calling-code.

After repair, rerun the exact reproduction command. Keep the failing fixture as a regression seed for the observability page. For the order-status function-calling assistant, remember the claim you are restoring: when the model emits a valid tool call, the runtime executes only the allowlisted function.

Store a one-command reproduction for: executing a hallucinated tool name, or answering status without a tool call. The command should use orders map + tool schema for get_order_status or a minimal mutant of it. Paste the failing output into notes/failure-before.txt (or your shell scrollback as copied text). After the fix, paste notes/failure-after.txt and keep both.

Retries belong only on transient faults. If the failure is bad input, a bad allowlist, or a logic bug in the order-status tool caller, retrying will amplify cost without repairing trust around let the model request a read-only get_order_status tool, then answer from the tool result.

Classify the failure as prevent, detect, contain, or recover—using this lab’s language, not a generic poster. For function-calling-code, the first fix should usually be detect+prevent at the boundary, because executing a hallucinated tool name, or answering status without a tool call is cheaper to stop early than to explain in production prose.

At the debugging stage for function-calling-code, the job is narrower than finishing a product demo. You are creating one progressive evidence piece about orders map + tool schema for get_order_status that later pages inherit without redefining success. Keep that fixture small enough to inspect by hand, keep outputs copy-pasteable as text, and refuse to narrate this baseline as if it were a production SLA: assistant that answers without tools on the same prompts.

For this page specifically, success looks like before/after evidence for the characteristic failure while still centering the user decision to let the model request a read-only get_order_status tool, then answer from the tool result. If you cannot point to a file, command, or assertion that proves that for the order-status tool caller, stay on this page instead of advancing.

Glossary: tool · Glossary: structured output · Cheatsheet: production ops signals

Previous · Next