Errors, retries, and truthful failure

Different failures need different responses. A useful taxonomy includes:

• Validation error: arguments are missing, malformed, contradictory, or outside the schema.
• Authorization error: the actor lacks permission or the credential scope is insufficient.
• Not found or conflict: the resource does not exist, changed state, or collides with another operation.
• Rate limit or temporary service error: the provider is overloaded or asks the client to slow down.
• Timeout or network error: the client did not receive a conclusive response.
• Permanent provider rejection: the operation violates provider rules or uses unsupported data.
• Internal bug: application code failed while preparing, invoking, or normalizing the tool.

Return a structured result rather than a paragraph the model must interpret:

{
  "ok": false,
  "error": {
    "code": "RATE_LIMITED",
    "retryable": true,
    "retry_after_ms": 2000,
    "safe_message": "The calendar service is temporarily busy."
  }
}

Do not send raw stack traces, SQL fragments, credentials, or provider internals to the model or user. Keep detailed diagnostics in protected logs linked by trace ID.

Retries help with transient failures such as a rate limit, a dropped connection before any request was sent, or a provider’s temporary 503 response. They do not fix invalid arguments, missing permission, a nonexistent resource, or a policy rejection. Repeating permanent failures wastes time and may trigger abuse controls.

Use bounded exponential backoff with jitter so many clients do not retry in lockstep. Respect provider retry guidance. Set a total time budget and maximum attempts. A conceptual policy might be:

{
  "max_attempts": 3,
  "initial_delay_ms": 250,
  "backoff_multiplier": 2,
  "jitter": true,
  "total_budget_ms": 5000
}

Retries should normally occur in application code, not through repeated model deliberation. The application understands transport details and can preserve one logical operation. Asking the model to emit the same call again costs tokens, changes arguments unpredictably, and complicates duplicate protection.

For writes, retry only with an idempotency key or a reliable status-check mechanism. A read can usually be repeated without a side effect, though it still has cost and privacy implications.

A timeout means the application stopped waiting. It does not necessarily mean the provider stopped working. This distinction is especially important for writes.

If a weather read times out, the application can safely report that no forecast was retrieved and perhaps retry within budget. If a payment creation times out after bytes were sent, the state is unknown until verified. Do not label it failed and immediately create another payment.

Prefer a workflow with an operation ID or idempotency key:

{
  "operation_id": "op_71c9",
  "idempotency_key": "payment:invoice_882:approval_d11f",
  "status": "unknown",
  "next_action": "query_status"
}

The application queries status. If the provider confirms completion, return success. If it confirms rejection, return failure. If status remains unknown, escalate or tell the user that confirmation is pending. This may feel less satisfying than a confident answer, but it is accurate.

Set separate connection, read, and overall workflow timeouts when the platform supports them. A short interactive deadline may coexist with background reconciliation, but the user-facing message must explain that distinction.

When a call fails, choose the least risky useful next step:

1. Correct a validation issue using known evidence, or ask for missing information.
2. Retry a transient read within the attempt and time budget.
3. Verify an ambiguous write by operation ID or idempotency key.
4. Offer a supported fallback that preserves the user’s intent.
5. Escalate to a person or queue when consequences require investigation.
6. Stop and report the limitation truthfully.

A fallback should not silently weaken safety. If the email tool is unavailable, showing a draft is reasonable; pretending it was sent is not. If a private search fails, switching to public web search may disclose the query or provide irrelevant evidence. Ask before changing data sources when privacy or meaning changes.

The model should not improvise unsupported provider behavior. Give it normalized codes and guidance such as “ask the user to choose another time” for CALENDAR_CONFLICT, while keeping retry and policy decisions deterministic where possible.

Final wording should map to observed state:

• Confirmed success: “The event was created for 4:00 p.m.”
• Confirmed failure: “The calendar rejected the event because that calendar is read-only.”
• Safe read failure: “I couldn’t retrieve the forecast after two attempts.”
• Unknown write: “The request was sent, but I couldn’t confirm whether the event was created. I’m checking its status; avoid creating another copy.”
• Partial success: “The event was created, but invitations were not sent.”

Avoid “Done,” “Booked,” or “Sent” unless the result proves that exact claim. An accepted job may still be pending. A generated draft is not delivery. A provider’s 200 response may contain an application-level error.

Partial results should be explicit. If two of three lookups succeed, name the missing source and avoid conclusions that depend on it.

Record attempt count, latency, provider code, timeout stage, idempotency key, status-check outcome, fallback, and final wording. Aggregate these signals to find unreliable tools, overly short deadlines, bad schemas, or recurring authorization gaps.

Protect the system from retry storms and model loops. Cap calls per request, enforce cost and wall-clock budgets, detect repeated identical failures, and stop when no new evidence is available. An agent or model repeatedly calling a broken tool is not persistence; it is uncontrolled failure amplification.

Test failure paths intentionally. Simulate rate limits, malformed results, slow responses, dropped connections after writes, conflicts, and partial provider outages. A demo that tests only successful calls has not tested the tool system’s most important claims.