Tools: when models need the outside world
Safe execution boundaries
Safe tool use requires several independent gates: valid data, authenticated identity, authorized action, least privilege, and duplicate-resistant execution.
Before you start
Why this matters
A model proposes refund_order with a valid-looking order ID and amount. The JSON matches the schema. Is it safe to run?
Not yet. The order may belong to another customer. The signed-in employee may lack refund authority. The amount may exceed policy. The user may have asked only what a refund would look like. A previous attempt may already have succeeded. Schema validity answers “Is this shaped correctly?” It does not answer “Should this actor perform this action now?”
1Learn the idea
Read
Validate syntax and meaning
Start with structural validation: known tool name, valid JSON, required fields, expected types, allowed enum values, lengths, formats, and no unexpected properties. Reject malformed calls before they reach business logic.
Then apply semantic rules. For a calendar event, the end must follow the start, the duration must be within limits, and the timezone must be supported. For a refund, the amount must be positive, use the order currency, not exceed the refundable balance, and comply with policy.
{
"name": "refund_order",
"arguments": {
"order_id": "ord_3192",
"amount_minor": 2500,
"currency": "USD",
"reason": "duplicate_charge"
}
}
Using integer minor units avoids ambiguous floating-point money values. An enum for reason gives audit and policy code stable categories. Still, the application must load the authoritative order and compare the proposal with current state.
Never repair dangerous arguments silently. If the model proposes an end time before a start time, ask for clarification or produce a new proposal. Quietly swapping them may create an event the user never approved.
Read
Authenticate identity outside the model
Authentication establishes who is making the request. It comes from a trusted session, signed token, service identity, or equivalent mechanism—not a sentence in the conversation.
Do not expose fields such as is_admin, tenant_id, or acting_user_id for the model to set. The application derives these from trusted context:
{
"trusted_context": {
"actor_user_id": "usr_204",
"tenant_id": "tenant_blue",
"roles": ["support_agent"]
},
"untrusted_proposal": {
"order_id": "ord_3192",
"amount_minor": 2500
}
}
The distinction remains even if the user is legitimate. Prompt injection in retrieved content or an accidental model output could try to alter identity claims. Trusted identity must never be overwritten by generated data.
Service-to-service calls need identity too. Record both the human actor and the executing service so an audit can explain who initiated the operation and which component performed it.
Read
Authorize the exact action
Authorization asks whether that identity may perform this operation on this resource under current conditions. It is narrower than “the user is logged in.”
A support agent may view an order but refund only up to $50. A manager may approve larger refunds. A customer may refund only their own eligible order. An application should check tool, resource ownership, tenant, amount, state, time window, and approval evidence in deterministic code.
An allowlist limits which tools are available in the current route or workflow. It should be generated per request. However, hiding a tool from the model is defense in depth, not the sole check. The executor must reject any call not currently authorized, even if a model or client sends it directly.
For high-impact operations, use human review or dual control. Show the reviewer the exact side effect and relevant source data, not merely the assistant’s persuasive summary.
Read
Apply least privilege and data minimization
Give each tool the smallest provider permission it needs. A weather read does not need calendar scope. A calendar availability tool may not need event titles. A draft-email tool does not need permission to send.
Separate read and write credentials where possible. Constrain database queries by tenant in code. Restrict network destinations and file paths. Set size, cost, rate, and time limits. These boundaries reduce damage if the model chooses poorly or hostile content influences its proposal.
Results should be minimized too. If the model needs available time blocks, return busy intervals rather than private event descriptions. If it needs to confirm a refund, return the amount and status rather than an entire payment-provider payload.
Treat tool results as untrusted content. A web search result or document can contain instructions aimed at the model. Label results as data, keep system policy authoritative, and never let returned text expand the allowlist or bypass confirmation.
Read
Make writes idempotent
Networks fail in ambiguous ways. The application may send a request, lose the response, and retry. Without protection, one intended payment can become two.
Idempotency means repeating the same logical request does not repeat the side effect. Generate a key from the operation or approved proposal and store the first outcome:
{
"idempotency_key": "refund:ord_3192:approval_8ac1",
"operation": "refund_order",
"arguments_hash": "sha256:4b9d...",
"actor_user_id": "usr_204"
}
The provider or application should return the original result when the same key and arguments recur. If the key is reused with different arguments, reject it. Keep the key long enough to cover realistic retry and replay windows.
Idempotency does not authorize an operation. It prevents duplication after the other gates pass. Nor does it mean every operation is naturally safe to repeat: “set account status to closed” may be idempotent, while “add $10 credit” is not unless wrapped in a unique transaction.
Read
Bind approval to execution
A vague “yes” should not authorize a changed proposal. Bind confirmation to a canonical representation of tool name, resource, arguments, actor, and expiry. If any material field changes, obtain new approval.
Just before execution, recheck mutable facts. An order may have been refunded by another agent. A calendar slot may now be occupied. Authorization may have been revoked. This time-of-check/time-of-use gap is why a valid earlier proposal cannot be trusted forever.
Record the proposal, validation outcomes, approval identity, idempotency key, execution result, and timestamps under one trace ID. Logs should redact credentials and sensitive content while preserving accountability.
Continue learning · glossary & guides
- Can you explain why schema validation is not authorization?
- Which fields must always come from trusted application state?
- How does idempotency differ from permission?
- Glossary: tool allowlist · Snippet: tool allowlist check · Lesson: Guardrails in code