Worked case: payments email

An unsafe workflow looks efficient:

Email arrives → AI decides “duplicate” → refund tool runs → AI sends confirmation

The team may justify this by saying duplicate-payment requests are routine. Yet the message alone is not evidence. The customer might see a pending authorization and a settled charge, two valid shipments, or a charge from another account. The refund tool may fail after the email is sent.

The shortcut combines four separate decisions:

• Is this the correct customer and order?
• Are there actually two settled charges?
• Does policy allow this refund?
• Did the refund execute successfully?

Because these decisions are hidden inside one fluent response, a reviewer cannot tell where an error occurred.

The assistant extracts the order number and requested amount, then retrieves:

• customer and order identifiers;
• payment transaction IDs;
• status of each transaction;
• amount, currency, and timestamps;
• prior refunds or disputes;
• applicable refund policy.

Retrieval should use the customer’s authorized account context. The model should not search unrelated accounts because a name looks similar. If the order number is missing or belongs to another account, the workflow asks for clarification rather than guessing.

The system labels each item as source fact or model interpretation. “Transaction T2 is settled” is a source fact from the payment provider. “Likely accidental duplicate” is an interpretation that still needs policy evaluation.

Code—not free-form model judgment—checks facts that have exact rules:

• both charges have the same merchant, amount, and currency;
• both are settled rather than one being pending;
• no refund already exists;
• the refund amount is within the employee’s authority;
• the request is inside the allowed time window.

The AI can explain the result, but deterministic checks should enforce hard limits. If the configured approval threshold is $50, the model cannot talk the system into refunding $89 automatically.

Suppose the record shows two settled $89 charges three minutes apart, one shipment, and no prior refund. The checks pass, but the amount exceeds the auto-refund threshold. The workflow prepares a recommendation for review.

The reviewer sees:

Request: Customer claims duplicate $89 charge for order 4812.

Evidence: Transactions T1 and T2 are settled, same amount and currency, three minutes apart; one order and one shipment; no previous refund.

Policy: Duplicate settled charges may be refunded. Amounts above $50 require approval.

Proposed action: Refund T2 for $89 to the original payment method.

Customer draft: “We confirmed a duplicate charge and approved an $89 refund. You will receive a confirmation after it is processed.”

Warnings: None. If evidence were incomplete, the warning would be prominent rather than omitted.

The reviewer can approve, reject, edit the amount, or escalate to payments operations. The interface requires a reason for rejection or changes so recurring system errors become visible.

Approval does not mean the refund happened. The application calls the payment provider using an idempotency key so a retry does not create two refunds. It waits for a confirmed result.

Only after success does the system send a confirmation containing the provider’s reference and a realistic timing statement. If execution fails, it does not send the success draft. It records the error and routes the case for retry or manual handling.

This ordering—approve, execute, verify, notify—prevents a common failure: promising an action that never completed.

Now change one fact: T2 is a pending authorization, not a settled charge. The AI still produces a convincing duplicate-charge explanation.

The deterministic check fails. The workflow blocks the refund proposal and drafts a different response explaining that pending authorizations may disappear, using approved policy language. If the pending item later settles, the customer can reopen the case.

Another bad path: the payment provider times out after approval. The system uses the idempotency key to check status before retrying. It never assumes timeout means failure, because the provider may have processed the refund while the response was lost.

Track more than “AI accuracy”:

• percentage of proposals approved unchanged;
• percentage edited, rejected, or escalated;
• incorrect refunds prevented by checks or reviewers;
• duplicate executions prevented;
• time from request to decision and from approval to confirmation;
• complaints or reversals after approval;
• failure rates by amount, market, and request type.

High approval rates can be good, but inspect them alongside review time and sampled quality. A 99% approval rate achieved by rubber-stamping is not evidence of safety.