A support agent receives “look up order A-1 and refund $15 if appropriate.” The model can select a tool and propose arguments. That proposal does not prove that the requester owns the order or approved a refund. The server executing the tool must make that decision immediately before it reads data or moves money.
The failure mode
An email, webpage, or tool response can carry instructions that an agent mistakes for an authorized request. OWASP recommends scoped tools, input validation, access checks, and human approval for sensitive actions. A prompt telling an agent to behave safely is useful guidance, but it cannot enforce a permission boundary.
Imagine a support assistant for a marketplace with multiple merchants. It can look up orders and prepare refunds. Each call must pass four gates: an allowlisted tool, a resource inside the authenticated tenant, a user with the right role, and an approval bound to the exact sensitive action. Amounts are expressed in cents to avoid floating-point surprises.
Copyable Python example
Save the following as gate.py and run python3 gate.py. It is a dry run: there is no payment provider or network request.
from dataclasses import dataclass
@dataclass(frozen=True)
class Context:
user_id: str
tenant_id: str
role: str
@dataclass(frozen=True)
class Order:
tenant_id: str
owner_user_id: str
status: str
total_cents: int
def execute_tool(ctx, call, orders, approvals):
name = call.get("name")
if name not in {"lookup_order", "refund_order"}:
raise PermissionError("tool not allowed")
order_id = call.get("order_id")
if not isinstance(order_id, str) or not order_id:
raise ValueError("order_id required")
order = orders.get(order_id)
if order is None or order.tenant_id != ctx.tenant_id:
raise PermissionError("order not available")
if name == "lookup_order":
if ctx.role != "owner" and order.owner_user_id != ctx.user_id:
raise PermissionError("order not available")
return {"order_id": order_id, "status": order.status}
amount = call.get("amount_cents")
if ctx.role != "owner":
raise PermissionError("owner required")
if type(amount) is not int or not 0 < amount <= order.total_cents:
raise ValueError("invalid refund amount")
approval_id = call.get("approval_id")
if approvals.get(approval_id) != (ctx.user_id, order_id, amount):
raise PermissionError("matching approval required")
del approvals[approval_id]
return {"dry_run": True, "refund_cents": amount}
orders = {
"A-1": Order("shop-1", "buyer-1", "shipped", 5000),
"B-1": Order("shop-2", "buyer-2", "pending", 2000),
}
approvals = {"ok-1": ("owner-1", "A-1", 1500)}
owner = Context("owner-1", "shop-1", "owner")
calls = [
{"name": "lookup_order", "order_id": "A-1"},
{"name": "lookup_order", "order_id": "B-1"},
{"name": "refund_order", "order_id": "A-1",
"amount_cents": 2000, "approval_id": "ok-1"},
{"name": "refund_order", "order_id": "A-1",
"amount_cents": 1500, "approval_id": "ok-1"},
{"name": "refund_order", "order_id": "A-1",
"amount_cents": 1500, "approval_id": "ok-1"},
]
for call in calls:
try:
print(execute_tool(owner, call, orders, approvals))
except (PermissionError, ValueError) as error:
print(type(error).__name__, error)
What I actually tested
I ran a local Python 3 version of this example. Looking up A-1 returned its status. The other tenant’s B-1 was blocked. A $20 refund failed against a $15 approval. The approved $15 call returned a dry-run result, and replaying it failed. These checks cover local logic only; I did not test a model, an MCP server, or a payment integration.
In this demonstration, approvals stands for decisions created by a server workflow after authenticating a human. The agent must not invent an approval or grant itself authority. In production, persist the approver, order, amount, currency, expiration, and unique action ID. Consume approvals atomically so two concurrent requests cannot reuse one. Add real authentication, audit logs, spending limits, and a recovery path.
A practical rollout checklist
- List every tool the agent can invoke, separating read-only operations from irreversible changes.
- Derive user and tenant context from the authenticated session, never from model-generated text.
- At execution time, validate tool name, argument types, resource ownership, and permissions. Require exact-action approval for refunds and deletions.
- Test another user’s record, another tenant, a changed amount, and approval replay.
Any unexpected success is a reason to hold the sensitive tool. Passing these local cases is a starting point; concurrency and end-to-end production behavior still need testing.
