Security Resources

⌘K
  1. Home
  2. Security Resources
  3. Secure Code Explain
  4. LLM06: Excessive Agency in AI Agents

LLM06: Excessive Agency in AI Agents

Here is an example of Python code that is vulnerable to excessive agency:

🥺 Vulnerable Code

tools = [
    shell_tool,                          # arbitrary commands
    sql_tool(connection=admin_pool),     # full read and write
    email_tool(send_as="billing@example.com"),
    refund_tool(),                       # no amount limit
]

agent = Agent(model="support-agent", tools=tools, max_iterations=50)

# The message is written by whoever opened the ticket
agent.run(customer_message)

The agent holds the application's full privileges and the input comes from the public. One injected instruction inside a support ticket turns into an unlimited refund, a query across every customer, or mail sent from your billing address. Fifty iterations with no time budget also means a loop can run for a long while before anyone notices, and nothing here records what the agent actually did.

😎 Secure Code

Here is a version of the same code that is secured against excessive agency:

tools = [
    read_orders_tool(connection=readonly_pool,
                     scope=lambda: {"customer_id": ctx.customer_id}),   # server-side scope
    refund_tool(max_amount=Decimal("50.00"),
                currency="USD",
                requires_approval_above=Decimal("10.00")),
    reply_tool(templates=SUPPORT_TEMPLATES),      # no free-form outbound email
]

agent = Agent(model="support-agent", tools=tools, max_iterations=6, timeout_s=30)
result = agent.run(customer_message, context=ctx)

for call in result.tool_calls:
    audit.write(actor="support-agent", customer=ctx.customer_id, tool=call.name,
                args=call.args, decision=call.decision, run_id=result.id)

if result.pending_approval:
    queue_for_human(result)

Each tool carries the smallest privilege that still does the job, and the customer id is taken from the session context rather than from anything the model produced - that single change removes most cross-account abuse. Money has a hard cap and a human approval threshold, replies come from templates, iterations and wall-clock time are bounded, and every call is auditable by run id. Anything irreversible waits for a person.