TKN.

Cost control for autonomous agents

Last verified

An agent decides how many tokens to spend, and you don't find out until it's done. That's the whole problem. A normal API call has a cost you can estimate before you make it; an agent run does not.

Why cost accelerates

The mechanism people underestimate: each step resends the accumulated context.

Step 1 sends a small prompt. Step 20 sends the original prompt plus nineteen steps of reasoning, tool calls, and tool output. Cost per step climbs as the run continues, so a 40-step run costs much more than twice a 20-step run.

Add three multipliers:

  • Tool output is uncontrolled. A tool that returns a 50,000-token file has just put that in the context permanently.
  • Reasoning tokens are billed and invisible. They're output tokens you never see in the response.
  • Failure is expensive. An agent that can't complete its task doesn't stop — it retries, reasons about the failure, and retries again, with a growing transcript each time.

The worst case isn't "the agent did the task expensively." It's "the agent could not do the task and spent your budget discovering that."

The five guards

All enforced in your code. None of them in the prompt.

1. A hard step cap

MAX_STEPS = 25
for step in range(MAX_STEPS):
    result = agent.step()
    if result.done:
        break
else:
    raise AgentBudgetExceeded("hit step cap without completing")

Boring and it works. Pick a number from observed successful runs, then add headroom.

2. A cumulative token budget

Better than a step cap, because steps are not equal in size.

MAX_TOKENS = 500_000
used = 0
while used < MAX_TOKENS:
    result = agent.step()
    used += result.usage.input_tokens + result.usage.output_tokens
    if result.done:
        break

Track input and output separately if you want a dollar figure — output typically costs several times more per token.

3. A wall-clock timeout

Catches everything the other two miss, including an agent stuck waiting on a tool.

4. A repeated-state check

The highest-value guard, and the one most often missing.

If the agent tries the same action with the same arguments twice, it is looping. Hash each tool call and terminate on a repeat:

seen = set()
sig = hash((tool_name, json.dumps(args, sort_keys=True)))
if sig in seen:
    raise AgentStuck(f"repeated action: {tool_name}")
seen.add(sig)

A stuck agent will otherwise burn its entire budget rediscovering the same failure.

5. Bounded tool output

Truncate anything a tool returns before it enters context. A tool that reads a file should read _part_ of a file, or return a summary. Unbounded tool output is the most common way a run's cost explodes for reasons unrelated to the task's difficulty.

What does not work

Telling the model to be frugal. "Be efficient with tokens" in a system prompt is a suggestion. It is not a constraint, and it will be ignored under pressure to complete the task.

Relying on provider spend limits. They're organisation-wide and lag usage. A limit stops the agent _and_ your production traffic, well after the damage.

Checking cost afterwards. By definition too late. The check has to be between steps.

Estimating before you run

STEPS = 25
AVG_CONTEXT_GROWTH = 2_000        # tokens added per step
BASE_CONTEXT = 5_000
INPUT_PER_M, OUTPUT_PER_M = 3.00, 15.00   # use your model's current prices

total_in = sum(BASE_CONTEXT + i * AVG_CONTEXT_GROWTH for i in range(STEPS))
total_out = STEPS * 800
print(f"~${total_in/1e6*INPUT_PER_M + total_out/1e6*OUTPUT_PER_M:.2f} per run")

Then multiply by how many runs a day you expect. This is the number that surprises people, and it takes a minute to compute.

Watching it

Guards bound one run. They don't tell you that agent runs across your account tripled this week.

For that you need spend at the account level compared against normal — which is what TKN does across OpenAI, Anthropic, xAI and Gemini, with a kill switch for when the answer is bad enough to stop everything.

And if you want the agent itself to be able to check, that's what the MCP server is for.

Sources

Related