TKN.

The runaway loop — how a bug becomes a four-figure bill

Last verified

A runaway loop is the most common cause of a surprise AI bill — more common than a leaked key, and much easier to cause. It is usually five lines of ordinary-looking code.

The five patterns

1. Retry without a cap

# every failure retries forever, and every retry is billed
while True:
    try:
        return client.chat.completions.create(...)
    except Exception:
        continue

The insidious part: a _successful but unsatisfactory_ response is not an exception. Code that retries until the output parses can loop indefinitely against a model that keeps producing something almost-right — and every attempt is charged.

Fix: a maximum attempt count, exponential backoff, and a distinction between retrying transport errors and retrying bad content.

2. The agent that won't stop

An agent loops until it decides it's done. When it can't complete the task — a broken tool, a missing file, an impossible goal — "done" never arrives. It tries, fails, reasons about the failure, tries again. Each cycle carries the full conversation history, so cost per step grows as the transcript lengthens.

This is the expensive one, because it accelerates.

Fix: a hard step cap, a cumulative token budget checked between steps, and a repeated-state check that terminates when the agent tries the same failing action twice. Cost control for autonomous agents.

3. Recursion without a depth limit

Summarise a document; if too long, split and summarise each part; each part splits again. Without a floor, this fans out geometrically. Ten items becomes a hundred calls becomes a thousand.

Fix: a depth limit and a total-call budget for the whole operation, not per branch.

4. Batch fan-out over a bigger dataset than you tested

Works beautifully on 10 rows. Someone points it at 100,000. Nothing is technically wrong — it is simply 10,000× the cost, executed faithfully.

Fix: estimate before you run. Rows × average tokens × price. If the number is uncomfortable, use the Batch API, a cheaper model, or a sample. Do the arithmetic _first_; it takes a minute.

5. Tests against the real API

An integration test suite that calls production endpoints, running on every push, across a matrix, with a flaky test that retries.

Fix: mock by default. Use recorded fixtures. If you must call the real API in CI, do it on a schedule with a dedicated key in a dedicated project with its own low spend limit.

Guards worth having

In your code:

  • Maximum retry counts everywhere, with backoff
  • A cumulative token budget per operation, checked between steps
  • A wall-clock timeout on anything agentic
  • Cost estimation before any batch job
  • A separate project and key for anything experimental

On your account:

  • A hard spend limit, set below the number that would ruin your week
  • Auto-recharge off, so a drained balance actually stops
  • Spend alerts at 50% and 90%

Operationally:

  • A daily look at spend. Nothing else catches "yesterday cost 40× normal" in time to matter.

Estimating a batch job first

ROWS = 100_000
AVG_INPUT_TOKENS = 800
AVG_OUTPUT_TOKENS = 200
INPUT_PER_M = 2.50    # check your model's current price
OUTPUT_PER_M = 10.00

cost = (ROWS * AVG_INPUT_TOKENS / 1e6) * INPUT_PER_M \
     + (ROWS * AVG_OUTPUT_TOKENS / 1e6) * OUTPUT_PER_M
print(f"${cost:,.2f}")

Run this before, not after. It takes a minute and it has saved people four-figure sums.

If it's happening right now

  1. Stop the process. Kill the job, scale the worker to zero, disable the cron.
  2. If you can't reach it fast, revoke the key — every request fails immediately. How to revoke an OpenAI API key.
  3. Turn off auto-recharge so the balance stops refilling.
  4. Then work out what happened.

Revoking is drastic and it is the correct move when you cannot reach the process. An outage you chose beats a bill you didn't.

Sources

Related