LLM proxy

An LLM proxy sits between your application and the provider. You change your base URL; it forwards requests and records tokens, latency and computed cost.

What it buys: per-request cost, per-customer attribution with tagging, caching, and fallback routing.

What it costs: a third party in your production request path — extra latency, an availability dependency, and visibility of prompt and response content. It also only sees traffic routed through it, so keys used elsewhere are invisible.

Proxy vs billing API.