LLM cost tracking without a proxy or an SDK
Last verified
Yes, and it is the default worth starting from. Provider billing APIs report your spend directly. A tool that reads them needs no SDK, no gateway, and no change to your application code — and critically, nothing sits in your request path.
The tradeoff is real and worth stating up front: you get daily granularity and attribution by project, key and model. You do not get per-request or per-customer cost.
The architectural difference
Proxy-based (Helicone, Portkey, LiteLLM gateway): you change your base URL. Every request goes through the vendor, which counts tokens and multiplies by a price table.
SDK-based (Langfuse and similar): you instrument your code. The SDK reports usage to the vendor's backend.
Billing-API-based (TKN, and the provider dashboards themselves): the tool authenticates to the provider's billing API with an admin credential and reads what you were charged. Your application is untouched and unaware.
What billing-API tracking gets you
Nothing in the request path. This is the whole argument. A proxy is an availability dependency — if it degrades, your product degrades. A billing reader cannot affect your application because it never sees a request.
No prompt or response exposure. A proxy necessarily sees your traffic. Billing data is aggregates: dollars, tokens, models. If your prompts contain anything sensitive, this difference is not academic.
Zero integration. No code change, no deploy, no SDK version to keep current. It works identically whether your calls come from a server, a cron job, a notebook, or a teammate's laptop — which the SDK approach does not.
The numbers are the billed numbers. Proxies compute cost from observed tokens and a price table. That table can be stale, and it can't know about your credits, discounts or caching. A billing API returns what you will actually be charged.
It covers traffic you forgot about. Every call on the account shows up, including the prototype from March and the script a teammate runs locally. A proxy only sees what was routed through it — which means the thing most likely to surprise you is the thing it misses.
What it costs you
Be clear-eyed:
Daily, not real-time. Billing APIs report daily. No polling frequency changes that. If you need to catch a runaway loop within minutes, this is the wrong tool.
No per-request cost. You cannot answer "what did this specific call cost".
No per-feature or per-customer attribution beyond what your key and project structure already encodes. One key per feature gets you part of the way; genuine multi-tenant cost-per-customer needs request tagging, which needs a proxy or an SDK.
No caching, no routing, no fallbacks. Proxies bundle real features beyond observation. If you want automatic failover from one provider to another, that requires something in the path.
An admin credential is needed. Which is its own tradeoff — is it safe to give an app your admin key?
When a proxy is genuinely the right answer
Use one — and don't let anyone talk you out of it — if you need:
- Cost per customer in a multi-tenant product, especially for usage-based billing
- Cost per feature or endpoint, to decide what to optimise
- Prompt-level debugging alongside cost
- Automatic failover between providers
- Response caching to cut spend directly
These are real needs and billing APIs cannot serve them. Helicone and Langfuse are good at this; that is what they are for.
When billing-API tracking is right
- You want to know what you're spending without a project to build it
- You cannot or will not put a third party in your production request path
- Your prompts are sensitive
- You use several providers and want one number
- Your traffic is not all in one place — servers, scripts, notebooks, teammates
- What you actually want is "tell me before the invoice does", not "profile my requests"
That last one is most people, most of the time.
Both is fine
They are not mutually exclusive and the combination is common: a proxy for the production service you're optimising, and a billing-API view for the account total that catches everything the proxy never saw. The failure mode a proxy cannot cover — a forgotten key spending money outside your routed traffic — is exactly the one that produces surprise invoices.