LLM cost tracking tools — which one for which job
Last verified
Every comparison in this space is written by a vendor, including this one. So here is the rule we'll hold to: each tool gets the case where it is genuinely the right answer, including when that means not using ours.
The three architectures
Almost every tool here is one of three things, and the architecture determines what it can do.
Proxy / gateway — you change your base URL; requests route through the vendor. _Helicone, Portkey, LiteLLM._ Sees every request, so it can do per-request cost, caching, and fallbacks. Cost: a third party in your production request path.
SDK / instrumentation — you add their library to your code. _Langfuse, Traceloop, LangSmith._ Rich traces without a proxy hop. Cost: code changes, and it only sees what you instrumented.
Billing API reader — reads the provider's own billing data with an admin credential. _Provider dashboards, TKN._ Nothing in your request path, no code change, and the numbers are the billed numbers. Cost: daily granularity, no per-request detail.
What each is for
Helicone — proxy
Best for per-request cost and latency with minimal setup. Change a base URL and you have per-request visibility, caching, and a generous free tier.
Pick it if you want request-level cost quickly and are comfortable with a proxy in your path. Skip it if you can't route production traffic through a third party, or your prompts are sensitive.
Langfuse — SDK, open source
Best for deep tracing and self-hosting. Traces multi-step chains and agents properly, and you can run it on your own infrastructure — which resolves the data-residency objection entirely.
Pick it if you're debugging agent behaviour, or compliance means the data can't leave your estate. Skip it if you only want a spend figure — it's a lot of machinery for that.
Portkey — gateway
Best for multi-provider routing with cost attached. Fallback chains, load balancing across providers, and cost dashboards tied to routing decisions.
Pick it if you're already routing across many models and want one control plane. Skip it if you use one or two providers directly — the gateway is the product, and you'd be paying for the part you don't need.
ccusage and friends — local log readers
Best for Claude Code and Codex session costs. Reads local session logs, needs no credential at all, free and open source.
Pick it if you want to know what your coding agent costs. Skip it if you need metered API spend across providers — different data source entirely.
Provider dashboards — free
Best for a definitive answer to a specific question. Accurate, official, zero setup.
Pick it if you want the number once. Skip it if the problem is that nobody remembers to look — which is the actual problem most of the time.
TKN — billing API reader, on a phone
Best for knowing your combined spend before the invoice, and being able to stop a leak. Reads OpenAI, Anthropic, xAI and Gemini billing APIs, one figure with a per-provider breakdown, keys in the device Keychain behind Face ID, and a kill switch that revokes at the provider.
Pick it if you want a daily number across providers without building anything, and you care about being able to respond when you're not at a desk.
Skip it if you need per-request, per-feature or per-customer attribution. TKN cannot do that, and no billing-API tool can. Use a proxy.
Decision table
| If your question is | Use |
|---|---|
| "What did this specific request cost?" | Helicone or Portkey |
| "Which customer is unprofitable?" | A proxy with request tagging |
| "Why is my agent doing that?" | Langfuse |
| "What does Claude Code cost me?" | ccusage, or a local log reader |
| "What are we spending across all providers?" | A billing API reader |
| "How do I hear about a spike before the invoice?" | A tracker with thresholds |
| "How do I stop a leaked key right now?" | TKN, or the provider console |
| "I want the number once, for free" | The provider dashboard |
What nobody in this category does well
Worth saying, since it applies to us too:
- Real-time from billing data is impossible. Anything claiming real-time is proxying or estimating. Why billing APIs are daily.
- Cross-provider totals never exactly match the invoices. Credits, discounts, tax and timezone boundaries all move the number.
- Almost none of these can act. They report. Stopping the spend is still a manual trip to a console — which is the specific gap TKN's kill switch is built for.
The common setup
Not either/or. A very reasonable stack:
- A proxy on the production service you're actively optimising, for per-request detail
- A billing-API view for the account total — which catches the prototype key, the local script, and the teammate's notebook that the proxy never sees
- A hard spend limit at each provider as a backstop
The failure a proxy structurally cannot catch is spend that didn't go through it. That is usually the spend that surprises you.