TKN.

LLM cost & billing glossary

Short, precise definitions for the vocabulary of AI API billing and key security.

Keys & security

  • Admin keyAn organisation-scoped credential that reads billing data and manages other keys. Higher blast radius than an inference key.
  • API keyA credential authenticating requests to a provider. Whoever holds it can spend your money.
  • Blast radiusHow much damage one compromised credential can do. The reason to scope and separate keys.
  • Credential stuffingReplaying leaked credentials in bulk against many targets, hoping some still work.
  • Envelope encryptionEncrypting data with a key that is itself encrypted by a master key, so the two can be stored separately.
  • ExfiltrationUnauthorised removal of data or credentials from an environment.
  • Key revocationPermanently invalidating a credential at the provider. The only action that actually stops a leaked key.
  • Key rotationReplacing a credential with a new one on a schedule or after exposure. Order matters.
  • Key scopeThe boundary of what a credential can reach — a project, an organisation, or a workspace.
  • KeychainApple's OS-level encrypted credential store. Device-only storage that survives app compromise.
  • Kill switchA deliberate control that revokes keys immediately. Distinguished from a dashboard by being able to act.
  • Least privilegeGranting only the access actually required. A read-only need should not carry write permission.
  • LLMjackingTheft and resale of AI API credentials so an attacker gets model access that the key owner pays for.
  • Management keyxAI's administrative credential. Separate from the xai- inference key, and required for billing data.
  • Secret scanningAutomated detection of credentials in code, history and artefacts. Used by defenders and attackers alike.
  • Secure EnclaveA dedicated hardware security processor on Apple devices. Keys it protects never enter main memory in usable form.
  • Service accountA non-human identity that owns credentials, so a key survives the person who created it leaving.

Architecture & agents

  • AgentA loop where a model repeatedly acts, observes results, and decides what to do next until it finishes.
  • Agent loopThe act–observe–decide cycle an agent runs. Needs hard external limits, because the model will not stop itself.
  • EmbeddingA vector representation of text used for search and retrieval. Cheap per call, expensive in bulk.
  • Fine-tuningTraining a model on your own examples. Costs at training time and usually carries a higher inference rate.
  • Function callingThe provider-native mechanism for a model to request a structured function invocation.
  • InferenceRunning a trained model to produce output. The metered, per-token activity you are billed for.
  • LLM gatewayA proxy with routing intelligence — fallbacks, load balancing and a single API across many providers.
  • LLM proxyA service your requests route through, which observes tokens and computes cost per request.
  • MCP (Model Context Protocol)An open protocol for giving models access to external tools and data through a standard interface.
  • MCP serverA process exposing tools to an AI client over the Model Context Protocol, usually over stdio.
  • ObservabilityTracing, logging and metrics for LLM applications. Answers why something happened, not what it cost.
  • RAG (retrieval-augmented generation)Retrieving relevant documents and putting them in the prompt. Retrieved context is billed as input tokens.
  • StreamingReturning tokens as they are generated. Improves perceived latency; changes nothing about cost.
  • Structured outputConstraining a model to emit valid JSON against a schema. Removes retry loops that quietly cost money.
  • Tool callA model's request to invoke a function, and the result returned to it. Each round trip resends the conversation.
  • TracingRecording the steps of a multi-step LLM operation as a linked tree of spans.

Billing & finance

  • Auto-rechargeAutomatic top-up of a prepaid balance from a saved card. It removes the natural cap on a bad day.
  • Billing cadenceHow often billing data updates. Every major AI provider reports daily, not in real time.
  • Budget alertA notification when spend crosses a percentage of a configured budget. Notifies; does not stop.
  • Burn rateThe pace at which you spend. For AI APIs, daily spend is the number that reveals a problem in time.
  • ChargebackBilling internal teams for the AI spend they generate, so budgets sit with the people making decisions.
  • COGS for AICost of goods sold — inference spend belongs here, not in R&D. It sets your gross margin.
  • Cost attributionAssigning spend to a feature, customer, team or environment. Billing APIs attribute by key and project only.
  • Cost per customerInference spend attributable to one customer. The number that determines whether a usage-based plan is profitable.
  • Costs APIA provider endpoint returning spend in dollars, in daily buckets. The number you will actually be invoiced.
  • Gross marginRevenue minus COGS, as a percentage. AI inference costs push it below classic SaaS levels.
  • Hard limitA spend threshold that stops API traffic when reached. The only control that bounds your worst case.
  • Invoice reconciliationMatching API-reported spend against the invoice. They will differ, for predictable reasons.
  • Postpaid billingUsage now, invoice later. Convenient, and it has no natural spending ceiling.
  • Prepaid creditsA balance bought in advance and drawn down by usage. The empty balance is a natural spending cap.
  • ShowbackReporting internal cost by team without charging them. Makes consumption visible without the billing machinery.
  • Soft limitA spend threshold that notifies you but does not stop anything. Useful as a warning, useless as a cap.
  • Spend limitA configured ceiling on API spend. A hard limit stops traffic; a soft limit only notifies.
  • Unit economicsRevenue and cost per unit — user, request, or job. For AI products, inference is usually the dominant variable cost.
  • Usage APIA provider endpoint returning token counts by model, project, key or user. For optimisation, not billing.

Tokens & models

  • Batch APISubmit work for asynchronous processing at a discount, typically around 50%, in exchange for a slower turnaround.
  • BPE (byte-pair encoding)The compression-derived algorithm behind most LLM tokenizers — it merges frequent character pairs into single tokens.
  • Context windowThe maximum tokens a model can consider at once — prompt plus history plus output. A ceiling, not a budget.
  • Cost per million tokensThe standard unit of LLM API pricing. Input and output are quoted separately, and output is usually several times more.
  • Input tokenA token you send to the model — prompt, system message, history and tool output. Cheaper than output tokens.
  • Output tokenA token the model generates. Typically 3–5× the price of an input token, because generation is sequential.
  • Prompt cachingReusing an already-processed prompt prefix at a reduced rate. The single largest cost saving for repeated system prompts.
  • Rate limitA cap on request or token throughput. Protects the provider's capacity — it does not cap your spend.
  • Reasoning tokensInternal tokens a reasoning model generates before answering. Billed as output, invisible in the response.
  • RPM (requests per minute)A throughput limit on request count per minute. Independent of how expensive each request is.
  • TokenThe unit AI APIs bill in. Roughly 3–4 characters of English text — not a word, not a character.
  • TokenizerThe algorithm that splits text into tokens. Each provider uses a different one, so token counts are not comparable.
  • TPM (tokens per minute)A throughput limit on tokens processed per minute, shared across your organisation.