Rate limit
Last verified
A rate limit caps how much you can send in a window, usually expressed as requests per minute (RPM) and tokens per minute (TPM). Exceeding it returns 429.
The important distinction: a rate limit is not a spend limit. One request to an expensive model with a large context can cost more than a thousand cheap ones. Throughput and cost are only loosely related.
Rate limits are also shared across the organisation, which is why unexplained 429s on your own traffic can be a signal that someone else is using your key.
For bounding cost, use a spend limit.