Batch API
Last verified
A Batch API accepts a set of requests, processes them asynchronously within a stated window, and charges a discounted rate — commonly around 50% of standard pricing.
Right for anything not user-facing: bulk classification, embedding a corpus, evaluation runs, dataset generation, backfills.
Wrong for anything a user is waiting on.
The discount is large enough that it should be the default question for any bulk job. Teams routinely pay full price for overnight work that had no latency requirement at all.