LLM gateway
Last verified
An LLM gateway is a proxy that also decides _where_ a request goes: falling back to a second provider on failure, load balancing across models, or normalising many providers behind one API shape.
Useful when you genuinely run multi-provider. Overkill when you call one or two providers directly, because the routing is the product and you would be paying for the part you do not use.
Same architectural tradeoff as any proxy: it is in your request path.