Llmgateway

LLM Gateway is an open source API gateway that routes requests to more than 250 models from over 40 providers, including OpenAI, Anthropic and Google, through one OpenAI compatible endpoint. Teams keep their existing OpenAI SDK code, change the base URL and key, and can switch providers without touching application logic. The gateway handles smart routing by price, latency, throughput or uptime, automatic fallback to a healthy provider, and Redis backed response caching. It tracks cost and usage per project, with analytics for requests, tokens, spend, error rate and cache hit rate, broken down by provider and model. Project scoped API keys carry spend limits and can be rotated without code changes. Guardrails cover prompt injection, personal data detection and redaction, and secrets detection. Endpoints cover chat completions, embeddings, images, video, speech and transcription. Enterprise features include SSO and SAML with SCIM, audit logs, per project routing overrides, compliance based provider policies and a white label chat app. The company states it is SOC 2 Type II compliant, and the whole stack can be self hosted with Docker or used as a hosted cloud service. It is built for developers and companies running production LLM features who want one integration, one bill and one view of cost. The Free plan costs $0 and allows bring your own provider keys at no charge. Credits used through the hosted service carry a 5 percent platform fee, with volume discounts. Non US cards add a 1.5 percent international fee. Three models are free with rate limits of 20 requests per minute. Enterprise pricing is custom, with a 30 day trial and full payload retention billed at $0.01 per million tokens.