LiteLLM

LiteLLM is an open source AI gateway and LLM proxy that connects applications to models, agents and MCP servers through a single OpenAI compatible API. It supports more than 100 providers, so developers can swap models without changing application code, and it adds day zero support for newly released models. Platform teams use it to put an organization's whole AI stack behind one key. Virtual keys can be scoped by team, project or app, with single sign on, spend tracking, budgets and rate limits to cap cost before requests run. It also provides model routing and fallbacks, request and response logging, Prometheus metrics, and governance controls. Internal, fine tuned and self hosted models can sit behind the same gateway. It can be installed in the cloud, on premises or in an air gapped environment, and a Python SDK is also available. The open source gateway is free to self host. LiteLLM Enterprise adds governance, security and support, including support with service level agreements up to 24 hours a day. Enterprise pricing is sized to the customer's annual gateway request capacity, deployment architecture and support needs, never per token, and the vendor does not publish a price. Buyers contact sales for a quote. Enterprise is built on the same code as the open source gateway.