LiteLLM: the free gateway that puts 100+ LLM APIs behind one endpoint

LiteLLM is an MIT-licensed proxy you run yourself. It gives every model provider in your stack one OpenAI-compatible endpoint, one key format, per-team budgets and automatic fallbacks when a provider fails. The alternative it replaces is the hosted router you would otherwise pay a platform fee to use.

How it works

Your application talks to LiteLLM as if it were OpenAI. LiteLLM reads a YAML file that maps model names to providers and keys, then forwards each request to the right place and returns the response in the same shape. Virtual keys let you issue a different credential to each app or team, with its own budget and model access list. Spend is recorded per key, which is how you find out which team is burning the budget. Fallback chains mean a failed provider retries somewhere else instead of returning an error to the user.

What you need

• Python 3 or Docker. The current SDK is 1.108.2 on PyPI, checked in November 2026.
• At least one provider key, and a config file listing your models.
• Optional Postgres for virtual keys and spend tracking, and optional Redis if you run more than one instance and want shared rate limits.
• A front end such as Open WebUI if humans are meant to chat. LiteLLM is plumbing, not a chat window.

What it does well

• One integration instead of one per provider. Applications keep speaking the OpenAI format and never learn which vendor answered.
• Cost control that is easy to explain to a finance team: keys, budgets, per-model pricing and a spend report.
• Fallbacks that work at the model level, so a rate limit on one provider is a routing decision rather than an outage.
• Local and hosted models side by side: point one alias at a hosted API and another at Ollama or vLLM on the same box.

Where it falls short

• It is a hop you now own. Your application's uptime includes the gateway's uptime.
• The enterprise/ directory is separate code under a commercial licence. Keys, budgets and routing are in the open core; single sign-on, audit logs and some UI features are not.
• The config file grows quickly once model aliases, per-team limits and fallback chains stack up.
• Debugging a bad response means checking the gateway logs as well as the provider's, which is one more place to look.

How to install LiteLLM

Option 1, one-click install: Hostinger deploys LiteLLM as a preconfigured Docker app on a managed server, so there is no command line and nothing to configure. It runs the gateway with cost tracking enabled.

Option 2, install it yourself: install the proxy with pip install 'litellm[proxy]', write a config.yaml listing your models and keys, then start it with litellm --config config.yaml. The SDK alone can be installed with pip install litellm if you only want the Python client.

FAQ

Is LiteLLM free? The open-source gateway is MIT licensed and free to self-host. The enterprise/ folder is separate code under its own commercial terms.

Do I need Postgres and Redis? Not for a config-only proxy. Virtual keys and spend tracking want a database, and shared rate limits across replicas want Redis. A single instance for a small team runs without either.

Does it add latency? One network hop between your app and the provider. Usually milliseconds against model responses measured in seconds, but it is a hop to monitor.

Can it serve local models as well as APIs? Yes. That is one of its better uses: mixed aliases across hosted providers and a local endpoint, chosen per request.

What is the difference between the SDK and the proxy? The SDK is a Python library that calls many providers through one function. The proxy is the server, adding keys, budgets, logging and a shared endpoint for other applications.

How do I keep spend under control? Set a budget per virtual key, not per model, and let the spend report show which key is consuming it. A single team-wide budget tells you nothing until it is already gone.

Last checked: November 2026. Licence read from the repository's LICENSE file, including the enterprise carve-out, and the version from PyPI.