LiteLLM

Founded 2023

An open-source LLM gateway and Python SDK that standardizes access to 100+ models using the OpenAI API format.

Cost

Free

Rating

★ People love it

Time to value

Quick Setup (< 1 day to basic use)

You can use LiteLLM to manage multiple LLM providers—like OpenAI, Anthropic, Vertex AI, Bedrock, Hugging Face, and local models—through one unified interface. It handles cost tracking, model fallbacks, logging, and rate limits automatically. Perfect for platform teams or developers wanting consistent and efficient LLM access across cloud or self-hosted setups.

What LiteLLM does

Call any of 100+ LLMs using OpenAI-style APIAutomatically fall back to backup models if one failsTrack usage and spending per user, team, or projectEnforce budgets and rate limitsLog input/output to tools like Langfuse, Helicone, MLflowDeploy unified LLM access via proxy or Python SDKUnified interface across more than 100 LLM providersOpenAI-style SDK and REST proxy for seamless switchingIntegrated fallback and retry logic across modelsBuilt-in spend tracking, budgets, and rate limitingPlug-and-play logging to observability toolsOpen-source with enterprise-grade hosting and support