One API for 40+ LLM providers

Ship AI features on every model, without rewriting your stack

LLM Gateway puts one OpenAI-compatible endpoint in front of 40+ providers. Route by price or latency, fall back automatically when a provider degrades, and see exactly what every request cost.

See the integration
One API key, 40+ providersOpenAI-compatible endpointsAutomatic failover between models
40+providers behind one endpoint
3 linesto switch the whole app
<80msgateway overhead at p50
99.99%routing availability target
playgroundlocked · sign in to route a requestLocked
Playground

Route one request across 40+ providers

Pick a provider and model, set the sampling parameters and see the routed request, the fallback order and the estimated cost before it leaves your browser.

$0.00519estimated · 27 in / 512 out
  • Falls back to Anthropic if OpenAI rate limits
  • Cost recorded per key, per request
  • Streaming enabled · temp 0.2
Responseno request yet

Run the sample prompt to see the routed answer, the provider that served it and the exact cost of the call.

ProviderOpenAI
ModelGPT-4o
Routestream + fallback
Cost$0.00519

Sandboxed preview: the gateway decisions are simulated in your browser and no provider call is made.

Playground is locked

Sign in with Google and open the playground with an active plan. Accounts without a subscription are sent to pricing.

How it works

From one request to a routed, priced answer

The gateway is a thin, opinionated layer: your request goes in, a policy picks the provider, failures are absorbed, and the result carries its own cost.

01Point

Send one OpenAI-shaped request

Keep the SDK you already use and change the base URL. Chat completions, streaming, tool calls and embeddings all arrive in the same shape.

02Route

Pick a model or a policy

Name a model directly, or route by price ceiling, latency target and context length, and let the gateway pick the cheapest provider that satisfies it.

03Fall back

Survive provider incidents

When a provider errors, rate limits or times out, the request is retried on the next healthy provider with the same body and streamed back to the caller.

04Measure

Attribute every token

Every call is logged with provider, model, latency, token counts and cost, so spend can be broken down by key, project, team or feature.

Platform capabilities

Everything a team needs around the model call

Routing is the start. The gateway also carries the keys, the policies, the traces and the cost records that production traffic needs.

Unified routing layer

One endpoint for every provider, with request normalisation, tool calling, JSON mode and streaming preserved across vendors.

Model catalog

Browse 40+ providers and their models with live context windows, modalities and per-million-token prices before you ship.

Reliability policies

Retries, timeouts, hedged requests, circuit breakers and provider fallbacks configured per route instead of per client.

Cost and usage analytics

Cost per feature, per customer and per key, plus budget alerts and hard limits that stop a runaway loop before the invoice does.

Key and team management

Issue scoped keys per environment, rotate provider credentials centrally and keep vendor secrets out of your application code.

Observability

Traces for every request with provider latency, retry history, token breakdown and the policy that produced the route.

Providers and models

40+ providers, one catalog, live prices

A snapshot of the catalog behind the gateway. Prices are per million tokens and move with the vendor, so routing policies stay honest.

OpenAI3 models
  • GPT-4o$2.50 / $10.00
  • GPT-4o mini$0.15 / $0.60
  • o3-mini$1.10 / $4.40
Anthropic3 models
  • Claude Sonnet 4.5$3.00 / $15.00
  • Claude Haiku 4.5$1.00 / $5.00
  • Claude Opus 4.1$15.00 / $75.00
Google2 models
  • Gemini 2.5 Pro$1.25 / $10.00
  • Gemini 2.5 Flash$0.30 / $2.50
Meta Llama2 models
  • Llama 4 Maverick$0.27 / $0.85
  • Llama 4 Scout$0.11 / $0.34
DeepSeek2 models
  • DeepSeek V3.2$0.28 / $0.42
  • DeepSeek R1$0.55 / $2.19
Mistral2 models
  • Mistral Large 2$2.00 / $6.00
  • Codestral$0.30 / $0.90
xAI2 models
  • Grok 4$3.00 / $15.00
  • Grok 4 Fast$0.20 / $0.50
Groq1 model
  • Llama 4 on LPU$0.11 / $0.34
Cohere2 models
  • Command A$2.50 / $10.00
  • Command R+$2.50 / $10.00
Perplexity2 models
  • Sonar Pro$3.00 / $15.00
  • Sonar$1.00 / $1.00
Together1 model
  • Qwen3 235B$0.20 / $0.60
Fireworks1 model
  • Kimi K2$0.60 / $2.50

Plus custom OpenAI-compatible endpoints, Azure deployments, Bedrock and self-hosted inference behind the same key.

Drop-in integration

Change the base URL, keep the rest

The gateway speaks the OpenAI API, so the migration is measured in lines, not sprints.

  • Drop-in OpenAI compatibility, so existing SDKs and agent frameworks keep working.
  • Model strings can name one provider or describe a routing policy.
  • Streaming, tool calls, structured outputs and vision pass through unchanged.
curl https://api.llmgateway.lol/v1/chat/completions \
  -H "Authorization: Bearer $LLM_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "smart/cheapest",
    "messages": [{ "role": "user", "content": "Summarise this diff" }],
    "stream": true
  }'
Reliability and failover

Provider incidents stop being your incidents

Failover is a policy, not an if-statement in every service. The gateway retries, hedges, breaks circuits and probes providers back into rotation.

Provider fallbacks

Ordered fallback chains per route: if the primary provider fails or rate limits, the next healthy provider answers the same request.

Timeouts and retries

Per-route timeouts with jittered retries, plus hedging for latency-sensitive traffic that cannot wait for a full timeout.

Circuit breakers

Failing providers are removed from rotation automatically and probed back in, so one vendor incident does not become yours.

Stream-safe recovery

Recovery happens before the first token is flushed, so clients never receive a half-written answer from a dead provider.

Cost and usage analytics

Every token attributed, every budget enforced

Spend is recorded per request and can be grouped by key, project, customer or feature — with limits that act before the invoice arrives.

$4,182Spend this month42% cheaper than a single-provider baseline
18.4MTokens routedInput and output tokens attributed per key
99.97%Successful requestsAfter automatic provider fallbacks
Support assistant$1,58938%
Document summarisation$1,21329%
Code review agent$87821%
Internal search$50212%

Example report for a product running support, summarisation, code review and search traffic through one gateway.

Pricing

Plans that scale with routed traffic

Monthly or annual billing through Stripe. Every plan includes the full provider catalog, routing policies and per-request cost records.

Developer

$19/ month

$190 a year, two months free

For a first product behind one gateway.

  • 2 million routed tokens a month
  • Every supported provider and model
  • Fallback chains and retry policies
  • Usage analytics by key
  • Community support
Choose Developer

Pro

Most popular

$49/ month

$490 a year, two months free

For teams shipping to production.

  • 25 million routed tokens a month
  • Cost ceilings and routing policies
  • Budget alerts and hard limits
  • Team keys, roles and audit history
  • Email support within one business day
Choose Pro

Scale

$199/ month

$1,990 a year, two months free

For platform teams with real traffic.

  • 150 million routed tokens a month
  • Custom routing, hedging and failover
  • Per-customer cost attribution and exports
  • SSO, audit logs and private networking
  • Priority support and architecture review
Choose Scale

Annual plans are billed once a year and include two months free compared with monthly billing. Cancel at any time from billing settings.

FAQ

Questions teams ask before switching

Short answers about the gateway, the providers and how billing works.

What is an LLM API gateway?

An LLM API gateway sits between your application and every model provider. You call one OpenAI-compatible endpoint and the gateway handles provider credentials, request normalisation, routing, retries and failover, then returns one usage and cost record for every call.

How many providers does LLM Gateway support?

More than 40 providers and several hundred models, including OpenAI, Anthropic, Google, Meta Llama, Mistral, DeepSeek, xAI, Groq, Cohere, Perplexity, Together and Fireworks. The catalog follows vendor releases, and any OpenAI-compatible endpoint can be added as a custom provider.

Do I have to rewrite my application?

No. The API is OpenAI compatible, so in most stacks the only change is the base URL and the API key. Streaming, tool calls, structured outputs, vision inputs and embeddings keep working, and switching models is a single string change.

How does failover work when a provider goes down?

Each route has an ordered provider list plus timeout, retry and circuit-breaker settings. If the primary provider errors, rate limits or times out before the first token, the request is retried on the next healthy provider with the same body, and failing providers are probed back into rotation automatically.

Can I see and control what I spend?

Yes. Every request records provider, model, token counts, latency and cost. Spend can be broken down by key, project, team or feature, with budget alerts and hard limits that stop traffic before it exceeds the ceiling you set.

Do I need a subscription to use the playground?

The playground on the home page is a gated preview that shows routing, fallback and cost behaviour in your browser. It needs a Google sign-in and an active plan; accounts without a plan are sent to the pricing page.

Route your first request in minutes

Sign in with Google, open the playground and price a request against the whole catalog before you change a line of production code.

LLM Gateway - One API for 40+ LLM Providers