- LLMFlux
- AI API gateway
AI API gateway
An AI API gateway for every model you ship with
LLMFlux gives your application one API key and one endpoint for a catalogue of AI models from Anthropic, DeepSeek, Google, xAI, Qwen and others. The gateway handles authentication, provider failover, streaming and usage records, so your code only chooses the model.
What the gateway does
Calling several AI providers directly means several SDKs, several keys, several billing accounts and several sets of error codes. A gateway collapses that into one integration. With LLMFlux every request goes to api.llmflux.dev; the gateway checks your key and credit balance, sends the request to a provider that serves the model you asked for, and passes the response back unchanged in the format you used.
One key for the catalogue
A single LLMFlux API key works for every model listed by GET /v1/models. Switching models is a change to the model field, not a new account.
Standard API formats
OpenAI Chat Completions, OpenAI Responses and Anthropic Messages are all accepted, so existing SDKs and tools work after a base-URL change.
Provider failover
On network errors, 429s and 5xx responses the gateway retries another provider for the same model and pauses the failing one for a 60-second cooldown.
Streaming
Server-sent events are passed through as they arrive, for chat, responses and messages.
Usage and cost per request
Each request is logged with model, tokens, status and cost. The dashboard shows daily usage; GET /v1/usage returns it from the API.
Key controls and limits
Create, name and revoke keys in the dashboard. Rate limits protect your account and the shared capacity.
How a request flows through LLMFlux
- Your app sends a request to
https://api.llmflux.dev/v1/chat/completions(or/v1/responses,/v1/messages) with your LLMFlux key. - The gateway validates the key, the model id and your credit balance, and applies rate limits.
- It forwards the request to a provider that serves that model. If the provider fails with a retryable error, it moves to the next one.
- The response streams back in the same API format you used, and the request is recorded with its token count and cost.
Provider credentials and routing configuration stay on the server. Your application only ever holds an LLMFlux key, and provider error text is not passed through; failures are mapped to documented gateway errors.
A first request
Any HTTP client works. This is the OpenAI-compatible Chat Completions endpoint:
curl https://api.llmflux.dev/v1/chat/completions \
-H "Authorization: Bearer $LLMFLUX_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "claude-sonnet-5-5",
"messages": [{ "role": "user", "content": "Explain what an API gateway does in one sentence." }]
}'The same call with the official OpenAI Python SDK — only base_url and api_key differ from a direct OpenAI integration:
from openai import OpenAI
client = OpenAI(
base_url="https://api.llmflux.dev/v1",
api_key="YOUR_LLMFLUX_API_KEY",
)
resp = client.chat.completions.create(
model="claude-sonnet-5-5",
messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)When a gateway is the right choice
You use more than one model
Compare models, use a fast model for simple tasks and a stronger one for hard ones, or migrate between model versions without new integrations.
You want one bill and one log
Prepaid credits and one request log replace a separate account, invoice and dashboard per provider.
You build developer tools
Coding agents and editors that speak the OpenAI or Anthropic format can point at LLMFlux with a base-URL setting.
You need predictable behaviour
You name the exact model in each request; the gateway does not silently swap it for a different model.
See the model catalogue for the current list and prices, or read how model routing works.
Documentation
Frequently asked questions
- What is an AI API gateway?
- An AI API gateway is a service that sits between your application and the AI model providers. Your app sends every request to one endpoint with one key; the gateway authenticates it, forwards it to a provider that serves the requested model, streams the answer back, and records usage and cost.
- Is an AI API gateway the same as an LLM gateway?
- Yes. The terms are used interchangeably. LLMFlux handles text and vision models through the OpenAI Chat Completions, OpenAI Responses and Anthropic Messages formats, plus image generation.
- Do I need to change my code to use LLMFlux?
- Usually only the base URL and the API key. Existing OpenAI or Anthropic SDK code keeps working; you pick a model id from the LLMFlux catalogue.
- What happens if an upstream provider fails?
- If a provider returns a network error, a rate limit (429) or a server error, the gateway tries the next provider configured for that model. A failing provider is paused for a short cooldown. You receive a standard gateway error only if no provider can serve the request.
- How is usage billed?
- LLMFlux uses prepaid credits. Each request is charged for the tokens it used at the per-model prices published in the model catalogue, and every request appears in your dashboard logs.
Related pages
Start with one key
Create an account, generate an API key, and send your first request in a few minutes.