One API in front of every model — governed and observable.
DVARA's LLM Gateway puts a single OpenAI-compatible endpoint in front of every major provider. Route by latency or cost, fail over automatically, cap spend, cache semantically, and see every token — with policy, PII, and audit on the same path.
The LLM Gateway ships in every DVARA install — see pricing.
Integrating providers one by one doesn't scale.
Wire your app straight to each provider and you own a different SDK for every one, hand-rolled retry and failover, no shared view of cost or tokens, and governance scattered across services. An LLM gateway consolidates all of it behind one endpoint every request already passes through — so routing, spend control, and governance become properties of the platform, not code in each app.
One endpoint. Every provider. Full control.
Point any OpenAI SDK at DVARA and reach every provider through the same /v1 chat, embeddings, and models shape. Switch models by changing a string.
OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Mistral, Cohere, Groq, and more — plus local models via Ollama — behind one endpoint.
Route by model prefix, round-robin, weighted, or latency- and cost-aware — pick the fastest or cheapest capable provider on every call.
When a provider errors or a circuit opens, DVARA retries on a capable alternative automatically — no client-side retry logic, no capability mismatches.
Per-tenant and per-key budgets, real-time cost attribution, and an optional per-call ceiling — enforced before the request ever goes upstream.
Serve semantically-equivalent prompts from cache and book $0 for the hit — no upstream call, lower latency.
Prometheus metrics and OpenTelemetry traces with GenAI semantic conventions on every call — tokens, latency, cost, provider, and model.
Policy-as-code, PII scanning, guardrails, and tamper-evident audit run inline — not a separate proxy you have to bolt on.
Run it in your own environment with your own provider keys. Prompts and credentials never leave your perimeter.
Point, route, govern, observe
- 1
Point your SDK
Change the base URL to your DVARA gateway and keep using the OpenAI SDK you already have. No rewrite.
- 2
Route
DVARA selects a provider by model prefix or your routing strategy — latency-aware, cost-aware, or weighted.
- 3
Govern & protect
Every call runs policy, guardrails, and PII checks, then fails over to a capable alternative or serves from cache as needed.
- 4
Meter & observe
Tokens and cost are attributed per tenant and key; metrics and traces flow to your existing stack.
See every provider, route, and dollar in DVARA Flightdeck
Common questions about LLM gateways
An LLM gateway is a single API layer in front of multiple LLM providers that handles routing, fallback, caching, cost control, observability, and governance — so your applications call one endpoint instead of integrating each provider directly.
An API gateway sees the HTTP envelope — TLS, paths, rate limits. An LLM gateway understands the AI semantics inside the request: models, tokens, tools, and prompts. That lets it route, budget, cache, and govern on attributes an API gateway can’t see.
Yes. DVARA exposes the OpenAI /v1 API shape (chat completions, embeddings, models), so existing SDKs and tools work by changing only the base URL.
The routing and governance overhead is small and runs on virtual threads. Semantic caching often makes net latency lower, since a cache hit skips the upstream provider call entirely.
Yes. DVARA runs in your own environment with your own provider credentials (BYOK). Prompts and keys never leave your perimeter — the foundation for data residency and sovereignty requirements.
OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Mistral, Cohere, and Groq, among others, plus local models via Ollama — all through the same endpoint.
Guides on running an LLM gateway
Why API gateways like Kong and NGINX can’t govern LLM and agent traffic — and the six gaps a purpose-built AI gateway closes.
Beyond the basics — streaming responses, JSON-schema-enforced structured outputs, and per-call token/cost/latency telemetry.
Point the OpenAI SDK you already have at DVARA and call every provider through one endpoint — a five-minute switch.
One governed API in front of every model — model-prefix, weighted, and capability-aware routing, with policy, cost, and PII applied on the same path.
Governed resilience: circuit breakers and capability-aware failover that keep AI traffic serving — and audited — when a provider degrades.
Attribution and enforcement, not just dashboards — real-time cost attribution, hard budget caps, model downgrade, and chargeback.
Governance that runs inside your perimeter — policy, PII, and audit before egress, with BYOK and vault-backed credentials.
Serve semantically similar prompts for $0, PII-safe by default — the cost-and-latency win that stays a governance-safe surface.
The telemetry layer under governance — Prometheus metrics, OpenTelemetry traces with the GenAI semantic conventions, and per-request structured logs.
A governance-first comparison framework — the categories on the market and the criteria that matter, weighted toward enforcement over dashboards.
One endpoint in front of every model.
The LLM Gateway ships in every DVARA install. Start a free 30-day trial, or see how it routes, governs, and meters your traffic.