Open-source gateway features

Reference for the open-source DVARA LLM Gateway, run from one gateway.yaml. The quickstart gets it running.

Governance, on every request and every response

• Policy as code — rules per workspace that deny a request or warn the calling agent, matched on the model, the requested max_tokens and the tools the request names.
• PII detection and redaction — finds emails, card numbers, national identifiers and more, and checks the checksum where the identifier has one, such as Luhn for cards and Verhoeff for Aadhaar. Block the request, redact the value, or log it; logging is the default. Pattern-based; named-entity recognition is not included in this build.
• Guardrails — prompt-injection defence and content filtering, on the request and the response. On by default; each workspace chooses whether a detection logs, flags or blocks.
• Rate limiting — requests and tokens per minute, per API key, counted inside the process. Off until you enable it. A request's tokens are counted once, from what it sends the model (text, tool calls, tool definitions, images), not from its JSON. The rate limit, the guardrail's input-token cap and the context-window check all use that same count.
• Tamper-evident audit — every decision written to an HMAC-chained log on local disk, with an offline verifier that reports what it checked as well as whether it passed.

Routing and reliability

• One API, 14 providers — OpenAI, Anthropic, Gemini, Bedrock, Azure OpenAI, Mistral, Cohere, Groq, Qwen, DeepSeek, Moonshot, ChatGLM, Grok and Ollama. See Supported providers.
• Routing strategies — by model prefix, round-robin, weighted split or canary, with failover.
• Retries, circuit breakers and timeouts — on by default, adjustable per provider, with failover along a route's own chain of backup providers and models. See Failover to another provider.
• Streaming, including streamed tool calls where the provider relays them.

Operations

• One file — providers, workspaces, routes, keys, policies, rate limits and output schemas in a gateway.yaml, with ${VAR} and ${VAR:-default} resolved from the environment so no secret is written to disk.
• Keys as fingerprints — the file holds a SHA-256 of each API key, never the key, so it is safe to commit.
• Prometheus metrics and OpenTelemetry tracing — /actuator/prometheus behind a metrics key, and OTLP trace export when you name a collector. See Observability.
• Embeddable — the engines and the /v1 runtime are Spring Boot starters for your own application. See Use it as a library.

The Anthropic Messages, OpenAI Responses and Gemini generateContent APIs, Batch and the response cache are described in APIs and providers.

OWASP Top 10 for LLM Applications

[!IMPORTANT] OWASP Top 10 for LLM Applications. The open-source gateway addresses these on every request and response, with no additional service:

	Risk	Open source	How
LLM01	Prompt injection	✅	direct, indirect and jailbreak pattern detection on the request
LLM02	Sensitive information disclosure	✅	PII detection and redaction on requests and responses: emails, cards, SSN, IBAN, passports, Aadhaar, PAN, medical identifiers and more
LLM03	Supply chain	◐	model allowlists in policy pin the approved models
LLM04	Data and model poisoning	—	a training-time risk, outside a gateway
LLM05	Improper output handling	✅	content filtering on the response and JSON schema validation of the output per route
LLM06	Excessive agency	◐	tool allowlists and denylists in policy; MCP and agent-to-agent governance are not in this build
LLM07	System prompt leakage	✅	extraction attempts caught on the request, leaked system prompt content caught on the response
LLM08	Vector and embedding weaknesses	—	lives inside the RAG pipeline, outside a gateway
LLM09	Misinformation	—	grounding detection is not in this build
LLM10	Unbounded consumption	✅	context-window limits, provider response size limits, per-provider timeouts and circuit breakers by default; per-key request and token rate limits and input token caps when enabled

✅ covered · ◐ partly covered · — not covered. The list is OWASP's 2025 edition.

What is in the open-source build

This is the complete LLM Gateway for a team running one gateway in front of its models. Everything that enforces — policy, PII detection and redaction, guardrails, rate limits, and the tamper-evident audit chain — is here and is Apache-2.0. Nothing that stops an attack depends on a licence.

Not published here: the MCP and A2A Gateways, and Flightdeck, the console that runs a fleet — with database-based configuration, SSO, fleet-wide audit with SIEM export, and cost attribution. Those are commercial components; see pricing for what they cover.