Version: 1.8.0

Connect model providers

DVARA Open Source and the Enterprise platform use the same 14 provider integrations. Open Source reads credentials and routes from gateway.yaml; the Enterprise platform adds Flightdeck-managed credentials and routing.

DVARA ships with fourteen first-class providers plus a built-in Mock provider for CI and local development: OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Mistral, Cohere, Groq, Ollama, Alibaba Qwen, DeepSeek, Moonshot (Kimi), Zhipu ChatGLM, and xAI Grok. It can also send Chat Completions traffic to another OpenAI-compatible endpoint, such as Fireworks AI, Together AI, Perplexity, or an internal gateway. See Additional OpenAI-compatible providers for that setup and its capability boundary.

Each first-class provider becomes available at startup when its required setting is present:

Credential-gated (most providers) — registers when its *_API_KEY env var is non-blank (OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY, MISTRAL_API_KEY, COHERE_API_KEY, GROQ_API_KEY, QWEN_API_KEY, DEEPSEEK_API_KEY, MOONSHOT_API_KEY, ZHIPU_API_KEY, XAI_API_KEY). Azure additionally requires AZURE_OPENAI_BASE_URL.
Flag-gated — registers when its *_ENABLED boolean is true: BEDROCK_ENABLED, OLLAMA_ENABLED, MOCK_PROVIDER_ENABLED. Bedrock registers on the flag alone, but every request still needs an explicit access-key and secret-key pair. Ollama is unauthenticated and Mock makes no upstream calls, so neither needs a credential.

If a provider is unavailable, requests that target it return HTTP 400 with no_provider instead of failing silently.

Environment variables configure a single shared credential

Setting OPENAI_API_KEY (or any provider key) via an environment variable registers one platform-wide credential that all workspaces share. This is fine for single-workspace deployments and local development.

In the Open Source distribution, put workspace-specific provider credentials in gateway.yaml using environment placeholders. In the Enterprise platform, create them in DVARA Flightdeck under Credentials. Per-workspace credentials take precedence over the platform environment value. See Credentials & BYOK.

For the same live model-call operation, configured policy and data controls run independently of which provider is selected. Provider capabilities and endpoint contracts still differ: embeddings, batches, vision, function calling, structured output, and streaming do not have one uniform provider path. See how a synchronous model call moves through DVARA and the matrix below before building a route.

This page documents how to enable each provider, what features it supports, and the provider-specific quirks that affect what you can send through it. Credentials can come from environment variables, the DVARA Flightdeck (BYOK per workspace), or a vault — see Credentials & BYOK for the full picture.

All supported providers

Provider	Model prefix	Example models	Notes
OpenAI	`gpt`, `o1`, `o3`, `o4`, `chatgpt`, `text-embedding`	`gpt-4o`, `gpt-4.1`, `o3-mini`, `chatgpt-4o-latest`, `text-embedding-3-small`	Native structured outputs and JSON mode; reasoning + alias model families
Azure OpenAI	`azure/`	`azure/gpt-4o`	Requires both `AZURE_OPENAI_API_KEY` and `AZURE_OPENAI_BASE_URL`
Anthropic	`claude`	`claude-sonnet-4-5`, `claude-3-5-haiku-20241022`	`json_schema` via tool-use rewrite; 200K context
Google Gemini	`gemini`	`gemini-2.0-flash`, `gemini-1.5-pro`	1M token context window; API key in URI
AWS Bedrock	`bedrock/`	`bedrock/anthropic.claude-3-sonnet-…`	SigV4 signing with an explicit access-key and secret-key pair
Mistral	`mistral`	`mistral-large-latest`, `mistral-small-latest`	OpenAI-compatible API; no vision
Cohere	`command`	`command-r-plus`, `command-r`	No structured output or JSON mode
Groq	`groq/`	`groq/llama-3.3-70b-versatile`	Ultra-low latency; no `json_schema`
Ollama	`ollama/`	`ollama/llama3.2`	Local inference; from 1.8.4 also vision, JSON mode, structured outputs, and embeddings
Qwen	`qwen`	`qwen2.5-72b-instruct`, `qwen-max`	Alibaba DashScope; conservative v1 capability declaration
DeepSeek	`deepseek`	`deepseek-chat`, `deepseek-reasoner`	Streaming + tools + JSON mode; `json_schema` declared off (reasoning models differ)
Moonshot	`moonshot`	`moonshot-v1-128k`, `moonshot-v1-32k`	Declares a 200K context window
ChatGLM	`glm`	`glm-4`, `glm-4-air`, `glm-4v-plus`	Vision is model-specific (`glm-4v-*` only); declared off at the provider level
Grok	`grok`	`grok-2-1212`, `grok-2-vision-1212`, `grok-3-latest`	Native structured outputs; vision over-declared at provider level (model-specific)
Mock	`mock/`	`mock/test`	Dev and CI only — no upstream calls
Any other OpenAI-compatible	custom prefix	Fireworks AI, Together AI, Perplexity, internal corporate gateways	Register a custom provider →

OpenAI

Model prefix: gpt for chat models, text-embedding for embeddings.

export OPENAI_API_KEY='<your-openai-api-key>'

Example models: gpt-4o, gpt-4o-mini, gpt-4.1, gpt-4.1-mini, o1-preview, o1-mini, o3-mini, o4-mini, chatgpt-4o-latest, text-embedding-3-small, text-embedding-3-large. OpenAI's chat catalogue spans three namespaces — classic GPT models (gpt-*), reasoning models (o1-*, o3-*, o4-*), and alias models (chatgpt-*); all three route to OpenAI under the same OPENAI_API_KEY. Embeddings route separately under the text-embedding-* prefix. Call GET /v1/models for the live list of models your deployment can see.

Features: Chat completions, embeddings, streaming, vision, tool calls, structured outputs (native json_schema and json_object).

Notes:

Credentials are read per request, so rotating your OpenAI key in the DVARA Flightdeck takes effect on the next request — no restart.
Chat vs embedding routing is by prefix: gpt* → chat, text-embedding* → embeddings.
In 1.8.5 the context window check uses 128,000 tokens for every OpenAI model, so large gpt-4.1 requests can be refused or trimmed below the model's real window. See the known issue.
response_format: json_schema and response_format: json_object pass through natively — no rewrite.

Azure OpenAI

Azure OpenAI is a separate provider from OpenAI because it uses deployment-based routing and a different auth header. Both api-key and base-url are required — the provider only registers when both are set.

export AZURE_OPENAI_API_KEY='<your-azure-openai-api-key>'
export AZURE_OPENAI_BASE_URL=https://my-resource.openai.azure.com/openai

Model prefix: azure/. The prefix is stripped before the call, so azure/gpt-4o routes to the deployment named gpt-4o on your Azure resource.

Features: Chat completions, streaming, vision, tool calls, structured outputs (native json_schema and json_object). 128K context.

Notes:

Azure uses the api-key header (not Authorization: Bearer). DVARA handles this automatically.
The base URL typically ends in /openai for the Microsoft-hosted path. Example: https://acme.openai.azure.com/openai.
For workspace-scoped BYOK credentials, the platform-wide DVARA_ENCRYPTION_MASTER_PASSWORD env var is required when any credential is stored in ENCRYPTED mode (the default) — it protects every encrypted credential, not just Azure. Zero-trust deployments that only ever create credentials in REFERENCE mode (vault-pointer-only, no secret material at rest) can omit it. See Credentials & BYOK.

Anthropic

Model prefix: claude

export ANTHROPIC_API_KEY='<your-anthropic-api-key>'

Example models: claude-sonnet-4-5, claude-3-5-haiku-20241022, claude-3-opus-20240229. Any model starting with claude is routed to Anthropic.

Features: Chat completions, streaming, vision, tool calls, structured outputs (emulated — see notes).

Notes:

system role messages are automatically extracted and passed as Anthropic's separate system field.
If you do not set max_tokens, DVARA defaults it to 1024 — Anthropic requires the field.
response_format: json_object is implemented by injecting a system prompt instruction. The response is still standard text; you parse it as JSON.
response_format: json_schema is implemented by rewriting the call into Anthropic's tool-use API with a synthetic structured_output tool. The tool's input is extracted and returned as the assistant message. This works transparently to the caller — you send OpenAI-format json_schema, you get OpenAI-format output.
If your client sends json_schema with strict: true, the response includes an X-Gateway-Strict-Downgraded: true header — Anthropic cannot natively enforce strict JSON Schema, and DVARA flags that for observability.
From 1.8.5 you can set Anthropic's base URL, as for other providers: dvara.llm-gateway.providers.anthropic.base-url, or base_url on an anthropic provider in gateway.yaml. Use it for a corporate egress endpoint or a test stub.
From 1.8.5 a client built for Anthropic, such as Claude Code, can call POST /v1/messages. On a route to Anthropic the request goes on as the client sent it, extended thinking included. See Govern Claude Code.
In 1.8.5 the context window check uses 200,000 tokens for every Claude model, so large Claude Sonnet 5.5 and Opus 5.5 requests can be refused or trimmed below the model's real window. See the known issue.

Google Gemini

Model prefix: gemini

export GEMINI_API_KEY='<your-gemini-api-key>'

Example models: gemini-2.0-flash, gemini-1.5-pro, gemini-1.5-flash.

Features: Chat completions, streaming, vision, tool calls, structured outputs (native responseMimeType and responseSchema). Declares a 1 million token context window.

Notes:

The Gemini REST API puts its API key in the URI query string instead of a header. DVARA resolves the key per request and appends it to the outgoing URL.
json_schema translates to generationConfig.responseMimeType: "application/json" plus generationConfig.responseSchema.
json_object translates to generationConfig.responseMimeType: "application/json" without a schema.
Because the context is 1M tokens, DVARA's input-size guardrails rarely trigger for Gemini with default settings. Tune guardrail.max-input-tokens per workspace if you want tighter limits for a specific workload.

AWS Bedrock

Model prefix: bedrock/. The prefix is stripped before sending the model ID to Bedrock, so any Bedrock-hosted model works.

export BEDROCK_ENABLED=true
export AWS_ACCESS_KEY_ID='<your-aws-access-key-id>'
export AWS_SECRET_ACCESS_KEY='<your-aws-secret-access-key>'
export AWS_REGION=us-east-1          # defaults to us-east-1 if unset

Example model IDs:

bedrock/anthropic.claude-3-sonnet-20240229-v1:0
bedrock/amazon.titan-text-express-v1
bedrock/meta.llama3-70b-instruct-v1:0
bedrock/mistral.mistral-7b-instruct-v0:2

Features: Chat completions, streaming, vision, tool calls, structured outputs (emulated via tool-use rewrite for Claude models hosted on Bedrock).

Authentication: AWS SigV4 request signing, computed per request. Credentials are resolved per request, so rotating them in the DVARA Flightdeck or in your vault takes effect on the next call.

Notes:

AWS credentials are required explicitly today — either as the AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY env vars (shown above), or as workspace-scoped credentials registered in the DVARA Flightdeck under Credentials. Setting BEDROCK_ENABLED=true and leaving the access / secret keys blank activates the provider at startup but every request fails with a credential error from the SigV4 signer. The AWS SDK's default credential provider chain (IRSA on EKS, EC2 instance metadata, container credentials) is not supported; an explicit key pair is mandatory on every deployment that uses Bedrock.
response_format: json_schema rewrites the call into Bedrock's toolConfig with a synthetic tool; the tool's input is extracted as the response text.
response_format: json_object injects a system message instructing the model to reply in JSON.
When the client sends json_schema with strict: true, the response includes X-Gateway-Strict-Downgraded: true — Bedrock does not natively enforce strict schemas.

Mistral

Model prefix: mistral

export MISTRAL_API_KEY='<your-mistral-api-key>'

Example models: mistral-large-latest, mistral-small-latest, mistral-medium-latest. Any model starting with mistral is routed here.

Features: Chat completions, streaming, tool calls, structured outputs (native json_schema and json_object). 128K context.

Notes:

Mistral's public API is OpenAI-compatible, so DVARA talks to it using the standard OpenAI request and response shapes. You do not need a different client.

Limitations: Pixtral vision models (pixtral-12b, pixtral-large) are not supported in this release. See Provider capability gaps for the full list and the safe routing workaround.

Cohere

Model prefix: command

export COHERE_API_KEY='<your-cohere-api-key>'

Example models: command-r-plus, command-r, command-light. Any model starting with command is routed here.

Features: Chat completions, streaming. 128K context.

Notes:

Cohere uses its v2 chat API at api.cohere.com/v2. DVARA translates OpenAI-style requests to Cohere's format and maps responses back — clients see standard OpenAI format.
Finish reason mapping: COMPLETE → stop, MAX_TOKENS → length.

Limitations: Vision, tool calls, and structured outputs are not supported in this release. See Provider capability gaps for the error codes you'll see and the routing workaround.

Groq

Model prefix: groq/. The prefix is stripped before sending the model name to Groq — groq/llama-3.3-70b-versatile sends llama-3.3-70b-versatile to the API.

export GROQ_API_KEY='<your-groq-api-key>'

Example models: groq/llama-3.3-70b-versatile, groq/mixtral-8x7b-32768, groq/gemma2-9b-it.

Features: Chat completions, streaming, JSON mode (json_object). 131K context. Groq is well known for ultra-low inference latency on supported models — it's a good choice for interactive UX where wall-clock matters more than cost.

Notes:

Groq's API is OpenAI-compatible. DVARA reuses the standard OpenAI request/response shapes.
On a mixed route, json_schema requests are automatically routed to a capable provider; json_object requests can still go to Groq.

Limitations: Vision, tool calls, and json_schema structured outputs are not supported in this release; json_object mode works. See Provider capability gaps for the full list and the routing workaround.

Ollama

Model prefix: ollama/. The prefix is stripped before sending the model name to the Ollama API.

export OLLAMA_ENABLED=true
export OLLAMA_BASE_URL=http://localhost:11434   # optional, defaults to localhost:11434

Prerequisites: an Ollama server running at OLLAMA_BASE_URL with at least one model pulled:

ollama serve
ollama pull llama3.2

Example request:

curl -s -X POST http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer $DVARA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "ollama/llama3.2", "messages": [{"role": "user", "content": "Hello!"}]}'

The response is an ordinary OpenAI chat completion — a local model reached through DVARA looks the same on the wire as a hosted one, which is the point of routing it here:

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "model": "ollama/llama3.2",
  "choices": [
    {"index": 0, "message": {"role": "assistant", "content": "Hello! How can I help?"}, "finish_reason": "stop"}
  ],
  "usage": {"prompt_tokens": 4, "completion_tokens": 7, "total_tokens": 11}
}

Ollama needs no credentials of its own, but the DVARA key is still required — it identifies the workspace, which is what every governance and metering decision keys on. Without it you get 401 api_key_required, and a key that is not in the store returns 401 invalid_api_key. If the model has not been pulled, the call fails at the provider rather than at the gateway.

Features: Chat completions, streaming, and tool calls. 32K context. From 1.8.4 the provider also handles:

Vision. DVARA sends each image to Ollama as a base64 data: URL. Ollama does not accept an https image URL, so DVARA fetches the image and inlines it when image fetching is on (dvara.llm-gateway.image-fetch.enabled, off by default). With fetching off, an https image URL is refused before the call with 400 unsupported_capability. The model must be a vision model; if it is not, Ollama refuses the call and DVARA answers with an error.
JSON mode and structured outputs. response_format json_object and json_schema reach Ollama as OpenAI's response_format, on plain and streamed calls. Ollama follows a JSON schema from version 0.5. For an older Ollama, set OLLAMA_STRUCTURED_OUTPUTS=false (dvara.llm-gateway.providers.ollama.structured-outputs, default true): the provider then stops declaring structured outputs, capability-aware routing skips it for json_schema traffic, and a json_schema request sent to it with an explicit ollama/ model is refused with 400. JSON mode stays on.
Embeddings. Every ollama/ model is offered on POST /v1/embeddings. Ollama decides which of its models can embed; a model that cannot is refused by Ollama, and DVARA answers 400 provider_rejected_request naming the model.
Streamed tool calls. A request with both tools and stream: true can now route to Ollama. The stream carries OpenAI-compatible delta.tool_calls[].

Before 1.8.4, images, response_format, embeddings, and streamed tool calls were refused before the call.

Notes:

Ollama's endpoints are unauthenticated by design. Do not expose them to the public internet — run the Ollama server on localhost, inside a private network, or behind a separate authenticated reverse proxy.
The provider only registers when OLLAMA_ENABLED=true, so Ollama won't show up in GET /v1/models for deployments that haven't opted in.

Limitations: Vision and structured outputs depend on the model and the Ollama version you run, not on DVARA. Ollama streams use estimated usage. There is no Batch API. See Provider capability gaps.

Alibaba Qwen

Model prefix: qwen

export QWEN_API_KEY='<your-dashscope-api-key>'

Example models: qwen2.5-72b-instruct, qwen2.5-coder-32b-instruct, qwen-max, qwen-plus. Any model starting with qwen routes to the DashScope compatible-mode endpoint at https://dashscope.aliyuncs.com/compatible-mode/v1.

Features: Chat completions, streaming. 32K context.

Notes:

Qwen's catalogue spans text, vision (Qwen-VL family), and reasoning variants. Capabilities are declared conservatively at the provider level — streaming on, everything else off. DVARA filters structured-output and tool-call requests from automatic routes, but vision is not a primary route filter. Route image traffic explicitly to a tested Qwen vision model.
The compatible-mode endpoint speaks the OpenAI wire format. The native DashScope API uses a different shape; DVARA doesn't talk to that one.

Limitations: Vision, tool calls, structured outputs, and JSON mode are conservatively-declared off in this release. See Provider capability gaps.

DeepSeek

Model prefix: deepseek

export DEEPSEEK_API_KEY='<your-deepseek-api-key>'

Example models: deepseek-chat, deepseek-reasoner. Any model starting with deepseek is routed here.

Features: Chat completions, streaming, tool calls, JSON mode. 64K context.

Notes:

DeepSeek's chat models support tools and JSON mode natively; structured outputs (json_schema enforcement) are declared off at the provider level because the reasoning model family (deepseek-reasoner) handles structured output differently from the chat family. Operators who need strict json_schema should route to OpenAI, Gemini, Mistral, or Azure OpenAI.
API base URL is https://api.deepseek.com/v1 — fully OpenAI wire-format compatible.

Limitations: Strict json_schema structured outputs and vision are not declared at the provider level. See Provider capability gaps.

Moonshot AI (Kimi)

Model prefix: moonshot

export MOONSHOT_API_KEY='<your-moonshot-api-key>'

Example models: moonshot-v1-8k, moonshot-v1-32k, moonshot-v1-128k. The brand is "Kimi" but the API uses the moonshot- prefix.

Features: Chat completions, streaming, tool calls, JSON mode. Declares a 200K context window.

Notes:

Moonshot supports long documents and conversation histories within its declared context window.
API base URL is https://api.moonshot.cn/v1.

Limitations: Vision and strict structured outputs not first-class. See Provider capability gaps.

Zhipu AI (ChatGLM)

Model prefix: glm

export ZHIPU_API_KEY='<your-zhipu-api-key>'

Example models: glm-4, glm-4-air, glm-4-flash, glm-4v-plus. The brand is "ChatGLM" but the API uses the glm- prefix.

Features: Chat completions, streaming, tool calls, JSON mode. 128K context.

Notes:

API base URL is https://open.bigmodel.cn/api/paas/v4.

Capabilities by model:

glm-4, glm-4-air, glm-4-flash — text only; chat + tools + streaming + JSON mode.
glm-4v-plus, glm-4v — vision-capable. The provider-level supportsVision is declared off in this release (conservative), so capability-aware routing won't auto-dispatch vision requests to ChatGLM. To use a glm-4v-* model for vision, configure an explicit route that targets it.

Limitations: Vision is conservatively declared off at the provider level. See Provider capability gaps.

Grok (xAI)

Model prefix: grok

export XAI_API_KEY='<your-xai-api-key>'

Example models: grok-2-1212, grok-2-vision-1212, grok-3-latest, grok-beta.

Features: Chat completions, streaming, vision (model-specific), tool calls, structured outputs (native), JSON mode. 131K context.

Notes:

API base URL is https://api.x.ai/v1. xAI's API is OpenAI-compatible at the wire format; DVARA reuses the standard request/response shapes.

Capabilities by model:

grok-2-1212, grok-3-latest, grok-beta — text only; chat + tools + streaming + structured outputs + JSON mode.
grok-2-vision-1212 — vision-capable. The provider family declares vision support even though most Grok models do not accept image input. A non-vision model can therefore return provider_error from the upstream rather than a pre-dispatch DVARA refusal. Route image traffic explicitly to a vision model.

Limitations: Vision is over-declared at the provider level — the per-model variance is the trade-off. See Provider capability gaps.

Mock Provider

The Mock provider returns configurable fake completions without calling any upstream API — ideal for integration tests, CI pipelines, load tests, and local development without real API keys. Model prefix is mock/.

Full setup, wiremock-style matchers, file-backed scenarios with hot reload, the DVARA Flightdeck authoring UI, Prometheus counters, and audit events are documented on the dedicated Mock Provider page.

Advanced: overriding the base URL

For the common case, you only need environment variables — the gateway's built-in configuration already maps each *_API_KEY env var to the right provider and points at the official upstream endpoint. You never write YAML for a standard OpenAI, Anthropic, Gemini, Mistral, Cohere, Groq, or Azure deployment.

The one time you do need YAML is when you want to override the upstream base URL, for example:

An OpenAI-compatible proxy (corporate LLM gateway, regional endpoint, OpenAI-protocol fork) that you want the OpenAI provider to talk to.
A Mistral / Groq fork hosted on your own infrastructure.
A non-default Ollama address (ollama-servers.internal:11434).

In those cases, create a tiny overrides.yml with just the provider base URL you want to change:

# overrides.yml
dvara:
  llm-gateway:
    providers:
      openai:
        base-url: https://llm.corp.internal/v1    # override — api-key still comes from env
      ollama:
        base-url: http://ollama-servers.internal:11434

Then tell the gateway to load it on top of the bundled config (not in place of it). The three recipes below show how to do that in each deployment topology. All three use the SPRING_CONFIG_ADDITIONAL_LOCATION environment variable so your override layers on the defaults — every other property (credentials, activation, routing, capabilities) keeps working unchanged.

VM / Desktop — bare Java process

Two equivalent options. Either drop the override next to the jar or pass its location on the command line.

Option 1 — drop ./config/application.yml next to the jar:

/opt/dvara/
├── gateway.jar
└── config/
    └── application.yml   ← your overrides.yml contents, renamed

Spring Boot picks up ./config/application.yml automatically — no flags needed.

cd /opt/dvara
export OPENAI_API_KEY='<your-openai-api-key>'
java -jar gateway.jar

Option 2 — pass an explicit additional location:

Useful when the override file lives outside the jar directory (e.g. managed by a config-management tool).

export OPENAI_API_KEY='<your-openai-api-key>'
export SPRING_CONFIG_ADDITIONAL_LOCATION=file:/etc/dvara/overrides.yml

java -jar /opt/dvara/gateway.jar

Or as a command-line flag:

java -jar /opt/dvara/gateway.jar \
  --spring.config.additional-location=file:/etc/dvara/overrides.yml

Either form layers the override on top of the bundled application.yml — values in the override win; everything else uses the default.

Docker Compose

From 1.8.2 an Enterprise Gateway cannot run on its own: it has no database, and it enrols with Flightdeck and gets its configuration from it. Start from the quick-start stack in dvara-examples, which wires Flightdeck, PostgreSQL, the enrolment secret and the Gateway's spool volume (see the Quickstart). Add the override to its dvara-gateway service with a docker-compose.override.yml beside the Compose file, which Compose reads automatically:

# docker-compose.override.yml
services:
  dvara-gateway:
    image: ghcr.io/dvarahq/dvara-gateway:1.8.5
    environment:
      SPRING_CONFIG_ADDITIONAL_LOCATION: file:/config/
    volumes:
      - ./overrides.yml:/config/application.yml:ro

Compose merges this into the stack's dvara-gateway service: the environment variable is added to the stack's own, and the mount is added beside the gateway-data volume. Pin dvara-flightdeck to the same version. Create overrides.yml beside the Compose file with the YAML from the top of this section, then bring the stack up:

docker compose up -d

The trailing slash on SPRING_CONFIG_ADDITIONAL_LOCATION=file:/config/ tells Spring to scan that directory for any application*.yml — which is why the mount target is /config/application.yml. The bundled application.yml inside the image keeps loading; your file layers on top.

Kubernetes

From 1.8.2 the Gateway needs Flightdeck to enrol with and get its configuration from, so deploy it with the DVARA Helm chart rather than a bare Deployment. Store the override in a ConfigMap, then mount it on the Gateway through the chart's llmGatewayServer.extraVolumes, extraVolumeMounts and extraEnv values.

# configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: dvara-overrides
data:
  application.yml: |
    dvara:
      llm-gateway:
        providers:
          openai:
            base-url: https://llm.corp.internal/v1
          ollama:
            base-url: http://ollama-servers.internal:11434

# overrides-values.yaml
llmGatewayServer:
  extraEnv:
    - name: SPRING_CONFIG_ADDITIONAL_LOCATION
      value: file:/config/
  extraVolumes:
    - name: overrides
      configMap:
        name: dvara-overrides
  extraVolumeMounts:
    - name: overrides
      mountPath: /config
      readOnly: true

Apply the ConfigMap, then add the values file to your install or upgrade:

kubectl apply -f configmap.yaml
helm upgrade --install dvara oci://ghcr.io/dvarahq/charts/dvara --version 1.8.5 \
  -f my-values.yaml -f overrides-values.yaml

The chart does not watch this ConfigMap. After you change it, restart the Gateway pods to pick it up, for example with kubectl rollout restart on the Gateway workload.

Common mount mistake

Don't mount your override over /workspace/config/application.yml (or wherever the image keeps its bundled application.yml). That replaces the defaults entirely — you lose every property the gateway expected to find, and startup fails with cryptic configuration-resolution errors.

Always mount into a separate directory (/config, /etc/dvara, or similar) and set SPRING_CONFIG_ADDITIONAL_LOCATION=file:/that-directory/. The overrides layer on top of the bundled defaults instead of replacing them. If startup logs show Spring loading zero application config files when you expected one, the mount path is wrong.

See Configuration Reference for the full set of properties you can override from YAML, and the Deployment pages for full topology-specific install guides.

Capabilities matrix

This matrix describes what DVARA 1.8 declares for each external provider family. A "yes" is a provider-level routing declaration, not a promise that every model in that provider's catalogue supports the feature.

Provider	Chat	Stream	Embeddings	Batch	Vision	Structured output	Native tools	Streamed tools	Streaming usage
OpenAI	yes	yes	yes	yes	yes	native	yes	yes	exact when upstream reports it¹
Azure OpenAI	yes	yes	no	yes	yes	native	yes	yes	exact when upstream reports it¹
Anthropic	yes	yes	no	no	yes	tool rewrite	yes	yes	exact from the stream⁴
Google Gemini	yes	yes	no	no	yes	native	yes	yes	exact from the stream⁴
AWS Bedrock	yes	yes	no	no	yes	tool rewrite	yes	yes	exact from stream metadata
Mistral	yes	yes	no	no	no	native	yes	yes	exact when upstream reports it¹
Cohere	yes	yes	no	no	no	no	no	no	estimated
Groq	yes	yes	no	no	no	JSON mode only	no	no	exact when upstream reports it²
Ollama	yes	yes	yes⁵	no	yes⁵	native⁵	yes	yes⁵	estimated
Qwen	yes	yes	no	no	no³	no³	no³	no³	exact when upstream reports it¹
DeepSeek	yes	yes	no	no	no	JSON mode only³	yes	yes	exact when upstream reports it¹
Moonshot	yes	yes	no	no	no	JSON mode only	yes	yes	exact when upstream reports it¹
ChatGLM	yes	yes	no	no	no³	JSON mode only	yes	yes	exact when upstream reports it¹
Grok	yes	yes	no	no	yes³	native	yes	yes	exact when upstream reports it¹

¹ DVARA asks these OpenAI-compatible stream APIs to include usage and preserves the terminal usage block when the upstream returns one. If the upstream omits that block, DVARA records an estimate.

² Groq reports stream usage in its final provider metadata. DVARA preserves it when present and estimates usage when it is absent.

³ These are conservative provider-level declarations. Qwen is declared off for vision, structured output, and tools. DeepSeek declares JSON mode but not strict json_schema. ChatGLM vision is off even though glm-4v-* models exist. Grok vision is on even though it is model-specific. Use explicit routes when a provider catalogue mixes models with different capabilities.

⁴ From 1.8.4. Anthropic and Gemini streams report exact usage on their last event. Before 1.8.4, DVARA estimated usage for these streams.

⁵ From 1.8.4. Before 1.8.4, Ollama declared none of these. Structured output is declared only while OLLAMA_STRUCTURED_OUTPUTS is true (the default); JSON mode is declared either way.

All 14 integrations support chat completions and streaming. OpenAI supports embeddings, and from 1.8.4 so does Ollama. Only OpenAI and Azure OpenAI support the Batch API. The built-in Mock provider is separate from the external matrix: it supports chat, streaming, batch, JSON mode, and wrapped structured output for local tests, but it does not make an upstream call or support embeddings, vision, or tools.

For a request with both tools and stream: true, DVARA requires yes in both tool columns. The stream then exposes OpenAI-compatible delta.tool_calls[]; see Function calling for assembly and governance behavior. GET /v1/models exposes native tool-call support but not the separate streamed-tool declaration in 1.8.

Structured-output, JSON-mode, tool, streamed-tool, and batch declarations are hard routing gates. Vision is a failover gate, but not a gate on the primary route because vision support can vary by model. A provider that cannot accept an image rejects it when called. Context-window limits are also enforced; the limits declared by each integration appear in the provider sections above.

One caveat: an explicit model prefix overrides routing. If a client sends model: "mistral-large-latest" and the route has Mistral + OpenAI, DVARA routes to Mistral regardless of what the request contains — the explicit prefix wins over capability selection.

For OpenAI-compatible long-tail providers wired via base-URL override (Fireworks AI, Together AI, Perplexity, vLLM hosts, LiteLLM proxies, internal corporate gateways), capabilities depend on the upstream — see Additional OpenAI-compatible providers.

See Routing for how this interacts with failover, latency-aware routing, and canary splits. See Multimodal and vision for request examples and a worked route setup for vision traffic.

Discover available models

GET /v1/models asks every registered provider for its current model list. The lookup is best-effort: if one provider fails, DVARA logs that failure, omits that provider's models, and still returns 200 with the models it could retrieve. An empty list therefore means "nothing was discovered," not necessarily that every configured provider is healthy.

The response is a merged list, not a deduplicated catalogue. If two providers return the same model ID, both entries remain. Each entry receives the capability declaration of the provider that returned it; DVARA does not probe each model separately. This is especially important for model-specific vision support.

The public capability object contains streaming, vision, tool-call, structured- output, JSON-mode, and context-window fields. It does not expose batch or streamed-tool-call support in 1.8, even though routing enforces both internally.

Provider capability gaps in this release

Some upstream provider features are not implemented in this release. The table below is the single source of truth for what's missing per provider; the per-provider sections above link here rather than restating it. The governance pipeline (policy, PII, audit, budget, guardrails) applies to every request regardless of which capability the upstream supports — gaps are about how DVARA talks to the upstream, not about what governance covers.

The columns:

Capability — the upstream feature.
Status — what DVARA does today when the request reaches that provider.
Workaround — the safe configuration to use until the gap is closed.

Provider	Capability	Status in this release	Workaround
Mistral	Vision (Pixtral 12B, Pixtral Large)	Not supported. A request containing an image is refused before the upstream call with HTTP 400 `unsupported_capability`.	Use a vision-capable provider (OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI). See Multimodal and vision.
Cohere	Vision	Not supported. Sending image content returns HTTP 400 `unsupported_capability`.	Use a vision-capable provider (see above).
Cohere	Tool calls	Not supported. Sending tool-use or tool-result content blocks returns HTTP 400 `unsupported_capability`.	Use a tool-capable provider (OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Mistral).
Cohere	Structured outputs / JSON mode	Not supported. Sending `response_format: json_schema` or `json_object` to a Cohere-only route returns HTTP 400 `no_capable_provider` before dispatch.	Configure a route that includes a structured-outputs-capable provider; capability-aware routing automatically excludes Cohere from the candidate pool.
Groq	Vision	Not supported. Sending image content returns HTTP 400 `unsupported_capability`.	Use a vision-capable provider.
Groq	Tool calls	Not supported. Sending tool-use or tool-result content blocks returns HTTP 400 `unsupported_capability`.	Use a tool-capable provider.
Groq	Structured outputs (`json_schema`)	Not supported. Sending `response_format: json_schema` to a Groq-only route returns HTTP 400 `no_capable_provider`. `json_object` mode works.	Use `response_format: json_object` if Groq's ultra-low latency matters and the schema is enforced client-side, or route to OpenAI, Gemini, Mistral, or Azure OpenAI for `json_schema`.
Ollama	Image URLs	From 1.8.4 an https image URL is refused before the call with HTTP 400 `unsupported_capability` unless image fetching is on. Ollama accepts only base64 images.	Send the image as base64, or turn on `dvara.llm-gateway.image-fetch.enabled` so DVARA fetches and inlines it.
Ollama	Structured outputs before Ollama 0.5	Ollama follows a JSON schema from 0.5. With `OLLAMA_STRUCTURED_OUTPUTS=false`, a `json_schema` request is not routed to Ollama; `json_object` still works.	Upgrade Ollama, or set `OLLAMA_STRUCTURED_OUTPUTS=false` and route `json_schema` traffic to a capable provider.

Structured-output and tool requirements automatically remove providers that do not declare those capabilities. Vision is different: it is not a primary-route filter because support can vary by model within one provider. Put only known vision-capable models and providers on an image route, and test the exact model before production. See Routing and Multimodal and vision.

Next steps

Routing — route by model, latency, cost, weight, or region, with canary splits and shadow traffic.
Credentials & BYOK — workspace-scoped provider keys, rotation, and vault integration.
Workspace isolation — how DVARA resolves the workspace from an API key and cascades per-workspace configuration.
Multimodal and vision — sending images through the gateway, supported providers, and route configuration for vision traffic.
API — interactive OpenAPI reference for every /v1/* endpoint, with a configurable Try it panel.

All supported providers​

OpenAI​

Azure OpenAI​

Anthropic​

Google Gemini​

AWS Bedrock​

Mistral​

Cohere​

Groq​

Ollama​

Alibaba Qwen​

DeepSeek​

Moonshot AI (Kimi)​

Zhipu AI (ChatGLM)​

Grok (xAI)​

Mock Provider​

Advanced: overriding the base URL​

VM / Desktop — bare Java process​

Docker Compose​

Kubernetes​

Capabilities matrix​

Discover available models​

Provider capability gaps in this release​

Next steps​

All supported providers

OpenAI

Azure OpenAI

Anthropic

Google Gemini

AWS Bedrock

Mistral

Cohere

Groq

Ollama

Alibaba Qwen

DeepSeek

Moonshot AI (Kimi)

Zhipu AI (ChatGLM)

Grok (xAI)

Mock Provider

Advanced: overriding the base URL

VM / Desktop — bare Java process

Docker Compose

Kubernetes

Capabilities matrix

Discover available models

Provider capability gaps in this release

Next steps