Run the open-source gateway

Reference for the open-source DVARA LLM Gateway, run from one gateway.yaml. The quickstart gets it running.

Audit log

To keep an audit record, set two variables:

DVARA_AUDIT_FILE_PATH=/var/lib/dvara/audit.log
DVARA_AUDIT_HMAC_SECRET=$(openssl rand -base64 32)

Without the path, events are dropped. With the path, the gateway refuses to start unless the secret is at least 32 characters and is not a placeholder: not the shipped default, and not a value from DVARA's own examples, such as dev-only-change-me. The chain is tamper-evident, not tamper-proof: it detects an edited, removed or reordered record, and anyone holding the secret can rewrite it.

Verify the chain, from the module jars or from the runnable jar

DVARA_AUDIT_HMAC_SECRET=... java \
  -cp dvara-gateway-autoconfigure/target/dvara-gateway-autoconfigure-<version>.jar:dvara-gateway-core/target/dvara-gateway-core-<version>.jar \
  com.dvarahq.autoconfigure.audit.FileAuditChainVerifier /var/lib/dvara/audit.log

The two jars and a JDK are the whole requirement, so the check can run on another machine. From the runnable jar instead:

DVARA_AUDIT_HMAC_SECRET=... java -cp dvara-gateway-server-<version>-app.jar \
  -Dloader.main=com.dvarahq.autoconfigure.audit.FileAuditChainVerifier \
  org.springframework.boot.loader.launch.PropertiesLauncher /var/lib/dvara/audit.log

The verifier exits 0 when the chain verified over at least one record, 1 when a record failed, naming the line, and 2 when nothing was checked. The secret is read from the environment, never from an argument.

To write the record somewhere else, implement AuditWriter from dvara-gateway-core and register it as a bean; the file writer stands down. HmacSigner in the same module gives you the bytes to sign and the previous-hash rule.

Deployment

The jar from a release and the jar from a build run the same way, as a plain Java process on port 8080. It reads gateway.yaml from the working directory; to keep the file elsewhere, name it:

DVARA_CONFIG_FILE=/etc/dvara/gateway.yaml \
OPENAI_API_KEY=sk-... \
java -jar dvara-gateway-server-<version>-app.jar

The image ghcr.io/dvarahq/dvara-gateway-oss is the same jar with a JRE. Its working directory is /app, so a file mounted at /app/gateway.yaml is found with no further setting, and every environment variable above is passed with -e. To keep the audit log, mount a volume at /var/lib/dvara and point DVARA_AUDIT_FILE_PATH into it. Pin a version tag in production.

JVM options in the image

The image starts Java directly, with no shell, so JAVA_OPTS does nothing. Its defaults are in JAVA_TOOL_OPTIONS: -XX:+UseG1GC -XX:MaxRAMPercentage=75 -XX:+AlwaysPreTouch -XX:+ExitOnOutOfMemoryError. Add or change an option with JDK_JAVA_OPTIONS, which comes after the defaults, so its value wins: -e JDK_JAVA_OPTIONS=-XX:MaxRAMPercentage=60. Setting JAVA_TOOL_OPTIONS replaces all the defaults. G1 is the default collector: ZGC stalled under load when the gateway shared its cores with other work, and G1 did not. To run ZGC, pass -XX:-UseG1GC -XX:+UseZGC; naming a second collector without turning G1 off stops the boot with Multiple garbage collectors selected. A wrong option always stops the boot with Unrecognized VM option, so a typo cannot go unnoticed.

The image carries an AOT cache (/app/app.aot), built when the image is built by starting the gateway once. It makes startup about a third faster. It was built under G1: under another collector the JVM logs that it cannot use the cache and starts without it.

Calls to providers

Every provider is called through the JDK HTTP client. It keeps idle connections to each provider and opens another whenever all are busy, so calls do not queue for a small pool. When the application runs on virtual threads (spring.threads.virtual.enabled, on by default), so does the client's own work; otherwise it uses the JDK's default pool of platform threads.

Variable	Default	Meaning
DVARA_UPSTREAM_HTTP_CONNECT_TIMEOUT	10s	Time to open a connection to a provider
DVARA_UPSTREAM_HTTP_READ_TIMEOUT	none	Longest wait for an answer to start; none, since a slow model can take minutes
DVARA_UPSTREAM_HTTP_VERSION	HTTP/2 where offered	HTTP_1_1 or HTTP_2

An empty value falls back to Spring Boot's spring.http.clients.* settings. The idle connections are tuned with JVM system properties in JDK_JAVA_OPTIONS: -Djdk.httpclient.keepalive.timeout=<seconds> and -Djdk.httpclient.connectionPoolSize=<n>.

Providers without a key in the environment

A provider that takes an API key is on when its key is set (OPENAI_API_KEY) or when its switch is (OPENAI_ENABLED=true). With the switch, no key has to be in the environment or the config: the key is looked up on each call, from wherever this build's secret provider reads it. Each provider has the pair: ANTHROPIC_ENABLED, GEMINI_ENABLED, AZURE_OPENAI_ENABLED (Azure still needs its base URL), MISTRAL_ENABLED, COHERE_ENABLED, GROQ_ENABLED, QWEN_ENABLED, DEEPSEEK_ENABLED, MOONSHOT_ENABLED, ZHIPU_ENABLED and XAI_ENABLED. Setting only the key works as before.

Observability

Health probes at /actuator/health/liveness and /actuator/health/readiness are open. Metrics at /actuator/prometheus answer only to the key in DVARA_ACTUATOR_METRICS_API_KEY, and the other actuator endpoints to DVARA_ACTUATOR_API_KEY; with a key unset, that endpoint returns 401. Traces go to OpenTelemetry when OTEL_EXPORTER_OTLP_ENDPOINT names a collector and nowhere otherwise.

Every request is traced by default (TRACING_SAMPLING_PROBABILITY=1.0), which suits development. Each traced request costs a few percent of the gateway's CPU at high rates, so in production set a lower rate, such as 0.1, unless every trace is needed.