Skip to main content
Version: 1.9.0

How the gateway runs without a database

From 1.8.2 the Enterprise DVARA Gateway has no database. DVARA Flightdeck is the only process that connects to PostgreSQL, and the only one that migrates its schema. A Gateway pod:

  • enrols with Flightdeck on its first start and gets a certificate for Flightdeck's /internal channel;
  • reads its configuration from the signed bundle Flightdeck publishes;
  • writes audit, usage, cost and tool-call records to a local spool, and ships them to Flightdeck, which stores them;
  • gets revocations from a separate signed deny-list.

This is the only way an Enterprise Gateway runs. There is nothing to opt into and no switch to turn it off. DVARA Open Source is different: it reads one gateway.yaml file and needs neither Flightdeck nor PostgreSQL.

Upgrading from 1.8.1 or earlier

Up to 1.8.1 a Gateway pod connected to PostgreSQL by default, and this page described an opt-in mode with two switches, a shadow comparison and a cut-over. All of that was removed in 1.8.2. See what moves on upgrade.

Why the gateway holds no database credential​

Blast radius. The Gateway is the process that uses every workspace's provider API keys and PII keys. If it also held a PostgreSQL credential, a compromise of the Gateway would be a compromise of the database. A Gateway pod holds no database credential at all, which is stronger than any set of GRANTs, because there is nothing to grant.

Availability. A Gateway pod keeps serving from the bundle it already holds for as long as a Flightdeck or PostgreSQL outage lasts. Its records wait in the spool and ship when Flightdeck is back.

What a gateway needs​

SettingEnvironment variableWhat it is
dvara.data-plane.control-plane.base-urlDVARA_DATA_PLANE_CONTROL_PLANE_BASE_URLFlightdeck's /internal address, for example https://flightdeck:8443. The bundle comes from here and the spool drains here. Required.
dvara.llm-gateway.local-store.pathDVARA_LLM_GATEWAY_LOCAL_STORE_PATHThe spool file, on a volume that survives a restart. Required.
dvara.data-plane.control-plane.enrolment.shared-secretDVARA_DATA_PLANE_CONTROL_PLANE_ENROLMENT_SHARED_SECRETThe same value as Flightdeck's DVARA_INTERNAL_ENROLMENT_SHARED_SECRET, at least 32 characters. Or use a one-time token instead.
dvara.data-plane.control-plane.enrolment.directoryDVARA_DATA_PLANE_CONTROL_PLANE_ENROLMENT_DIRECTORYWhere the pod keeps its key and certificate. Setting it turns enrolment on for the pod. Put it on the same persistent volume.
dvara.data-plane.control-plane.snapshot-pathDVARA_DATA_PLANE_CONTROL_PLANE_SNAPSHOT_PATHWhere the pod saves the last bundle it verified, so it can start while Flightdeck is down. Recommended.

The Gateway refuses to start without the first two and names the missing one. It also refuses to start, and names the setting with what to do instead, when it is given SPRING_DATASOURCE_URL, DVARA_BOOTSTRAP_FILE, GATEWAY_BOOTSTRAP_FILE or dvara.data-plane.serve-from-bundle=false.

dvara.data-plane.control-plane.public-key is optional. Enrolment delivers the key Flightdeck signs bundles with. A key you set still wins.

This is the Gateway and Flightdeck wiring from the Enterprise Compose file, trimmed to the parts that matter here:

services:
flightdeck:
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/dvara # Flightdeck only
DVARA_INTERNAL_ENROLMENT_ENABLED: "true"
DVARA_INTERNAL_ENROLMENT_SERVER_NAMES: flightdeck,localhost
DVARA_INTERNAL_ENROLMENT_SHARED_SECRET: ${DVARA_ENROLMENT_SECRET}

gateway:
environment:
# No database settings: the gateway refuses to start with one.
DVARA_DATA_PLANE_CONTROL_PLANE_BASE_URL: https://flightdeck:8443
DVARA_DATA_PLANE_CONTROL_PLANE_ENROLMENT_DIRECTORY: /var/lib/dvara/enrolment
DVARA_DATA_PLANE_CONTROL_PLANE_ENROLMENT_SHARED_SECRET: ${DVARA_ENROLMENT_SECRET}
DVARA_DATA_PLANE_CONTROL_PLANE_SNAPSHOT_PATH: /var/lib/dvara/bundle-snapshot.json
DVARA_LLM_GATEWAY_LOCAL_STORE_PATH: /var/lib/dvara/spool.db
volumes: ["gateway-data:/var/lib/dvara"]

volumes:
gateway-data:

Generate the secret once with openssl rand -base64 32 and give both services the same value.

How a gateway enrols​

Flightdeck issues the certificates for its /internal channel, so you do not have to make them by hand.

  • Flightdeck keeps its own certificate authority. With dvara.internal.enrolment.enabled=true it creates the authority on first start and serves /internal on a second HTTPS port, dvara.internal.enrolment.port (default 8443). Its server certificate names dvara.internal.enrolment.server-names (default flightdeck and localhost, plus the host name). Set these to the name your gateways put in base-url. Enrolment is off by default in Flightdeck itself; the Compose file and the Helm chart turn it on.
  • With the shared secret, a gateway that has no certificate and no token enrols on its own. Set the same value, at least 32 characters, on Flightdeck (dvara.internal.enrolment.shared-secret) and on every gateway. Flightdeck refuses to start with a shorter value. The gateway trusts Flightdeck's reply only if it carries a signature made with the secret.
  • With a one-time token, an owner creates the token on the Flightdeck page Gateway Enrolment for the gateway's scope, for example gateway:default. A token works once, for 24 hours. Set it in dvara.data-plane.control-plane.enrolment.token. The gateway trusts Flightdeck only if Flightdeck's authority matches the pin in the token.
  • Either way, the gateway makes its own key and gets a 30-day certificate for its scope, which it renews itself 10 days before it expires. Enrolment also delivers the key Flightdeck signs bundles with.

Start Flightdeck first. A gateway that cannot enrol within about a minute exits, and your restart policy starts it again.

The certificate's name is the pod's scope. CN=gateway:eu may fetch the gateway:eu bundle and nothing else. With a wrong name the pod authenticates, then receives 403 for its own scope. Check the name before anything else.

If you make the certificates yourself​

Certificates you make still work. In the chart, set dataPlane.enrolment.enabled: false and supply the secret through dataPlane.tls, which sets both properties below. By hand, two properties mention the certificate and they do different jobs:

propertywhat it does
spring.http.clients.ssl.bundlecarries the certificate. It applies to the HTTP client the pod fetches with.
dvara.data-plane.control-plane.ssl-bundlechecks the CN against the scope. Nothing else.

Usually both name the same bundle. Setting only the second gives a pod that starts cleanly, passes its scope check, and then fails every fetch with PKIX path building failed, because no certificate was attached to the client.

Note the plural in spring.http.clients.ssl.bundle

The older spelling spring.http.client.ssl.bundle is silently ignored: no warning, no error, just a pod with no certificate.

The config bundle​

DVARA Flightdeck materializes every workspace, key, route, policy, credential, price, budget, schema, prompt, plugin, MCP server, A2A agent, and the current platform licence into one document and signs it with Ed25519. Pods fetch it, verify the signature, and activate the complete verified bundle.

A first, renewed or replaced licence is applied to a running pod on the next bundle update, without a restart. From 1.8.2 this includes the first licence on a pod that started without one. The workspace and call limits follow it on the next request.

You can read the public key through the owner-only bundle-signing-key operation described in the API overview, for example to pin it. It is the one narrow read-only operational route retained under /v1/admin. If you set no signing key, DVARA Flightdeck generates one and keeps it, so every enrolment delivers the same key.

Rotating the signing key is a rolling change, not a flag day. A pod accepts public-key plus a list of additional-public-keys: add the incoming key fleet-wide first, then switch Flightdeck to sign with it, then drop the old one.

How long a change takes to arrive​

A change made in Flightdeck reaches a gateway after a delay, and a gateway's records reach Flightdeck after a delay. At the default settings:

WhatWorst caseSettings
A config change (a new API key, a policy, a route)about 10 sdvara.config.bundle.republish-interval-ms (5 s, Flightdeck) and dvara.data-plane.control-plane.poll-interval-ms (5 s for the bundle, gateway)
A revocation (a revoked key)about 10 sdvara.internal.deny-list.republish-interval-ms (5 s, Flightdeck) and, from 1.8.5, dvara.data-plane.control-plane.deny-list-poll-interval-ms (5 s, gateway)
A revoked provider credentialabout 10 sThe same as a revoked key, from 1.9.0. Before 1.9.0 a gateway that had just used the credential could keep using it for up to 30 s more.
A session kill made in Flightdeckabout 5 sFrom 1.8.5 Flightdeck publishes the list the moment a kill is saved, so only the gateway's 5 s poll remains. Before 1.8.5, about 15 s.
A loop auto-kill made on another gatewaymillisecondsFrom 1.8.5 gateways in one cluster relay kills to each other. Without a cluster, the kill arrives through the revocation list.
Usage, cost, audit and tool-call rows in Flightdeckabout 30 sdvara.llm-gateway.local-store.drain-interval-ms (30 s, gateway)

From 1.8.5 each gateway poll has its own property: dvara.data-plane.control-plane.poll-interval-ms for the bundle and dvara.data-plane.control-plane.deny-list-poll-interval-ms for the revocation list, both 5 s by default. Before 1.8.5 one property set both, with 10 s for the list. An install that set poll-interval-ms keeps that value for the list until it sets the new property. A script that changes something in Flightdeck and then calls the gateway must wait for the change to arrive. A script that reads a gateway's record in Flightdeck must wait for the spool to drain.

Cold start during an outage​

Set dvara.data-plane.control-plane.snapshot-path to a durable path and the pod saves every bundle it verifies, then reloads it at boot before the first fetch. The snapshot is written atomically and verified again on load, so a crash cannot leave a torn file and a tampered one is rejected.

From 1.8.2 a gateway on a fresh volume can also start while Flightdeck is down. Gateways keep a copy of the bundle and the revocation list in the cluster they form among themselves and, when dvara.llm-gateway.rate-limit.shared-cache.url is set, in that Valkey or Redis. At start a gateway reads every source, checks each copy's signature, and applies the newest valid one, so an older copy cannot bring back something already revoked.

One gap remains

A brand-new pod on a fresh volume cannot serve when Flightdeck is down and no other gateway or shared cache holds a copy. There is nowhere for it to get configuration from.

Staleness is reported, never enforced​

A pod serving old configuration shows it on /actuator/gateway-status once the bundle passes staleness-warn-seconds (default one hour). It is an alarm, not a cut-off. A pod is never stopped from serving because its config is old, because that would turn a control-plane outage into a data-plane outage.

The audit spool​

Records land in the local spool and ship to DVARA Flightdeck in idempotent batches. The chain is per-pod: each pod signs its own HMAC chain, so edits, gaps, or reordering stay detectable while Flightdeck is unavailable.

The spool carries both the shared LLM and MCP audit chain and the A2A plane's separate tamper-evident chain. The pod signs and stores each chain locally, then drains each one to Flightdeck without combining their order.

From 1.8.5 /actuator/gateway-status shows the A2A backlog in its own field. Its spool block has a2a_audit_depth beside audit_depth, which counts the LLM and MCP chain alone. The "spool behind" warning says how many waiting records belong to each.

Audit cannot be shed. The spool is bounded, and when it fills, usage, cost and observability rows are dropped, never audit events. Telemetry can be rebuilt or lost without harm; an audit event cannot.

A full volume stops traffic instead of losing audit. When free room on the spool volume falls below dvara.llm-gateway.local-store.audit-reserve-bytes (64 MiB by default), the gateway refuses /v1, /mcp and /a2a calls with 503 audit_store_full and a Retry-After header, and reports not ready.

Tokenization degrades visibly. Past a per-workspace high-water mark the pod stops minting reversible PII tokens and applies the workspace's pii.degraded-action, which is BLOCK by default. This applies only to TOKENIZE. Irreversible REDACT stores nothing and keeps working.

Drain on shutdown. On SIGTERM the pod flushes the spool, bounded by shutdownDrainSeconds (default 20). Every routine path (rolling update, scale-in, node drain) is therefore graceful, and the volume covers only the ungraceful ones. Your termination grace period must exceed that value, or the flush is cut off.

From 1.8.4 the drain always ends within that budget, even when Flightdeck has stopped answering. Before, one call that never answered could hold the pod in shutdown until it was killed. When the budget runs out, the gateway logs how many records are still spooled and stops. Those records stay on the volume and are shipped after the next start.

Revocation still works during an outage​

A second, minimal channel carries a signed deny-list of API keys, workspaces, sessions and credentials. Gateways poll it every 5 seconds (10 before 1.8.5) and alarm at five minutes rather than the bundle's hour.

Every entry on the list takes effect on every plane it applies to: a revoked key and a suspended workspace are refused on /v1, /mcp and /a2a, and a revoked credential is no longer used for model calls. From 1.9.0 each refusal is also in the audit trail on every plane; see audit events.

From 1.8.5 one session kill covers all three planes. A session is a workspace and an X-Session-Id, and an agent sends the same ID to models, tools and peers. Every plane reads every kill on the list, so a kill stops the session's model calls, tool calls and A2A hops. A kill stays on the list, and in Flightdeck, for 24 hours (dvara.internal.deny-list.session-kill-retention). Before 1.8.5 a kill left the list after 75 minutes, and a killed agent that kept retrying was served again after that.

Gateways in one cluster also pass kills to each other directly, within milliseconds. The relay can only add kills, never lift one, and nothing on the request path waits for it. The revocation list stays the authority, so a kill still arrives when the cluster is absent or split. /actuator/gateway-status reports the cluster size in its sessionKills section: a fleet whose gateways each report 1 is not relaying. Where multicast is blocked, list the gateways in dvara.cluster.members; see session-kill settings.

It never hard-refuses: a pod that cannot reach the channel keeps serving and keeps the revocations it already holds. Only revocations issued during the outage are delayed.

deny-list-url defaults to the same Flightdeck as the bundle. That is convenient, and it gives up the independence the channel exists for. Point it at a mirror if that guarantee matters to you.

What this means for operators​

  • The pod's configuration is read-only. Make changes in DVARA Flightdeck; the next verified bundle carries them to the pod.
  • A bootstrap file belongs on Flightdeck. Set DVARA_BOOTSTRAP_FILE on Flightdeck, which seeds the database the bundle is published from.
  • A session reaches Flightdeck a few seconds after the gateway serves it. From 1.8.3 a Flightdeck kill that names its workspace, as every Flightdeck kill button does, works in that window too: it is saved, audited, and published on the revocation list, so the gateway refuses the session within one poll.
  • A kill stops a session, not an agent. An agent can start a new session ID. To stop an agent, revoke its API key.

On Kubernetes​

The chart runs this setup with no values to set:

  • It turns enrolment on and generates the shared secret on install (secrets.enrolmentSharedSecret), keeping it across upgrades. Tools that render without a cluster, such as helm template or Argo CD, cannot read it back: set it explicitly there. With secrets.existingSecret, that Secret must carry the key enrolment-shared-secret.
  • Flightdeck serves enrolment and /internal on port 8443 through a separate -internal Service. dataPlane.controlPlane.baseUrl defaults to it; set it only when Flightdeck runs elsewhere.
  • The gateway always has a data volume at llmGatewayServer.persistence.mountPath (/var/lib/dvara). By default it is an emptyDir: it survives a container restart but not the pod's deletion, and a new pod enrols again.

llmGatewayServer.persistence.enabled: true turns the workload into a StatefulSet, because a Deployment cannot give each replica its own read-write volume. That is a change of kind: updates become ordered, pods get stable ordinals, and zonal volumes pin a pod to its zone. It is recommended for production and off by default.

PostgreSQL is supplied to Flightdeck only, as SPRING_DATASOURCE_* through flightdeck.extraEnv. See Deploy on Kubernetes.

Three availability questions, answered three different ways​

A gateway can lose contact with Flightdeck. What it should do then looks like one question and is three, and they are deliberately not answered alike:

when the pod cannot check…it…because
a budgetservesEnforcement must fail closed on evidence of a breach, not on the absence of evidence. An overshoot is money, it is visible afterwards in the ledger, and it reconciles.
a revocationserves, brieflyA revocation is late, not absent: the deny-list is a second channel with its own short poll, and it lands when contact resumes. The window is bounded and alarmed.
PII tokenizationrefuses by defaultA workspace asked for reversible TOKENIZE, but the pod cannot mint a token. It applies pii.degraded-action, whose default is BLOCK, rather than silently sending the raw value. Irreversible REDACT does not depend on this path.

Reversibility is the whole rule. The first two failures can be undone or paid for after the fact; the third cannot. So the third is the one place DVARA fails closed, and serving instead takes an explicit per-workspace opt-in (pii.degraded-action). An unrecognised value for that setting resolves to BLOCK, so a typo cannot open the door.

What moves on upgrade​

Moving from 1.8.1 or earlier to 1.8.2 or later:

  • Remove the database settings from the gateway. A gateway given SPRING_DATASOURCE_URL, DVARA_BOOTSTRAP_FILE, GATEWAY_BOOTSTRAP_FILE or dvara.data-plane.serve-from-bundle=false refuses to start. The database pool, socket-timeout and migration variables are no longer read on the gateway.
  • Give the gateway Flightdeck's address, the enrolment secret and a volume for its spool, as shown above. Move a bootstrap file to Flightdeck.
  • Helm: dataPlane.serveFromBundle and dataPlane.dbMigrate were removed. A values file that still sets either, or that gives the gateway SPRING_DATASOURCE_* or DVARA_BOOTSTRAP_FILE through llmGatewayServer.extraEnv, fails helm upgrade with the setting named. dataPlane.controlPlane.publicKey is now optional.
  • Removed with the database: the config cache in front of the database and its stale-read fallback (dvara.llm-gateway.caching.distributed.enabled, dvara.llm-gateway.stale-reads.*, the gateway_config_stale_seconds metric), and the shadow comparison of each bundle against the database (the gateway_bundle_shadow_divergence_total metric).

What this does not do​

/internal/ingest, /internal/config/bundle, /internal/deny-list, /internal/pii/dek/* (including erase), /internal/batch, and /internal/approvals/* are pod-to-Flightdeck paths. They require the pod's certificate and are not a public API. Keep them behind an internal listener or firewall rule.

  • It does not remove PostgreSQL. DVARA Flightdeck still needs it and is the only database client.
  • It does not change what the gateway serves, or how governance behaves.