How the gateway runs without a database
From 1.8.2 the Enterprise DVARA Gateway has no database. DVARA Flightdeck is the only process that connects to PostgreSQL, and the only one that migrates its schema. A Gateway pod:
- enrols with Flightdeck on its first start and gets a certificate for
Flightdeck's
/internalchannel; - reads its configuration from the signed bundle Flightdeck publishes;
- writes audit, usage, cost and tool-call records to a local spool, and ships them to Flightdeck, which stores them;
- gets revocations from a separate signed deny-list.
This is the only way an Enterprise Gateway runs. There is nothing to opt into
and no switch to turn it off. DVARA Open Source is different: it reads one
gateway.yaml file and needs neither Flightdeck nor PostgreSQL.
Up to 1.8.1 a Gateway pod connected to PostgreSQL by default, and this page described an opt-in mode with two switches, a shadow comparison and a cut-over. All of that was removed in 1.8.2. See what moves on upgrade.
Why the gateway holds no database credential
Blast radius. The Gateway is the process that uses every workspace's
provider API keys and PII keys. If it also held a PostgreSQL credential, a
compromise of the Gateway would be a compromise of the database. A Gateway pod
holds no database credential at all, which is stronger than any set of
GRANTs, because there is nothing to grant.
Availability. A Gateway pod keeps serving from the bundle it already holds for as long as a Flightdeck or PostgreSQL outage lasts. Its records wait in the spool and ship when Flightdeck is back.
What a gateway needs
| Setting | Environment variable | What it is |
|---|---|---|
dvara.data-plane.control-plane.base-url | DVARA_DATA_PLANE_CONTROL_PLANE_BASE_URL | Flightdeck's /internal address, for example https://flightdeck:8443. The bundle comes from here and the spool drains here. Required. |
dvara.llm-gateway.local-store.path | DVARA_LLM_GATEWAY_LOCAL_STORE_PATH | The spool file, on a volume that survives a restart. Required. |
dvara.data-plane.control-plane.enrolment.shared-secret | DVARA_DATA_PLANE_CONTROL_PLANE_ENROLMENT_SHARED_SECRET | The same value as Flightdeck's DVARA_INTERNAL_ENROLMENT_SHARED_SECRET, at least 32 characters. Or use a one-time token instead. |
dvara.data-plane.control-plane.enrolment.directory | DVARA_DATA_PLANE_CONTROL_PLANE_ENROLMENT_DIRECTORY | Where the pod keeps its key and certificate. Setting it turns enrolment on for the pod. Put it on the same persistent volume. |
dvara.data-plane.control-plane.snapshot-path | DVARA_DATA_PLANE_CONTROL_PLANE_SNAPSHOT_PATH | Where the pod saves the last bundle it verified, so it can start while Flightdeck is down. Recommended. |
The Gateway refuses to start without the first two and names the missing one.
It also refuses to start, and names the setting with what to do instead, when it
is given SPRING_DATASOURCE_URL, DVARA_BOOTSTRAP_FILE,
GATEWAY_BOOTSTRAP_FILE or dvara.data-plane.serve-from-bundle=false.
dvara.data-plane.control-plane.public-key is optional. Enrolment delivers the
key Flightdeck signs bundles with. A key you set still wins.
This is the Gateway and Flightdeck wiring from the Enterprise Compose file, trimmed to the parts that matter here:
services:
flightdeck:
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/dvara # Flightdeck only
DVARA_INTERNAL_ENROLMENT_ENABLED: "true"
DVARA_INTERNAL_ENROLMENT_SERVER_NAMES: flightdeck,localhost
DVARA_INTERNAL_ENROLMENT_SHARED_SECRET: ${DVARA_ENROLMENT_SECRET}
gateway:
environment:
# No database settings: the gateway refuses to start with one.
DVARA_DATA_PLANE_CONTROL_PLANE_BASE_URL: https://flightdeck:8443
DVARA_DATA_PLANE_CONTROL_PLANE_ENROLMENT_DIRECTORY: /var/lib/dvara/enrolment
DVARA_DATA_PLANE_CONTROL_PLANE_ENROLMENT_SHARED_SECRET: ${DVARA_ENROLMENT_SECRET}
DVARA_DATA_PLANE_CONTROL_PLANE_SNAPSHOT_PATH: /var/lib/dvara/bundle-snapshot.json
DVARA_LLM_GATEWAY_LOCAL_STORE_PATH: /var/lib/dvara/spool.db
volumes: ["gateway-data:/var/lib/dvara"]
volumes:
gateway-data:
Generate the secret once with openssl rand -base64 32 and give both services
the same value.
How a gateway enrols
Flightdeck issues the certificates for its /internal channel, so you do not
have to make them by hand.
- Flightdeck keeps its own certificate authority. With
dvara.internal.enrolment.enabled=trueit creates the authority on first start and serves/internalon a second HTTPS port,dvara.internal.enrolment.port(default8443). Its server certificate namesdvara.internal.enrolment.server-names(defaultflightdeckandlocalhost, plus the host name). Set these to the name your gateways put inbase-url. Enrolment is off by default in Flightdeck itself; the Compose file and the Helm chart turn it on. - With the shared secret, a gateway that has no certificate and no token
enrols on its own. Set the same value, at least 32 characters, on Flightdeck
(
dvara.internal.enrolment.shared-secret) and on every gateway. Flightdeck refuses to start with a shorter value. The gateway trusts Flightdeck's reply only if it carries a signature made with the secret. - With a one-time token, an owner creates the token on the Flightdeck page
Gateway Enrolment for the gateway's scope, for example
gateway:default. A token works once, for 24 hours. Set it indvara.data-plane.control-plane.enrolment.token. The gateway trusts Flightdeck only if Flightdeck's authority matches the pin in the token. - Either way, the gateway makes its own key and gets a 30-day certificate for its scope, which it renews itself 10 days before it expires. Enrolment also delivers the key Flightdeck signs bundles with.
Start Flightdeck first. A gateway that cannot enrol within about a minute exits, and your restart policy starts it again.
The certificate's name is the pod's scope. CN=gateway:eu may fetch the
gateway:eu bundle and nothing else. With a wrong name the pod authenticates,
then receives 403 for its own scope. Check the name before anything else.
If you make the certificates yourself
Certificates you make still work. In the chart, set
dataPlane.enrolment.enabled: false and supply the secret through
dataPlane.tls, which sets both properties below. By hand, two properties
mention the certificate and they do different jobs:
| property | what it does |
|---|---|
spring.http.clients.ssl.bundle | carries the certificate. It applies to the HTTP client the pod fetches with. |
dvara.data-plane.control-plane.ssl-bundle | checks the CN against the scope. Nothing else. |
Usually both name the same bundle. Setting only the second gives a pod that
starts cleanly, passes its scope check, and then fails every fetch with
PKIX path building failed, because no certificate was attached to the client.
spring.http.clients.ssl.bundleThe older spelling spring.http.client.ssl.bundle is silently ignored: no
warning, no error, just a pod with no certificate.
The config bundle
DVARA Flightdeck materializes every workspace, key, route, policy, credential, price, budget, schema, prompt, plugin, MCP server, A2A agent, and the current platform licence into one document and signs it with Ed25519. Pods fetch it, verify the signature, and activate the complete verified bundle.
A first, renewed or replaced licence is applied to a running pod on the next bundle update, without a restart. From 1.8.2 this includes the first licence on a pod that started without one. The workspace and call limits follow it on the next request.
You can read the public key through the owner-only bundle-signing-key operation
described in the API overview,
for example to pin it. It is the one narrow read-only operational route retained
under /v1/admin. If you set no signing key, DVARA Flightdeck generates one and
keeps it, so every enrolment delivers the same key.
Rotating the signing key is a rolling change, not a flag day. A pod accepts
public-key plus a list of additional-public-keys: add the incoming key
fleet-wide first, then switch Flightdeck to sign with it, then drop the old one.
How long a change takes to arrive
A change made in Flightdeck reaches a gateway after a delay, and a gateway's records reach Flightdeck after a delay. At the default settings:
| What | Worst case | Settings |
|---|---|---|
| A config change (a new API key, a policy, a route) | about 10 s | dvara.config.bundle.republish-interval-ms (5 s, Flightdeck) and dvara.data-plane.control-plane.poll-interval-ms (5 s for the bundle, gateway) |
| A revocation (a revoked key) | about 10 s | dvara.internal.deny-list.republish-interval-ms (5 s, Flightdeck) and, from 1.8.5, dvara.data-plane.control-plane.deny-list-poll-interval-ms (5 s, gateway) |
| A revoked provider credential | about 10 s | The same as a revoked key, from 1.9.0. Before 1.9.0 a gateway that had just used the credential could keep using it for up to 30 s more. |
| A session kill made in Flightdeck | about 5 s | From 1.8.5 Flightdeck publishes the list the moment a kill is saved, so only the gateway's 5 s poll remains. Before 1.8.5, about 15 s. |
| A loop auto-kill made on another gateway | milliseconds | From 1.8.5 gateways in one cluster relay kills to each other. Without a cluster, the kill arrives through the revocation list. |
| Usage, cost, audit and tool-call rows in Flightdeck | about 30 s | dvara.llm-gateway.local-store.drain-interval-ms (30 s, gateway) |
From 1.8.5 each gateway poll has its own property:
dvara.data-plane.control-plane.poll-interval-ms for the bundle and
dvara.data-plane.control-plane.deny-list-poll-interval-ms for the revocation
list, both 5 s by default. Before 1.8.5 one property set both, with 10 s for
the list. An install that set poll-interval-ms keeps that value for the list
until it sets the new property. A script that
changes something in Flightdeck and then calls the gateway must wait for the
change to arrive. A script that reads a gateway's record in Flightdeck must
wait for the spool to drain.
Cold start during an outage
Set dvara.data-plane.control-plane.snapshot-path to a durable path and the pod
saves every bundle it verifies, then reloads it at boot before the first
fetch. The snapshot is written atomically and verified again on load, so a crash
cannot leave a torn file and a tampered one is rejected.
From 1.8.2 a gateway on a fresh volume can also start while Flightdeck is down.
Gateways keep a copy of the bundle and the revocation list in the cluster they
form among themselves and, when dvara.llm-gateway.rate-limit.shared-cache.url
is set, in that Valkey or Redis. At start a gateway reads every source, checks
each copy's signature, and applies the newest valid one, so an older copy cannot
bring back something already revoked.
A brand-new pod on a fresh volume cannot serve when Flightdeck is down and no other gateway or shared cache holds a copy. There is nowhere for it to get configuration from.
Staleness is reported, never enforced
A pod serving old configuration shows it on /actuator/gateway-status once the
bundle passes staleness-warn-seconds (default one hour). It is an alarm, not a
cut-off. A pod is never stopped from serving because its config is old, because
that would turn a control-plane outage into a data-plane outage.
The audit spool
Records land in the local spool and ship to DVARA Flightdeck in idempotent batches. The chain is per-pod: each pod signs its own HMAC chain, so edits, gaps, or reordering stay detectable while Flightdeck is unavailable.
The spool carries both the shared LLM and MCP audit chain and the A2A plane's separate tamper-evident chain. The pod signs and stores each chain locally, then drains each one to Flightdeck without combining their order.
From 1.8.5 /actuator/gateway-status shows the A2A backlog in its own field.
Its spool block has a2a_audit_depth beside audit_depth, which counts the
LLM and MCP chain alone. The "spool behind" warning says how many waiting
records belong to each.
Audit cannot be shed. The spool is bounded, and when it fills, usage, cost and observability rows are dropped, never audit events. Telemetry can be rebuilt or lost without harm; an audit event cannot.
A full volume stops traffic instead of losing audit. When free room on the
spool volume falls below dvara.llm-gateway.local-store.audit-reserve-bytes
(64 MiB by default), the gateway refuses /v1, /mcp and /a2a calls with
503 audit_store_full and a Retry-After header, and reports not ready.
Tokenization degrades visibly. Past a per-workspace high-water mark the pod
stops minting reversible PII tokens and applies the workspace's
pii.degraded-action, which is BLOCK by default. This applies only to
TOKENIZE. Irreversible REDACT stores nothing and keeps working.
Drain on shutdown. On SIGTERM the pod flushes the spool, bounded by
shutdownDrainSeconds (default 20). Every routine path (rolling update,
scale-in, node drain) is therefore graceful, and the volume covers only the
ungraceful ones. Your termination grace period must exceed that value, or the
flush is cut off.
From 1.8.4 the drain always ends within that budget, even when Flightdeck has stopped answering. Before, one call that never answered could hold the pod in shutdown until it was killed. When the budget runs out, the gateway logs how many records are still spooled and stops. Those records stay on the volume and are shipped after the next start.
Revocation still works during an outage
A second, minimal channel carries a signed deny-list of API keys, workspaces, sessions and credentials. Gateways poll it every 5 seconds (10 before 1.8.5) and alarm at five minutes rather than the bundle's hour.
Every entry on the list takes effect on every plane it applies to: a revoked
key and a suspended workspace are refused on /v1, /mcp and /a2a, and a
revoked credential is no longer used for model calls. From 1.9.0 each refusal
is also in the audit trail on every plane; see
audit events.
From 1.8.5 one session kill covers all three planes. A session is a workspace
and an X-Session-Id, and an agent sends the same ID to models, tools and
peers. Every plane reads every kill on the list, so a kill stops the session's
model calls, tool calls and A2A hops. A kill stays on the list, and in
Flightdeck, for 24 hours (dvara.internal.deny-list.session-kill-retention).
Before 1.8.5 a kill left the list after 75 minutes, and a killed agent that kept
retrying was served again after that.
Gateways in one cluster also pass kills to each other directly, within
milliseconds. The relay can only add kills, never lift one, and nothing on the
request path waits for it. The revocation list stays the authority, so a kill
still arrives when the cluster is absent or split. /actuator/gateway-status
reports the cluster size in its sessionKills section: a fleet whose gateways
each report 1 is not relaying. Where multicast is blocked, list the gateways
in dvara.cluster.members; see
session-kill settings.
It never hard-refuses: a pod that cannot reach the channel keeps serving and keeps the revocations it already holds. Only revocations issued during the outage are delayed.
deny-list-url defaults to the same Flightdeck as the bundle. That is
convenient, and it gives up the independence the channel exists for. Point it
at a mirror if that guarantee matters to you.
What this means for operators
- The pod's configuration is read-only. Make changes in DVARA Flightdeck; the next verified bundle carries them to the pod.
- A bootstrap file belongs on Flightdeck. Set
DVARA_BOOTSTRAP_FILEon Flightdeck, which seeds the database the bundle is published from. - A session reaches Flightdeck a few seconds after the gateway serves it. From 1.8.3 a Flightdeck kill that names its workspace, as every Flightdeck kill button does, works in that window too: it is saved, audited, and published on the revocation list, so the gateway refuses the session within one poll.
- A kill stops a session, not an agent. An agent can start a new session ID. To stop an agent, revoke its API key.
On Kubernetes
The chart runs this setup with no values to set:
- It turns enrolment on and generates the shared secret on install
(
secrets.enrolmentSharedSecret), keeping it across upgrades. Tools that render without a cluster, such ashelm templateor Argo CD, cannot read it back: set it explicitly there. Withsecrets.existingSecret, that Secret must carry the keyenrolment-shared-secret. - Flightdeck serves enrolment and
/internalon port 8443 through a separate-internalService.dataPlane.controlPlane.baseUrldefaults to it; set it only when Flightdeck runs elsewhere. - The gateway always has a data volume at
llmGatewayServer.persistence.mountPath(/var/lib/dvara). By default it is anemptyDir: it survives a container restart but not the pod's deletion, and a new pod enrols again.
llmGatewayServer.persistence.enabled: true turns the workload into a
StatefulSet, because a Deployment cannot give each replica its own read-write
volume. That is a change of kind: updates become ordered, pods get stable
ordinals, and zonal volumes pin a pod to its zone. It is recommended for
production and off by default.
PostgreSQL is supplied to Flightdeck only, as SPRING_DATASOURCE_* through
flightdeck.extraEnv. See Deploy on Kubernetes.
Three availability questions, answered three different ways
A gateway can lose contact with Flightdeck. What it should do then looks like one question and is three, and they are deliberately not answered alike:
| when the pod cannot check… | it… | because |
|---|---|---|
| a budget | serves | Enforcement must fail closed on evidence of a breach, not on the absence of evidence. An overshoot is money, it is visible afterwards in the ledger, and it reconciles. |
| a revocation | serves, briefly | A revocation is late, not absent: the deny-list is a second channel with its own short poll, and it lands when contact resumes. The window is bounded and alarmed. |
| PII tokenization | refuses by default | A workspace asked for reversible TOKENIZE, but the pod cannot mint a token. It applies pii.degraded-action, whose default is BLOCK, rather than silently sending the raw value. Irreversible REDACT does not depend on this path. |
Reversibility is the whole rule. The first two failures can be undone or
paid for after the fact; the third cannot. So the third is the one place DVARA
fails closed, and serving instead takes an explicit per-workspace opt-in
(pii.degraded-action). An unrecognised value for that setting resolves to
BLOCK, so a typo cannot open the door.
What moves on upgrade
Moving from 1.8.1 or earlier to 1.8.2 or later:
- Remove the database settings from the gateway. A gateway given
SPRING_DATASOURCE_URL,DVARA_BOOTSTRAP_FILE,GATEWAY_BOOTSTRAP_FILEordvara.data-plane.serve-from-bundle=falserefuses to start. The database pool, socket-timeout and migration variables are no longer read on the gateway. - Give the gateway Flightdeck's address, the enrolment secret and a volume for its spool, as shown above. Move a bootstrap file to Flightdeck.
- Helm:
dataPlane.serveFromBundleanddataPlane.dbMigratewere removed. A values file that still sets either, or that gives the gatewaySPRING_DATASOURCE_*orDVARA_BOOTSTRAP_FILEthroughllmGatewayServer.extraEnv, failshelm upgradewith the setting named.dataPlane.controlPlane.publicKeyis now optional. - Removed with the database: the config cache in front of the database and its stale-read fallback (
dvara.llm-gateway.caching.distributed.enabled,dvara.llm-gateway.stale-reads.*, thegateway_config_stale_secondsmetric), and the shadow comparison of each bundle against the database (thegateway_bundle_shadow_divergence_totalmetric).
What this does not do
/internal/ingest, /internal/config/bundle, /internal/deny-list,
/internal/pii/dek/* (including erase), /internal/batch, and
/internal/approvals/* are pod-to-Flightdeck paths. They require the pod's
certificate and are not a public API. Keep them behind an internal listener or
firewall rule.
- It does not remove PostgreSQL. DVARA Flightdeck still needs it and is the only database client.
- It does not change what the gateway serves, or how governance behaves.