What fails open and what fails closed

A gateway on the request path is in the way of every model call. So every architecture review asks the same question about each thing the gateway depends on: when it is down, does the request go through or is it refused?

The answer is not one switch. It is decided control by control, and the rule behind the decisions is this:

• Enforcement fails closed on evidence of a problem, not on the absence of evidence. A budget store that cannot be read is not evidence that the budget is spent, so the request goes through. A control you switched on and that cannot run at all is evidence that the answer would be unchecked, so the request is refused.
• Where a mistake cannot be undone, it fails closed. Personal data sent to a provider cannot be recalled, so tokenization that cannot reach its keys does not quietly send the raw value. A missing audit record cannot be written later, so a full audit volume refuses calls.
• Where it can be reconciled later, it fails open, and says so. A rate limit or a budget that overshoots during an outage is counted and corrected afterwards. Each fail-open is counted in a metric or logged, so you can see when it happened.

The table below is that rule applied to each dependency, as the code does it today.

The table

Open means the request is served. Closed means it is refused. "Both" in the distribution column means DVARA Open Source and the Enterprise platform. A "workspace setting" is set per workspace in Flightdeck; the others are gateway properties.

Request controls

Control or dependency	Distribution	When it fails or cannot answer	Default	Setting that changes it	What records it
Pattern guardrails and PII patterns	Both	They run inside the gateway with no outside dependency, so there is nothing to fail	—	—	—
ML injection classifier (any provider, including the bundled one)	Enterprise	The check cannot run: a timeout, an error, an answer that cannot be read, or a classifier that is enabled and cannot run. From 1.10.0 a failed answer is not cached, so the next request asks again	Open	From 1.10.0: dvara.llm-gateway.guardrail.ml-classifier.fail-mode=CLOSED refuses with guardrail_plugin_error (500) for a workspace that enforces — every workspace except one whose guardrail action is FLAG or LOG	From 1.10.0: CONTROL_UNAVAILABLE in the audit record (at most one a minute per workspace) and gateway_control_unavailable_total{control, outcome}
Semantic guard	Enterprise	When no embedding can be made	Open	dvara.llm-gateway.guardrail.semantic-guard.fail-mode=CLOSED refuses with guardrail_plugin_error (500) for a workspace that enforces	gateway_guardrail_semantic_unavailable_total{outcome}
Guardrail plugins (your HTTP service)	Enterprise	Timeout or error from the plugin	Open	Per plugin: dvara.llm-gateway.guardrail.plugins.definitions[].fail-mode=CLOSED refuses the request	A WARN log
Presidio (named-entity PII)	Enterprise	The pattern detectors still run; names, places and other entities only Presidio finds are not found. A failed call is not cached	Open	From 1.10.0: dvara.llm-gateway.pii.presidio.fail-mode=CLOSED refuses with guardrail_plugin_error (500), whatever the workspace's PII action	From 1.10.0: CONTROL_UNAVAILABLE and gateway_control_unavailable_total
A model file that does not match its pin	Enterprise	From 1.10.0, for the injection classifier, the semantic-cache embedding model and the routing classifier: a different, missing or malformed pin stops the start; a file replaced while running is refused when it loads	Closed	dvara.llm-gateway.guardrail.ml-classifier.defender-model-sha256, dvara.llm-gateway.cache.semantic.embedding.model-sha256, dvara.llm-gateway.routing.intelligent.onnx.model-sha256; with no pin a model from a configured directory loads and its SHA-256 is logged, ready to pin	The start fails with the mismatch; each classifier detection records model_sha256
PII TOKENIZE, when the keys cannot be reached	Enterprise	The token cannot be minted	Closed — the raw value is never sent instead	The workspace setting pii.degraded-action (BLOCK by default)	The PII audit events
Policy engine, a policy that does not compile	Both	That one policy is skipped and named; the others still apply	—	Fix the policy	A compile-failure count on the reload, and the policy named in the log
Policy DENY rule whose context is missing	Enterprise	The rule cannot tell whether it matches	Closed	—	The policy decision in the audit record
MCP tool policy, A2A hop policy	Enterprise	The policy cannot be evaluated	Closed (policy_error, A2A_POLICY_ERROR)	—	The denied call in the audit record
Grounding switched on, with no detector in the build	Both	Requests that carry sources are refused before the provider is called	Closed (grounding_unavailable, 403)	Switch grounding off	The refusal
Grounding, an embedding fails	Enterprise	The claim counts as unverified, not grounded	Closed under BLOCK; recorded under LOG/FLAG	The workspace's grounding.action	gateway_grounding_embedding_failures_total, HALLUCINATION_DETECTED
Grounding, the request carries no sources	Enterprise	Nothing to check the reply against, so it is not checked	Open	Send sources with the request	From 1.10.0: gateway_grounding_unchecked_total, a WARN once per workspace, and X-Gateway-Grounding: unchecked
A streamed reply whose control cannot run	Both	The stream is refused before any of it is delivered	Closed	—	The refusal

Limits, budgets and quotas

Control or dependency	Distribution	When it fails or cannot answer	Default	Setting that changes it	What records it
In-process rate limiter	Both	Runs inside the gateway; counts per pod	—	—	—
Shared rate limiter on Valkey or Redis	Enterprise	The store is unreachable	Open	dvara.llm-gateway.rate-limit.shared-cache.failure-mode=CLOSED refuses instead	A log line
Fleet rate limiter on the embedded cluster	Enterprise	A member cannot answer an admission check	Open	None	gateway_rate_limit_fail_open_total{reason} and a WARN at most once a minute
Budget caps	Enterprise	The budget cannot be read	Open — an outage is not evidence the budget is spent	The workspace setting budget.cold-start-posture=closed means "never exceed, even at the cost of availability"	A log line; SPEND_COUNTER_SEEDED records how a cold-started counter was seeded
Plan quotas (calls, MCP calls, A2A hops)	Enterprise	A gateway that cannot count	Open	None	A WARN once per plane per process
Plan quotas, a tier name the catalogue does not know	Enterprise	Every quota control for that workspace	Open	Fix the tier name	gateway_tier_unresolved_ids

Flightdeck, configuration and audit

Control or dependency	Distribution	When it fails or cannot answer	Default	Setting that changes it	What records it
Flightdeck unreachable: configuration	Enterprise	The gateway keeps serving the last configuration it verified, and restarts from its saved copy	Open on the last known good	dvara.data-plane.control-plane.snapshot-path keeps a copy across restarts	configBundle.stale on /actuator/gateway-status after dvara.data-plane.control-plane.staleness-warn-seconds (1 hour)
Flightdeck unreachable: revocations	Enterprise	Revocations the gateway already holds stay in force, across its own restart when a snapshot path is set. One issued during the outage applies when Flightdeck is back	Open for new revocations	—	denyList.stale on /actuator/gateway-status
Flightdeck unreachable: audit records	Enterprise	Records wait in the gateway's spool and are sent when Flightdeck is back. Audit is never dropped; usage and cost rows can be dropped first if the spool fills	—	dvara.llm-gateway.local-store.* sizes	gateway_spool_shed_total{class}, gateway_drain_lag_seconds
The audit volume is full, or an audit write fails	Enterprise	/v1, /mcp and /a2a calls are refused with 503 audit_store_full and the pod reports not ready	Closed	dvara.llm-gateway.local-store.audit-reserve-bytes (64 MiB)	Readiness OUT_OF_SERVICE
SIEM export	Enterprise	Retried with back-off for about two and a half minutes, in order	—	—	gateway_siem_export_total
Licence expiry	Enterprise	Never touches traffic	—	—	The licence bar in Flightdeck

Providers

Control or dependency	Distribution	When it fails or cannot answer	Default	Setting that changes it	What records it
A model provider errors or times out	Both	Retried, then a fallback provider if the route has one; the circuit breaker stops calling a failing provider for a while	—	Retry, circuit breaker and fallback settings (resilience)	Retry, fallback and circuit metrics
Failover would leave the workspace's region	Enterprise	From 1.10.0 the fallback is not used	Closed (strict)	The workspace setting data-residency.failover=permissive	DATA_RESIDENCY_FAILOVER_BLOCKED or DATA_RESIDENCY_FAILOVER_ELSEWHERE

What is not there yet

This page describes the product as it is, so it also says where the answer is "not yet":

• Only some fail-opens are in the audit record. From 1.10.0 the ML classifier and Presidio write CONTROL_UNAVAILABLE when they cannot run. The rate limiters, budgets and quotas still show a fail-open only as a metric or a log line.
• Presidio's CLOSED cannot tell workspaces apart. It refuses whatever the workspace's PII action, because the PII scan is not told which workspace it is for.
• Grounding cannot check a reply that comes with no sources. See What fails open above and Ground responses in supplied sources.

Related

• What the gateway does not govern — the traffic that never reaches any of these controls.
• OWASP LLM & MCP Top 10 crosswalk
• Resilience and failover