Ground responses in supplied sources

Enterprise only

Not included in DVARA Open Source.

The grounding detector is off by default. DVARA Open Source exposes the configuration seam but does not include the detector.

When a response is supposed to be grounded in a set of source documents — a RAG answer, a summary, a citation — DVARA can catch claims that aren't actually supported by the sources. The detector splits the response into sentence-level claims, embeds each claim and each source passage, and flags any claim whose maximum cosine similarity to any source falls below a configurable threshold.

Grounding runs as a response-side check on non-streaming calls and as an at-stream-end check on SSE responses.

Follow one grounding decision

The two unavailable states behave differently. If grounding is enabled and the request includes usable sources but the running distribution has no detector, DVARA refuses the response with HTTP 403 grounding_unavailable.

If the Enterprise detector is active and an embedding fails, the check is not grounded. From 1.8.4 a failed embedding is never a grounded verdict. An error, a timeout, or an empty or all-zero vector counts as a failure. Each claim that could not be compared is reported as ungrounded and marked [not verified: embedding failed]. When a source fails, that is every claim; when one claim fails, that claim. The workspace action then applies as for any ungrounded answer:

• BLOCK refuses the response with hallucination_detected before delivery, on streaming and non-streaming requests.
• LOG and FLAG deliver the response and write the audit event with grounded: false.

The check counts under grounded="false", and gateway_grounding_embedding_failures_total counts the failure by stage. The audit event does not yet tell an embedding failure apart from an unsupported claim, so use that metric to find embedding outages. Up to 1.8.3 the check failed open: a failed embedding passed the response as grounded, even under BLOCK.

The sentence splitter applies a 5-word minimum per claim — sentences shorter than 5 words (after whitespace split) are filtered out and never embedded. So one-word answers like "Yes." / "False." / "It is." or short list bullets are skipped silently and the response is treated as grounded. The detector is designed for long-form responses; if your workload needs grounding on short-token outputs, gate it differently (e.g. structured-output schemas with required citation fields).

Send the sources with the request

The detector looks for sources in request.metadata["grounding.sources"] as a list of strings. Your client supplies them on each chat-completion request:

curl -s -X POST http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer $DVARA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "gpt-4o",
  "messages": [
    { "role": "system", "content": "Answer based only on the provided context." },
    { "role": "user", "content": "What is DVARA's auth model?" }
  ],
  "metadata": {
    "grounding.sources": [
      "DVARA supports three auth modes: built-in, OIDC, and SAML 2.0.",
      "The data plane always requires API keys."
    ]
  }
}'

A successful call returns the normal OpenAI-compatible response. If the detector finds an unsupported claim under the default LOG action, the response still reaches the caller and Flightdeck records HALLUCINATION_DETECTED. Under BLOCK, the caller receives the 403 response shown below instead.

If metadata.grounding.sources is missing or empty and grounding is enabled, the detector is a no-op — it doesn't fabricate sources for you.

Sources above dvara.llm-gateway.guardrail.grounding.max-source-length (default 10000 chars) are silently dropped. Lists above dvara.llm-gateway.guardrail.grounding.max-sources (default 50) have their tail dropped. This protects you from runaway embedding bills on pathological inputs.

Enable grounding deliberately

Set these in application.yml:

dvara:
  llm-gateway:
    guardrail:
      grounding:
        enabled: true
        similarity-threshold: 0.7    # cosine threshold; claims below this are ungrounded
        action: LOG                  # LOG, FLAG, or BLOCK
        max-sources: 50              # max source documents per request
        max-source-length: 10000     # max chars per source document

Or via environment variables — the gateway maps dvara.llm-gateway.guardrail.grounding.* YAML keys to DVARA_LLM_GATEWAY_GUARDRAIL_GROUNDING_* env vars automatically:

DVARA_LLM_GATEWAY_GUARDRAIL_GROUNDING_ENABLED=true
DVARA_LLM_GATEWAY_GUARDRAIL_GROUNDING_SIMILARITY_THRESHOLD=0.7
DVARA_LLM_GATEWAY_GUARDRAIL_GROUNDING_ACTION=LOG
DVARA_LLM_GATEWAY_GUARDRAIL_GROUNDING_MAX_SOURCES=50
DVARA_LLM_GATEWAY_GUARDRAIL_GROUNDING_MAX_SOURCE_LENGTH=10000

Property	Default	Description
enabled	false	Enable embedding-based grounding detection
similarity-threshold	0.7	Cosine similarity threshold — claims below this are ungrounded
action	LOG	LOG, FLAG, or BLOCK
max-sources	50	Maximum source documents per request (excess silently dropped)
max-source-length	10000	Maximum chars per source document (oversized silently dropped)

Choose what happens on a failed check

• LOG (default) — write a HALLUCINATION_DETECTED audit event and a Prometheus metric sample, then pass the response through unchanged. Use this in shadow mode while you tune the threshold.
• FLAG — behaviorally identical to LOG in the current release: the audit event and the metric sample are written, and the response is passed through unchanged. The value of setting FLAG is that the action label on the gateway_grounding_check_total counter reflects your intent, so dashboards can distinguish "running in shadow" from "running in warn-before-block" when you're staging a rollout. There is no X-Gateway-Grounding-Warning response header in this release — do not rely on a client-side header to drive UX.
• BLOCK — the original response bytes are discarded and the client receives a structured error (see Error code below for the wire format).

Start with LOG, measure the false-positive rate on real traffic, then promote to FLAG or BLOCK once you're confident.

Override the action for one workspace

The DVARA Flightdeck exposes per-workspace grounding settings on the Grounding tab of the workspace edit form (Identity → Workspaces → Edit). The form covers enabled, action, max-sources, and max-source-length with server-side validation. Updates emit a WORKSPACE_METADATA_UPDATED audit event with a diff of every changed key.

For Terraform, CI/CD and other programmatic tooling, the same overrides go through the Flightdeck's GitOps export/import as workspace metadata — export the config, edit the workspace's metadata object, and import it back:

{ "workspaces": [ { "id": "acme-corp", "metadata": {
      "grounding.enabled": "true",
      "grounding.action": "BLOCK",
      "grounding.max-sources": "20"
} } ] }

Metadata key	Type	Overrides
grounding.enabled	boolean	Enable / disable per workspace
grounding.action	BLOCK / FLAG / LOG	Action for this workspace
grounding.max-sources	int	Per-workspace source cap
grounding.max-source-length	int	Per-workspace source length cap

Missing keys fall back to the global config. If a stored action or numeric limit cannot be parsed, DVARA logs a warning and uses the global value instead of silently choosing a different workspace setting.

similarity-threshold is not overridable per workspace — it's bound at startup and applies to every workspace uniformly. Changing it requires a gateway restart. If different workspaces need different strictness, use per-workspace grounding.action (e.g. BLOCK for the strict workspace, LOG for the rest) rather than reaching for a per-workspace threshold.

Govern streaming responses

For streaming (stream=true) responses, DVARA evaluates grounding against the complete response. Delivery depends on the configured action:

• BLOCK — Deferred delivery holds the response until the check completes. A refused answer ends with finish_reason: content_filter and writes HALLUCINATION_DETECTED_STREAMING; no partial answer is released.
• LOG / FLAG — Immediate delivery relays events while DVARA retains a bounded copy for the final check and evidence.

If the upstream ends before a Deferred grounding check has a complete answer, DVARA refuses with STREAM_INCOMPLETE_UNGROUNDED. If Immediate accumulation reaches its bound, delivery continues and STREAMING_ENFORCEMENT_SUMMARY records scan_incomplete; the retained prefix is all that can be checked.

Verify the control with evidence

The grounding detector emits two Prometheus counters and two audit-event types. Hook these into your existing dashboards before turning the action up from LOG to BLOCK.

Measure grounding decisions

Metric	Type	Labels	Description
gateway_grounding_check_total	Counter	grounded (true/false), action (LOG/FLAG/BLOCK)	Incremented once per non-streaming response that runs through the detector.
gateway_grounding_embedding_failures_total	Counter	stage (source/claim)	From 1.8.4. Incremented when a source or a claim could not be embedded. The check it belongs to counts as grounded="false".

PromQL to dashboard the ungrounded rate — the percentage of responses where the detector flagged at least one ungrounded claim:

sum(rate(gateway_grounding_check_total{grounded="false"}[5m]))
/
sum(rate(gateway_grounding_check_total[5m]))

PromQL to alert on regressions — fire when ungrounded rate exceeds 10% over a 15-minute window:

sum(rate(gateway_grounding_check_total{grounded="false"}[15m]))
/
sum(rate(gateway_grounding_check_total[15m]))
> 0.1

PromQL to alert on embedding failures, which make every affected check ungrounded:

sum by (stage) (rate(gateway_grounding_embedding_failures_total[5m])) > 0

PromQL to track the rollout from LOG → FLAG → BLOCK — a stacked breakdown by action:

sum by (action) (rate(gateway_grounding_check_total[5m]))

Review audit events

An audit event is written only when the detector flags at least one ungrounded claim — fully grounded responses do not emit an event. Two event types are emitted depending on the response mode:

• HALLUCINATION_DETECTED — non-streaming responses.
• HALLUCINATION_DETECTED_STREAMING — streaming responses, written at stream end.

Envelope fields are the event ID, timestamp, workspaceId, event type, and payload. The trace ID is not part of this audit event.

Payload fields:

Field	Type	Description
grounded	boolean	Always false on these events — the event is only written when the check fails.
confidence	double	Detector's confidence in the grounding decision (0.0–1.0).
overall_similarity	double	Aggregate cosine similarity across all claims.
ungrounded_claim_count	int	Number of sentence claims that fell below the similarity threshold.
ungrounded_claim_hashes	list of strings	SHA-256 hashes of the flagged claims. The audit payload does not retain the claim text. Use the request's trace context and your own approved content-retention system when an investigation needs the original response.
source	string	Present only on HALLUCINATION_DETECTED_STREAMING, with value streaming_response. Distinguishes the streaming origin in log queries.

Handle a blocked or unavailable check

When action=BLOCK and the detector finds an ungrounded claim, the gateway returns HTTP 403 with type: guardrail_violation and code: hallucination_detected — the same envelope shape as the other guardrail enforcement codes (guardrail_blocked, pii_detected).

{
  "error": {
    "type": "guardrail_violation",
    "code": "hallucination_detected",
    "message": "Response contains ungrounded claims: The policy was approved in 2024.",
    "trace_id": "a6783439db1f46a6bfed511a0011e955"
  }
}

When the running distribution has no detector, the same HTTP 403 envelope uses code: grounding_unavailable and explains that an embedding-backed detector is not present. DVARA refuses this request before calling a provider, so the attempt does not incur provider usage. This is the expected Open Source result when grounding is enabled and the request supplies sources; disable grounding or run the Enterprise platform.

Tune the threshold with representative traffic

The default threshold (0.7) is a conservative starting point. Things to keep in mind:

• Claim tokenizer quality matters more than the embedding model. A bad sentence split can attribute unrelated content to a claim.
• Source passage length affects similarity scores — very long passages dilute similarity. Break sources into semantically coherent chunks (1–3 sentences each).
• Embedding model affects absolute similarity values. Grounding uses the same in-process embedding service as the semantic cache. The default simple-hash embedder is deterministic and catches close paraphrases but isn't semantically rich; switching the semantic cache to the neural onnx provider (all-MiniLM-L6-v2) makes grounding semantically richer too.
• False positives are most common on responses that paraphrase sources heavily or on lists where each bullet is short. Raise the threshold cautiously — below 0.5 is usually too permissive.

Continue the response-governance path

• Guardrails overview — the broader guardrail system and how grounding fits in
• ML & Plugin Guardrails — complementary input-side protections
• Configuration — full property reference