PII Detection and Redaction

DVARA detects and enforces policies on Personally Identifiable Information (PII) and Protected Health Information (PHI) in prompts and LLM responses. This prevents sensitive data from being forwarded to external LLM providers without explicit authorization.

How It Works

PII enforcement runs at two points in the LLM request lifecycle:

1. Request scanning — After policy evaluation, before dispatching to the LLM provider. Detects PII in user messages and tool result blocks.
2. Response scanning — After receiving the LLM response, before returning to the client. Detects PII in assistant messages (output leak detection).

Additionally, PII is always stripped from requests before they are written to the response cache.

For MCP (Model Context Protocol) traffic, the DVARA MCP Proxy runs the same PII enforcement against tool calls:

1. Argument scanning — For tools/call operations, the proxy recursively scans every string value in the tool-call arguments before forwarding to the upstream MCP server.
2. Response scanning — After the MCP server responds (2xx only), the proxy recursively scans every string value in the response body for PII output leaks before returning to the agent.

MCP PII scanning uses the same configuration (global dvara.llm-gateway.pii.* and per-tenant pii.* metadata) and the same detection engine as LLM PII scanning.

Supported PII Types

Type	Label	Detection Method
Email addresses	email	Regex pattern
US phone numbers	phone_us	Regex (with optional +1)
International phone numbers	phone_intl	Regex (E.164 format)
Social Security Numbers	ssn	Regex (rejects 000/666/9xx prefixes)
Credit card numbers	credit_card	Regex + Luhn checksum validation
Dates of birth	dob	Regex (MM/DD/YYYY or MM-DD-YYYY)
IPv4 addresses	ipv4	Regex
US passport numbers	passport_us	Regex
Driver's license numbers	drivers_license	Keyword (driver's license) + alphanumeric ID
IBAN numbers	iban	Regex (international bank account)
Medical Record Numbers	mrn	Keyword (MRN/Medical Record) + digits
DEA numbers	dea	Format regex + DEA checksum validation
NPI numbers	npi	Context keyword "NPI" required + Luhn checksum
Person names	person_name	Salutation heuristic (Mr/Mrs/Dr/Prof), confidence 0.7

Actions

Each PII detection can trigger one of three actions:

Action	Behavior	Audit Event
LOG	Log the detection, forward request unchanged	PII_DETECTED
BLOCK	Reject the request with HTTP 400, type: pii_violation, code: pii_detected	PII_DETECTED
REDACT	Replace PII with reversible tokens, forward modified request	PII_REDACTED

When REDACT is active, PII values are replaced with tokens like {{PII_EMAIL_a1b2c3d4}}. The original values are encrypted (AES-256-GCM) and stored in-memory per tenant. Authorized admins can detokenize the content later via the admin API.

Response scanning emits a PII_OUTPUT_LEAK audit event on every detection. The response body is rewritten with tokenized placeholders only when the tenant action is REDACT; on LOG the response is returned unchanged, and BLOCK is not applied to response scanning (the audit event is your signal).

Configuration

Global Configuration

Add to application.yml:

dvara:
  llm-gateway:
    pii:
      enabled: true                                          # enable PII scanning
      provider: regex                                        # regex (default) or presidio
      default-action: LOG                                    # LOG, BLOCK, or REDACT
      scan-responses: true                                   # scan LLM responses for output leaks
      strip-before-cache: true                               # always redact before caching
      token-encryption-password: ${DVARA_LLM_GATEWAY_PII_TOKEN_ENCRYPTION_PASSWORD:}
      max-tokens-per-tenant: 50000                           # max stored PII tokens per tenant
      token-retention-days: 30                               # auto-expire after N days

Embedded Phileas Scanner (In-Process, No Third-Party Required)

For broader filter coverage without deploying a sidecar, enable the embedded Phileas scanner (Apache 2.0). Phileas is a Java library that runs entirely in-process with sub-millisecond per-call latency, no network calls, no API keys, and no external service to operate.

dvara:
  llm-gateway:
    guardrail:
      phileas:
        enabled: true     # opt-in

That's the only configuration required. When enabled, the Phileas scanner runs alongside the built-in regex detector and their results are merged. Default policy enables 22 filter types:

Category	Filters
Government IDs	SSN, PASSPORT_NUMBER, DRIVERS_LICENSE_NUMBER
Financial	CREDIT_CARD, IBAN_CODE, BANK_ROUTING_NUMBER, BITCOIN_ADDRESS, CURRENCY
Contact	PHONE_NUMBER, PHONE_NUMBER_EXTENSION, EMAIL_ADDRESS
Network / device	IP_ADDRESS, MAC_ADDRESS, URL
Addresses	STREET_ADDRESS, ZIP_CODE, STATE_ABBREVIATION
Dates & demographics	DATE, AGE
Vehicle / shipping	VIN, TRACKING_NUMBER
PHI	PHYSICIAN_NAME

Eight additional filters ship in the catalog but are opt-in per tenant because they match against dictionaries (names, cities, counties, states, hospitals, medical conditions) that collide with ordinary English on free-form chat workloads — too many false positives to auto-enable. To switch them on, list them explicitly in phileas.enabled-filters for the tenant: FIRST_NAME, SURNAME, LOCATION_CITY, LOCATION_STATE, LOCATION_COUNTY, HOSPITAL, MEDICAL_CONDITION, IDENTIFIER (the last over-matches long digit sequences without dictionary tuning).

Per-tenant filter restriction is editable from the Phileas Filters tab in the DVARA Flightdeck tenant form, or by setting the tenant metadata key phileas.enabled-filters to a comma-separated list of filter-type names.

Limitations. Phileas is best-effort pattern matching with no Luhn validation. It can mis-classify long digit sequences (e.g. classify a credit card number as a drivers license number depending on filter ordering). For production credit card and NPI detection, rely on the always-on built-in regex detector (which Luhn-validates). Use Phileas as an additive layer for the broader filter coverage it provides (MAC address, IBAN, bitcoin, VIN, etc.) that the curated regex set doesn't include.

Both the built-in regex detector and the Phileas scanner run when Phileas is enabled; detections are merged with the higher-confidence result winning on overlap. Rule labels differ between the two (regex:ssn vs phileas:ssn), so you may still see two audit events for the same match. To suppress duplicates, restrict the Phileas filter set per tenant to only the types not covered by the regex detector (e.g. MAC address, IBAN, bitcoin, VIN).

Presidio Integration (NER-Based Detection)

For industrial-grade PII detection using NER models, deploy Microsoft Presidio as a sidecar and configure:

dvara:
  llm-gateway:
    pii:
      provider: presidio
      presidio:
        endpoint: http://presidio-analyzer:3000/analyze      # Presidio analyzer URL
        language: en                                         # analysis language
        score-threshold: 0.5                                 # minimum confidence
        timeout-seconds: 5                                   # HTTP timeout
        cache-max-size: 1000                                 # LRU cache entries (0 = disabled)
        cache-ttl-seconds: 300                               # cache TTL

When provider=presidio, the gateway merges results from both the built-in regex detector (14 patterns + checksum validation) and Presidio (NER-based PERSON, LOCATION, ORGANIZATION detection). Overlapping detections are deduplicated, keeping the higher-confidence result.

Property	Default	Description
dvara.llm-gateway.pii.provider	regex	Detection provider: regex or presidio
dvara.llm-gateway.pii.presidio.endpoint	—	Presidio analyzer endpoint URL
dvara.llm-gateway.pii.presidio.language	en	Presidio analysis language
dvara.llm-gateway.pii.presidio.score-threshold	0.5	Minimum Presidio confidence score
dvara.llm-gateway.pii.presidio.timeout-seconds	5	HTTP call timeout
dvara.llm-gateway.pii.presidio.cache-max-size	1000	LRU cache max entries (0 = disabled)
dvara.llm-gateway.pii.presidio.cache-ttl-seconds	300	Cache entry TTL

Presidio is fail-open: if the sidecar is unreachable, detection falls back to regex-only results. Cached results use SHA-256 of the input text as cache key; delegate failures are cached with a 30-second TTL to prevent thundering herd.

Per-Tenant Configuration

The fastest way to override PII behavior for a tenant is the PII & DLP tab in the DVARA Flightdeck tenant form (Tenants → Edit). Form-based controls cover pii.enabled, pii.action, and pii.scan-responses. Updates emit a TENANT_METADATA_UPDATED audit event with a diff.

For Terraform, CI/CD, and other programmatic tooling, set the same metadata keys via the Automation API:

curl -X PUT http://localhost:8090/v1/admin/tenants/acme-corp \
  -H "Content-Type: application/json" \
  -d '{
    "metadata": {
      "pii.enabled": "true",
      "pii.action": "REDACT",
      "pii.scan-responses": "true",
      "pii.custom-patterns": {
        "employee_id": "EMP-\\d{6}",
        "internal_project": "PROJ-[A-Z]{2,4}-\\d{4}"
      }
    }
  }'

pii.custom-patterns is currently API-only — the UI form does not yet expose a custom-pattern editor.

pii.custom-patterns must be a nested object, not a stringified JSON value

The value of pii.custom-patterns must be a JSON object (label → regex map), as shown above. Passing it as a stringified JSON value (e.g. "pii.custom-patterns": "{\"employee_id\": \"EMP-\\d{6}\"}") silently fails — the gateway's tenant-config resolver checks the value with instanceof Map, the string fails the check, custom patterns are dropped, and no error is logged. The tenant simply gets zero custom-pattern matches.

Metadata Key	Values	Description
pii.enabled	true / false	Override global PII detection
pii.action	BLOCK / REDACT / LOG	Override default action
pii.scan-responses	true / false	Override response scanning
pii.custom-patterns	JSON object (label → regex)	Add custom regex patterns. Must be a nested object, not a stringified JSON value.

Custom Patterns

Custom patterns are specified as a JSON object where keys are labels and values are regex strings. They are detected as CUSTOM entity type:

{
  "employee_id": "EMP-\\d{6}",
  "internal_project": "PROJ-[A-Z]{2,4}-\\d{4}"
}

Admin API

Detokenize

Restore original PII values from redacted text:

curl -X POST http://localhost:8090/v1/admin/pii/detokenize \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Contact {{PII_EMAIL_a1b2c3d4}} about account",
    "tenant_id": "acme-corp"
  }'

Response:

{
  "text": "Contact user@example.com about account"
}

Requires owner or policy-admin role.

Purge Tokens

Remove all stored PII tokens for a tenant (irreversible):

curl -X DELETE http://localhost:8090/v1/admin/pii/tokens/acme-corp

Response:

{
  "tenant_id": "acme-corp",
  "tokens_removed": 1542
}

Requires owner role.

Audit Trail

All PII events are written to the audit trail. The audit payload includes entity types and counts but never includes the actual PII values.

LLM Traffic

Event Type	When
PII_DETECTED	PII found in request (action: LOG or BLOCK)
PII_REDACTED	PII redacted from request (action: REDACT)
PII_OUTPUT_LEAK	PII detected in LLM response

MCP Traffic

Event Type	When
MCP_PII_DETECTED	PII found in tool call arguments (action: LOG or BLOCK)
MCP_PII_REDACTED	PII redacted from tool call arguments (action: REDACT)
MCP_PII_OUTPUT_LEAK	PII detected in MCP server response

MCP PII audit events include server_id, tool_name, entity_count, entity_types, source (request / response), action, and trace_id — enough for auditors to filter PII activity by tool, by tenant, or by direction, and to correlate against the gateway trace for the originating LLM turn.

Example audit event payload (LLM PII path — the MCP path adds server_id / tool_name / action on top):

{
  "eventType": "PII_REDACTED",
  "tenantId": "acme-corp",
  "payload": {
    "source": "request",
    "entity_count": 2,
    "entity_types": "EMAIL, SSN",
    "entity_type_counts": {"EMAIL": 1, "SSN": 1}
  }
}

A few notes on the wire format that aren't obvious from a single example:

• tenantId is on the envelope, not in payload. Filter audit-event queries by the envelope field, not by a payload field.
• entity_types is a comma-separated string (Java List.toString().joining(", ")), not a JSON array. Receivers parsing as an array would fail.
• entity_type_counts is the per-type breakdown — useful for dashboards that want to plot email vs ssn vs credit_card over time without re-parsing the string above.
• LLM PII payloads do not include an action field — the event type itself (PII_DETECTED vs PII_REDACTED) carries that signal. The MCP PII path does include action since the same MCP_PII_DETECTED event type is used for both LOG and BLOCK actions.

Access control

PII admin operations are gated by platform role — there are no granular PII permission scopes; access follows DVARA's standard six-role model.

Operation	Roles
Detokenize (POST /v1/admin/pii/detokenize)	owner, policy-admin
Purge a tenant's PII tokens (DELETE /v1/admin/pii/tokens/{tenantId})	owner
Configure per-tenant PII settings	owner / policy-admin (Console); tenant admin (Portal → Data Protection)
View PII audit events	any admin role (results scoped to the caller's tenant)

Streaming PII Enforcement

When stream=true, the gateway wraps the SSE chunk iterator with buffered scanning that detects and redacts PII in-flight before forwarding to the client.

How it works:

1. Text deltas accumulate in a rolling buffer
2. When the buffer exceeds streaming-scan-window-size (default 256 chars), the safe region is scanned
3. An overlap margin (default 64 chars) catches PII spanning chunk boundaries
4. REDACT: PII tokens replaced in buffered text before forwarding
5. BLOCK: Stream terminates with finishReason=content_filter
6. LOG: Stream continues, summary audit event at end

Configuration:

Property	Default	Description
dvara.llm-gateway.pii.scan-streaming-responses	true	Enable PII scanning on streaming responses
dvara.llm-gateway.pii.streaming-scan-window-size	256	Chars buffered before scan trigger
dvara.llm-gateway.pii.streaming-overlap-margin	64	Chars retained between windows for boundary detection

Per-tenant override: set pii.scan-streaming-responses in the tenant metadata.

Security Considerations

• PII tokens are encrypted at rest using AES-256-GCM with the configured token-encryption-password
• Token storage is in-memory (not persisted to disk) — tokens are lost on gateway restart
• Audit events never contain the actual PII values, only entity types and counts
• The BLOCK action rejects the entire request — no partial content is forwarded
• Cache stripping always uses REDACT behavior regardless of the tenant's configured action