Skip to main content

Self-Hosted LLM Gateway: Governance That Runs Inside Your Perimeter

· 5 min read
DVARA Team
DVARA Team

For a lot of teams, the deciding question about an AI governance platform isn't what it does — it's where it runs. If your prompts contain patient records, financial data, source code, or anything a regulator cares about, routing them through a third party's servers is a compliance conversation you may not be allowed to have.

Self-hosting turns that from a data-egress problem into a governance guarantee. When the DVARA LLM Gateway runs inside your own infrastructure, the governance that matters most — Policy-as-Code evaluation, PII redaction, and the tamper-evident audit trail — happens before a request ever leaves your perimeter. A policy-denied or PII-blocked request never egresses at all. Self-hosting isn't just a deployment preference; it's what makes the governance enforceable on sovereign data.

What "self-hosted" means here​

A self-hosted deployment is one you run inside your own environment — your VPC, your Kubernetes cluster, your on-prem data center — rather than consuming it as a vendor-operated SaaS. Requests from your applications hit your instance. The Gateway still calls out to model providers (unless you're running fully local models), but it does so from inside your perimeter, under your network controls, with your keys, after your governance has run.

A quick honesty note: self-hosted is not the same as open-source. DVARA is a commercial, licensed platform you run entirely in your own infrastructure — you own the deployment and the data path without owning the source. The property that matters for sovereignty is where governance runs, not who wrote it.

What self-hosting actually buys you​

Ordered governance-first, because that's the reason to do it:

  • Governance runs before egress. The most sensitive thing you send a model is often the prompt itself. Self-hosting means the request is policy-checked, PII-scanned, and logged before it leaves your network — and a blocked request never leaves at all.
  • Audit stays in your control. The immutable, HMAC-signed audit trail lives in your database, not a vendor's. No third party is in your audit scope.
  • Data residency you control. You decide which region the platform runs in and which providers it may reach — how you keep EU traffic on EU infrastructure. (This is the bridge to the broader AI compliance and sovereignty story.)
  • Your keys, your vault. Provider credentials live in your secret store, not a vendor's.
  • Air-gap-friendly. Paired with local models, it can run with no outbound internet at all.

BYOK: bring your own keys​

A serious self-hosted platform is BYOK — bring your own keys. Each tenant's provider credentials are theirs; the Gateway resolves the right key per request and never forces everyone through one shared secret. Credentials can be stored encrypted at rest, or — better — as references into a vault (HashiCorp Vault, AWS Secrets Manager, Azure Key Vault), so the secret material never lands in the platform's own database at all. Rotate a key, and in-flight resolution picks up the new one without a redeploy.

For the strictest setups, strict BYOK refuses to let a tenant borrow a shared platform key at all — no own credential, no call — so you can guarantee cross-tenant key isolation as a governed policy.

What self-hosting doesn't do for free​

Self-hosting is a governance-and-data-path guarantee, not a magic wand:

  • You operate it. That's a PostgreSQL and a couple of stateless pods, with the usual monitoring — modest, but yours to run.
  • You still call external providers unless you run local models; self-hosting the platform doesn't self-host OpenAI.
  • It's a licensing/operating trade, not necessarily a cost win — you trade a vendor's per-call margin for your own infra and ops time.

For most regulated buyers, those are easy trades against the alternative of not being allowed to use the product at all.

An evaluation checklist​

  • Does it deploy into your VPC / cluster cleanly (containers, a Helm chart)?
  • Does governance run before egress — policy and PII enforced before the request leaves your network?
  • Does the audit trail stay in your database, outside any vendor's scope?
  • Is it BYOK, with per-tenant credential isolation (and a strict mode)?
  • Can credentials live in your vault as references, not just encrypted in its DB?
  • Can you pin it to a region and restrict which providers it may reach?

Where DVARA fits​

DVARA is an AI governance platform built to be self-hosted: it deploys into your own infrastructure as containers or a Helm chart, runs Policy-as-Code, PII, and audit governance before a request leaves your network, keeps the immutable audit trail in your own database, and is BYOK with per-tenant credential isolation and vault-referenced secrets. You control its region and provider reach. For the full regulatory picture, read the AI Compliance & Sovereignty guide, or the pillar, What Is an LLM Gateway?, for how self-hosting fits the platform.