Blog

Ask a team "which models is your application allowed to call, and under what conditions?" and the honest answer is usually "let me check the code." The rules — which models are approved, which tools an agent may invoke, what happens when a request is too large or comes from the wrong region — are scattered across if statements in a dozen services. No one can review them in one place, no one can test a change safely, and no one can say what a rule would have done before it ships.

LLM policy as code fixes that the same way infrastructure as code fixed server configuration: move the rules out of application code and into a declarative, version-controlled language that a single control point enforces on every call.

DVARA 1.0.0 is generally available today. We are launching an AI governance platform — the LLM Gateway and the MCP Proxy as components — for engineering teams running real production traffic and dealing with real auditors, real compliance officers, and real cost lines on a real budget.

This post covers why we built it, what is in 1.0.0-GA, and how to run it in your own infrastructure today.

In the previous post, you got DVARA running and sent requests to multiple providers through a single OpenAI SDK client. This post covers the next layer of governance platform capabilities — LLM streaming, structured JSON outputs, Prometheus observability, and local development with Ollama.

DVARA is an AI governance platform. It governs every LLM call your teams make — policy, PII, budgets, audit — under one control plane, and the LLM Gateway component is fully OpenAI-compatible so governance kicks in on the very first request. Every OpenAI SDK — Python, Node, Go, Java, Rust — lets you override the base URL. That single line of configuration is all it takes to route your LLM traffic through DVARA and unlock governance, multi-provider routing, and observability without touching your application code.

Organizations investing in AI governance typically focus on what goes into the LLM and what comes out — prompt injection detection, PII redaction in chat messages, content policy filtering on responses. That's necessary, but it covers only half the attack surface.

AI agents are moving from demos to production. Engineering teams are deploying autonomous systems built on LangChain, CrewAI, AutoGen, and custom agent frameworks that call external tools, query databases, send emails, and modify infrastructure — all without a human in the loop.

Your API gateway handles TLS termination, rate limiting, and request routing. It does these things well. But when your traffic carries prompts, tokens, tool calls, and model-specific payloads, an API gateway becomes a passthrough — it can see the HTTP envelope but not the AI semantics inside it.