Run Your First Governed LLM Call with the OpenAI SDK

April 6, 2026 · 7 min read

Correction, 2026-08-28

This post was published against 1.0.0 and referred to the gateway image as ghcr.io/dvarahq/dvara/dvara-llm-gateway. The package was renamed to ghcr.io/dvarahq/dvara-gateway in 1.7.0, so the original command no longer pulls anything. The commands below have been updated to the current image and tag; the rest of the post is unchanged.

DVARA is an AI governance platform. Its LLM Gateway accepts the OpenAI API shape, so an existing SDK can use DVARA by changing the base URL. Your application keeps the same client while model requests gain policy, PII checks, budgets, audit, routing, and operational evidence.

This guide walks you through starting DVARA, sending your first governed request, switching providers, and verifying that everything works — all in under five minutes.

What you get by pointing your SDK at DVARA

Before diving into setup, here's what changes when your SDK talks to DVARA instead of directly to OpenAI:

Policy enforcement on every call — DENY and WARN_AGENT rules across models, budgets, time-of-day, and data residency, enforced in-line before any provider call. Policies are scoped per-workspace (or platform-global) at the entity level; SHADOW policy status lets you test a candidate policy against real traffic before promoting to ACTIVE.
PII scanning and enforcement — configured prompts are scanned for sensitive data before provider dispatch. The per-workspace action defaults to LOG; BLOCK refuses, REDACT removes values permanently, and Enterprise TOKENIZE creates recoverable request-side tokens.
Tamper-evident audit trail — every request is logged with workspace, model, provider, token counts, and latency, HMAC-signed and hash-chained for tamper-evidence.
Budget caps that enforce — hard and soft limits in dollars, not just requests per second. Hard breaches reject the request with 402; soft breaches trigger automatic model downgrade.
Multi-provider routing — send model: claude-sonnet-4-5 and DVARA routes to Anthropic, model: gpt-4o routes to OpenAI, model: gemini-2.5-pro routes to Google, all through the same endpoint.
Failover — if a provider returns a 5xx, DVARA retries with a healthy alternative automatically.

And none of this requires a single line of application code change. Your SDK still sees standard OpenAI-format responses.

Step 1: Start the Gateway

git clone https://github.com/dvarahq/dvara-examples.git
cd dvara-examples/docker-compose/quick-start
cp .env.example .env      # set OPENAI_API_KEY
docker compose up -d

The gateway starts on port 8080. Verify it's running:

curl http://localhost:8080/actuator/health

You should see {"status":"UP"}.

Step 2: Send Your First Request

Since DVARA is fully OpenAI-compatible, point any OpenAI SDK at it.

Why api_key is still needed in the SDK

The provider API keys you set on the gateway (OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.) are used by the gateway to authenticate with upstream providers. Your application never sees them.

The api_key in the SDK client is a separate concern — it's the key your application uses to authenticate with the gateway itself. Create a workspace and issue a DVARA API key (prefixed gw_) in the DVARA Flightdeck at http://localhost:8090, and use that.

Every call needs a valid key. From DVARA 1.8.0, a request with no key, or with a key the gateway does not know, is refused with 401. A placeholder string such as "any-key" does not work.

Python

First, install the OpenAI Python SDK (requires Python 3.8+):

pip install openai

Create a file called dvara_test.py:

from openai import OpenAI

client = OpenAI(
    api_key="gw_your-workspace-key",  # issued in the DVARA Flightdeck
    base_url="http://localhost:8080/v1"
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What is the capital of India?"}]
)
print(response.choices[0].message.content)

Run it:

python dvara_test.py

You should see the model's response printed to your terminal.

TypeScript / Node.js

First, install the OpenAI Node.js SDK (requires Node.js 18+):

npm install openai

Create a file called dvara_test.mjs:

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "gw_your-workspace-key", // DVARA API key — use a real gw_ key in production
  baseURL: "http://localhost:8080/v1",
});

const response = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "What is the capital of India?" }],
});
console.log(response.choices[0].message.content);

Run it:

node dvara_test.mjs

You should see the model's response printed to your terminal.

curl

curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer gw_your-workspace-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "What is the capital of India?"}]
  }'

The response is a standard OpenAI chat completion object. Your application code doesn't know it's talking to a gateway.

Step 3: Add More Providers

The real power of an AI gateway appears when you add a second provider. The same SDK client, the same endpoint, the same code — just a different model name.

3.1 Switch to the multi-provider stack

The quick-start stack is wired for OpenAI only. Stop it, then use the multi-provider stack from the same dvara-examples repository, which is wired for several providers.

3.2 Start it with both provider keys

Set OPENAI_API_KEY and ANTHROPIC_API_KEY in the multi-provider stack's .env, then start it with docker compose up -d, as the Docker guide describes.

Wait for the gateway to start, then verify it's up:

curl -s http://localhost:8080/actuator/health

You should see {"status":"UP"}. To inspect the registered providers, use the live /v1/models listing — it queries each registered provider in turn:

curl -s http://localhost:8080/v1/models -H "Authorization: Bearer gw_..."

(For operator-only runtime status such as providers[] and routes[], call /actuator/gateway-status with Authorization: Bearer $DVARA_ACTUATOR_API_KEY. The commercial /actuator/license endpoint reports posture and licence metadata separately; neither endpoint returns the signed licence envelope.)

3.3 Send the same prompt to both providers

Create a file called multi_provider.py:

from openai import OpenAI

client = OpenAI(
    api_key="gw_your-workspace-key",
    base_url="http://localhost:8080/v1"
)

# Routes to OpenAI
print("=== OpenAI (gpt-4o) ===")
openai_response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Explain virtual threads in Java in 2 sentences"}]
)
print(openai_response.choices[0].message.content)

# Routes to Anthropic — same client, same endpoint, same format
print("\n=== Anthropic (claude-sonnet-4-5) ===")
anthropic_response = client.chat.completions.create(
    model="claude-sonnet-4-5",
    messages=[{"role": "user", "content": "Explain virtual threads in Java in 2 sentences"}]
)
print(anthropic_response.choices[0].message.content)

Run it:

python multi_provider.py

Output:

=== OpenAI (gpt-4o) ===
Virtual threads in Java, introduced in Project Loom, are lightweight threads
managed by the JVM rather than the OS. They allow applications to handle
millions of concurrent tasks without the memory overhead of platform threads.

=== Anthropic (claude-sonnet-4-5) ===
Virtual threads are lightweight threads introduced in Java 21 through Project
Loom that are managed by the JVM rather than the operating system. They enable
writing simple blocking code that scales to millions of concurrent operations
without the complexity of reactive programming.

Notice: your code didn't change between providers. DVARA routes by model prefix — gpt goes to OpenAI, claude goes to Anthropic — and translates the request/response format automatically.

Supported providers

Model prefix	Provider	Environment variable
`gpt`, `o1`, `o3`, `o4`, `chatgpt`	OpenAI	`OPENAI_API_KEY`
`claude`	Anthropic	`ANTHROPIC_API_KEY`
`gemini`	Google Gemini	`GEMINI_API_KEY`
`bedrock/`	AWS Bedrock	`BEDROCK_ENABLED=true` + `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY`
`azure/`	Azure OpenAI	`AZURE_OPENAI_API_KEY` + `AZURE_OPENAI_BASE_URL`
`mistral`	Mistral	`MISTRAL_API_KEY`
`command`	Cohere	`COHERE_API_KEY`
`groq/`	Groq	`GROQ_API_KEY`
`qwen`	Alibaba Qwen	`QWEN_API_KEY`
`deepseek`	DeepSeek	`DEEPSEEK_API_KEY`
`moonshot`	Moonshot (Kimi)	`MOONSHOT_API_KEY`
`glm`	Zhipu ChatGLM	`ZHIPU_API_KEY`
`grok`	xAI Grok	`XAI_API_KEY`
`ollama/`	Ollama (local)	`OLLAMA_ENABLED=true`

Add any provider by setting its env var and restarting the container. No config files needed.

What's Next

You now have a running AI gateway that routes to multiple providers through a single SDK.

Or jump straight to the docs:

LLM Gateway overview — the pillar guide: one API in front of every model, with routing, fallback, cost control, and observability.
Configure routing — round-robin, weighted, latency-aware, or cost-aware routing across providers.
Enable PII scanning — detect and redact sensitive data in prompts before they leave your network.
Set budget caps — prevent bill shock with per-workspace token budgets and automatic model downgrade.
Deploy with Docker Compose or Helm — production-ready deployment with PostgreSQL persistence.
Browse all SDK examples — Python, TypeScript, Java integration walkthroughs.

What you get by pointing your SDK at DVARA​

Step 1: Start the Gateway​

Step 2: Send Your First Request​

Python​

TypeScript / Node.js​

curl​

Step 3: Add More Providers​

3.1 Switch to the multi-provider stack​

3.2 Start it with both provider keys​

3.3 Send the same prompt to both providers​

Supported providers​

What's Next​

What you get by pointing your SDK at DVARA

Step 1: Start the Gateway

Step 2: Send Your First Request

Python

TypeScript / Node.js

curl

Step 3: Add More Providers

3.1 Switch to the multi-provider stack

3.2 Start it with both provider keys

3.3 Send the same prompt to both providers

Supported providers

What's Next