Structured Outputs

Dvara supports OpenAI-compatible response_format on POST /v1/chat/completions. Clients send a single format specification and the gateway transparently translates it to each provider's native mechanism.

Supported Formats

type	Description
text	Default. No constraint on the response format.
json_object	Instructs the model to return valid JSON.
json_schema	Instructs the model to return JSON conforming to a specific JSON Schema.

json_object Mode

Request the model to return valid JSON:

curl -s -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "user", "content": "List 3 colors as a JSON array."}
    ],
    "response_format": {"type": "json_object"}
  }'

json_schema Mode

Request the model to return JSON conforming to a specific schema:

curl -s -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "user", "content": "Extract the person name and age from: John is 30 years old."}
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "person",
        "schema": {
          "type": "object",
          "properties": {
            "name": {"type": "string"},
            "age": {"type": "integer"}
          },
          "required": ["name", "age"]
        },
        "strict": true
      }
    }
  }'

Provider Translation

The gateway translates response_format to each provider's native mechanism. Your application always sends the same OpenAI-format request.

Provider	json_object	json_schema	strict Support
OpenAI	Native passthrough	Native passthrough	Native
Anthropic	System prompt injection	Tool-use rewrite (structured_output tool)	Downgraded
Gemini	responseMimeType: "application/json"	responseMimeType + responseSchema	Native
Bedrock	System message injection	toolConfig rewrite (structured_output tool)	Downgraded
Ollama	Not supported (returns 400)	Not supported (returns 400)	N/A
Mock	Wraps response in JSON object	Wraps response in JSON object	N/A

How Anthropic Translation Works

For json_object, the gateway appends "Respond with valid JSON only." to the system message.

For json_schema, the gateway rewrites the request to use Anthropic's tool-use API:

1. Your JSON Schema is sent as the input_schema of a tool named structured_output
2. tool_choice is set to force the model to use this tool
3. The model's tool_use response block is extracted and returned as the content text

This translation is transparent — your application sees a normal content field with JSON text.

Strict Mode Downgrade

OpenAI and Gemini support strict: true natively (constrained decoding). For Anthropic and Bedrock, the gateway uses tool-use rewrite which provides best-effort schema enforcement but cannot guarantee strict validation.

When strict: true is requested on these providers:

1. The schema is sent as a tool input schema (best-effort enforcement)
2. The response includes header X-Gateway-Strict-Downgraded: true

Check for this header to decide whether to perform client-side schema validation:

response = client.chat.completions.create(
    model="claude-sonnet-4-5",
    messages=[...],
    response_format={"type": "json_schema", "json_schema": {..., "strict": True}}
)

# Check if strict was downgraded
if response.headers.get("X-Gateway-Strict-Downgraded") == "true":
    # Validate response against schema client-side
    validate_schema(response.choices[0].message.content)

Capability-Aware Routing

When response_format is present, the gateway filters the provider pool before routing to ensure only capable providers are considered:

response_format type	Required capability
json_schema	supportsStructuredOutputs = true
json_object	supportsJsonMode = true
text / absent	No filtering applied

No Capable Provider

If no provider on the route supports the requested format, the gateway returns HTTP 400:

{
  "error": {
    "code": "no_capable_provider",
    "message": "No provider supports response_format: json_schema. Providers on route: [ollama]. Capable providers: []"
  }
}

Failover Blocked

If the primary provider fails and no capable fallback exists, the gateway returns HTTP 503 with an X-Gateway-Failover-Blocked: capability_mismatch header:

{
  "error": {
    "code": "failover_capability_mismatch",
    "message": "Failover blocked: no fallback provider supports response_format: json_schema. Primary provider [openai] failed: upstream down"
  }
}

This header allows clients to distinguish between a normal provider outage (retry later) and a capability gap (reconfigure the route or remove the response_format requirement).

Validation Errors

Condition	HTTP	Error Code
Missing response_format.type	400	invalid_request
Unknown type (e.g., xml)	400	invalid_request
json_schema without schema	400	invalid_request
Unsupported provider (e.g., Ollama)	400	unsupported_response_format
No capable provider on route	400	no_capable_provider
Failover blocked by capability	503	failover_capability_mismatch