Skip to main content
Version: Latest (1.9.x dev)

Govern model calls that MCP tools ask for (sampling)

DVARA, the AI governance platform, puts every model call under the same policy, PII scan, budget and audit, including the ones an MCP tool asks for. An MCP tool server can ask for a model call while it runs. On MCP 2026-07-28 it does this by answering a tool call, a resource read or a prompt request with an input request of type sampling/createMessage: "run this prompt for me, then call me again with the answer". The specification leaves that model call to the client.

If a gateway passed the request back to your application, the model call would happen outside your governance: no policy, no PII scan, no budget, no cost record, no audit. So the DVARA MCP Gateway never passes it on. It does one of two things, chosen per workspace:

SettingWhat happens
Refuse it (default)The call fails with MCP_SAMPLING_REFUSED.
Make the call through the LLM GatewayThe Gateway makes the model call itself, as the caller, through the LLM Gateway. Then it sends the same tool call, resource read or prompt request again with the answer. Your application sees one call.
Enterprise only

Not included in DVARA Open Source.

What the model call gets​

When a workspace chooses the second option, the model call is an ordinary call to the LLM Gateway's /v1/chat/completions, made with the API key of the application that called the tool. It gets everything any other model call in that workspace gets:

  • the workspace's allowed models, routes and policies;
  • the PII scan on the prompt and on the answer, and the guardrails;
  • the budget, the cost record and the usage record, under the caller's key;
  • loop detection in the same session;
  • its own audit record, with the same trace id as the tool call, so you can join the two.

That audit record also names the MCP server that asked for the call, the request it asked on (a tool call, a resource read, a prompt or a task) and, where there is one, the tool and the task. You can find every model call a server caused from the LLM Gateway's own records. An application cannot set these fields on its own model calls: the Gateway marks only the calls it made itself.

The MCP audit trail records each model call (MCP_SAMPLING_CALL) and each round trip to the tool. Prompt and answer text is never written to the audit trail.

What a tool cannot do​

The Gateway answers only what the workspace allows. Each of these is refused before any model call is made, and the tool call fails with MCP_SAMPLING_REFUSED:

  • Choose the model. The model is the one the workspace set. The tool's model preference is recorded so you can see what it asked for, and is not used.
  • Use tools inside the model call. Requests with tools or toolChoice are refused.
  • Read other servers' context. includeContext other than none is refused.
  • Send images or audio. Only text is answered.
  • Ask for unlimited output. maxTokens is required. A value above the Gateway's limit (4,096 by default) is lowered to it.
  • Keep asking. One tool call, resource read or prompt request may take at most 3 sampling round trips and 8 model calls by default.
  • Run with someone else's credentials. The call is made with the caller's API key. A caller signed in with OAuth has no key to make it with, so it is refused.

If the LLM Gateway refuses the model call (a policy, the allowed models, the budget or a rate limit), the tool call fails with MCP_SAMPLING_FAILED and the LLM Gateway's reason. Nothing is sent to the tool in place of an answer.

Turn it on for a workspace​

You need the workspace admin role, or a platform role.

  1. In DVARA Flightdeck, open Agents → MCP Sampling (workspace admin), or the workspace's settings, section MCP sampling (platform roles).
  2. Choose Make the call through the LLM Gateway.
  3. Enter the Model the Gateway should ask for, for example gpt-4o-mini. It must be a model the workspace may call. A model is required.
  4. Save. Gateways apply the change with their next configuration update.

Every change is recorded in the audit trail. The setting is also exported and imported with the workspace in GitOps configuration files, as two keys:

mcp.input_requests.sampling: gateway
mcp.input_requests.sampling.model: gpt-4o-mini

An import that names an unknown mode, or gateway without a model, is refused.

Limits and settings for operators​

SettingDefaultMeaning
dvara.mcp-gateway.input-requests.sampling.max-tokens4096Most tokens one model call may ask for
dvara.mcp-gateway.input-requests.sampling.max-rounds3Most sampling round trips in one tool call
dvara.mcp-gateway.input-requests.sampling.max-model-calls8Most model calls in one tool call
dvara.mcp-gateway.input-requests.sampling.timeout60sHow long one model call may take
dvara.mcp-gateway.input-requests.sampling.llm-base-urlthis GatewayWhere the LLM Gateway is. Set it when the Gateway serves only HTTPS

None of these turns sampling on. Only a workspace setting does.

Because the model call is made with the caller's key from the Gateway itself, two things can refuse it that you might not expect:

  • an API key whose scopes do not include the LLM Gateway;
  • a workspace IP allow-list that does not include 127.0.0.1.

Sampling while a task runs​

A tool that runs as a task can ask for a model call while it waits. When the application that created the task polls it, the Gateway makes the model call with the same rules and limits as for a tool call, sends the answer back to that task, and reads the task again. The application sees the task move on. Only the application that created the task can poll it, so only its key can cause the model call.

What is not covered yet​

  • Sampling asked for by servers that use an older MCP revision. The Gateway answers sampling only from servers it reaches on 2026-07-28.
  • Tool use in sampling. A request that offers the model tools is refused.
  • A result that asks for a model call and, at the same time, for something else (a question for the user or the client's folders). It is refused.

Where to go next​