Analytics
Enterprise only
Not included in DVARA Open Source.
Every request through DVARA emits cost, latency, token, policy-decision, audit, and trace telemetry. Flightdeck shows that data according to the signed-in user's role and active workspace. Machine-readable feeds—Prometheus, OpenTelemetry, structured logs, and SIEM streams—carry the same evidence into your observability stack.
This page is a survey of what's available and where to find it. For deep dives, follow the links into the per-feature documentation.
The three lenses
| Lens | Audience | What it gives you |
|---|---|---|
| Platform-wide Flightdeck pages | Platform operators, security admins, finance | Cross-workspace operational view with live auto-refreshing panels (3–10s), role-based access controls, and click-through detail. |
| Workspace-scoped Flightdeck pages | Workspace users — application developers, team leads, auditors | The same evidence narrowed to the active workspace. Self-service for workspace admin, developer, and viewer roles. |
| Raw feeds (Prometheus scrape, OTLP push, stdout JSON logs, Kafka/Splunk/CloudWatch SIEM) | Your own observability stack — Grafana, Datadog, Tempo, Jaeger, Splunk, ELK, … | Machine-readable telemetry meant to feed your existing dashboards, alerts, and SIEM. DVARA does not try to replace your observability platform; it emits the standard formats so you don't have to. |
Use the dashboards for day-to-day operations and the raw feeds for long-term retention, custom dashboards, and alerting. Reports (chargeback, compliance) are the one-shot PDF / CSV artifacts you hand to finance or auditors.
Quick orientation by audience
| You are a … | Start here |
|---|---|
| Platform operator running the gateway | Dashboard for live health; Traffic → Latency for routing decisions; Prometheus scrape for alerting |
| Security / compliance lead | Governance → Audit for the event stream; Governance → Compliance Reports for SOC2 / HIPAA / GDPR rollups; SIEM stream for long-term retention |
| Finance / FinOps | Cost → Cost Dashboard for live spend; Cost → Chargeback Reports for monthly workspace reports; gateway_cost_dollars_total for time-series budget burn |
| Workspace developer or team lead | Dashboard for workspace request counts, tokens, and cost; Usage for per-model breakdowns; Audit for the workspace audit trail |
| Agent platform team | Agents → Analytics for session outcomes, approval funnels, and tool-call heatmaps; MCP → Sessions for per-session timelines |
The full surface
Every analytic the platform exposes, what it shows, where to view it, and the underlying endpoint.
Live dashboards
| Analytic | What it shows | Where to view | Endpoint |
|---|---|---|---|
| Operational dashboard | Req/s, P95 latency, error rate, tokens/min, provider health, cache stats, priority-admission stats | Dashboard | GET /actuator/prometheus + GET /actuator/gateway-status |
| Per-workspace dashboard | Request count, token usage, 30-day cost, active API keys, recent audit events — scoped to caller's workspace | Workspace Dashboard | Flightdeck page scoped to the active workspace |
| Cost dashboard | Total cost, Cost-by-Provider donut, Cost-by-Model bar, budget status, forecast cards, anomaly alerts, raw records table | Cost → Cost Dashboard | Flightdeck only — no API |
| Latency dashboard | EWMA latency per (provider, model) — sample count, raw last sample, freshness badge | Traffic → Latency (owner, policy-admin) | Flightdeck only — no API |
| Priority admission stats | Concurrent / max concurrent / load %, per-tier admitting vs throttling | Dashboard priority card (when routing.priority.enabled=true) | Flightdeck only — no API |
| Semantic cache stats | Cache size and clear-cache, from the shared store | Traffic → Cache (owner only) | Hit rate, average similarity and lookup counts are per-Gateway-process counters. Scrape gateway_semantic_cache_{hits,misses}_total{workspace} from /actuator/prometheus |
| Licence status | Posture, licensee, type, expiry, days remaining, runtime status badge | Operations → Licence and the status banner | Enterprise GET /actuator/license |
Telemetry & event streams
| Analytic | What it shows | Where to view | Endpoint |
|---|---|---|---|
| Token usage telemetry | Platform roles see per-request tokens and key metadata; workspace roles see workspace totals and model charts. | Cost → Token Usage or workspace Usage | Flightdeck only — no API |
| Audit event stream | HMAC-signed, hash-chained events—policy decisions, auth, config imports, licence transitions—with detail and export | Governance → Audit | Flightdeck only — no API |
| Budget cap usage | Current period spend vs limit, color-coded progress bar, period boundaries, dollars remaining | Cost → Budgets | Flightdeck only — no API |
| MCP tool-call telemetry | Per-call records and aggregate counts | MCP → Tool Calls | Flightdeck only — no API |
| Agent session timeline | Session summaries, tool use, per-call timeline, and JSON export | MCP → Sessions | Flightdeck only — no API |
| Approval queue stats | Pending decisions, history, status badges, and navigation count | MCP → Approvals | Flightdeck only — no API |
Comparison & experimentation analytics
| Analytic | What it shows | Where to view | Endpoint |
|---|---|---|---|
| Canary A/B report | Per-variant metrics on a canary-routed route; reset + live split-percentage update | Flightdeck /routes/{id}/canary | Flightdeck only — no API |
| Shadow policy report | Divergence stats (1h / 24h / 7d) + per-event diff between active and SHADOW policy | Flightdeck /policies/{id}/shadow | Flightdeck only — no API |
| Shadow routing report | Provider-vs-shadow comparison on a route (latency, status, response delta) | Flightdeck /routes/{id}/shadow | Flightdeck only — no API |
| Prompt experiment metrics | Per-variant A/B metrics snapshot for a prompt experiment | Prompts → Experiments | Flightdeck only — no API |
| Agent behaviour analytics | Session-outcomes donut, approval-funnel bar, top-20 agent leaderboard, tool-call heatmap (server × tool), policy firing frequency | Flightdeck /analytics | Flightdeck page (composed from MCP + audit data) |
| Model fingerprint drift | Drift report per golden prompt — when an upstream model's response semantics shifted | Flightdeck eval/fingerprint surface | Flightdeck only — no API |
| Eval pipeline reports | Per-model eval suite results vs golden corpus, drift detection, time-ranged report queries | Flightdeck /eval/reports | Flightdeck only — no API |
Downloadable reports
| Analytic | What it shows | Where to view | Endpoint |
|---|---|---|---|
| Chargeback reports | Per-workspace monthly rollup: workspace summary, model/provider breakdown, daily cost trend, forecasts, anomalies — PDF + CSV | Cost → Chargeback Reports | Flightdeck only — no API |
| Compliance reports | SOC2 / HIPAA / GDPR rollups aggregated from audit + token usage + workspace + policy data — PDF | Governance → Compliance Reports | Flightdeck only — no API |
Raw feeds — point your own stack at these
| Analytic | What it shows | Where to view | Endpoint |
|---|---|---|---|
| Prometheus metrics | Request, latency, token, cost, guardrail, budget, MCP tool-call, and email-delivery series when their corresponding component is running | Your Grafana / Datadog / Mimir | GET /actuator/prometheus on the DVARA Gateway and Flightdeck — Bearer $DVARA_ACTUATOR_METRICS_API_KEY. MCP and A2A series use the Gateway endpoint; they do not have separate scrape endpoints. |
| OpenTelemetry traces | Distributed spans — gateway.provider.chat, …stream, …embed; mcp.filter.*, mcp.server.call — with traceparent propagation | Your OTel collector → Jaeger / Tempo / Datadog | OTLP push to OTEL_EXPORTER_OTLP_ENDPOINT (default http://localhost:4318/v1/traces) |
| Structured access log | One JSON line per request: trace_id, method, path, status, latency_ms, model, provider, cache_status, opaque api_key ID (or anonymous), workspace_id, token counts, error_code | Your log aggregator | stdout (structured JSON) |
| SIEM stream | All audit events forwarded to Splunk HEC, CloudWatch Logs, or Kafka | Your SIEM | dvara.llm-gateway.siem.{splunk|cloudwatch|kafka}.* |
The opaque API-key ID is the stable join key across usage, cost, budgets,
structured logs, audit events, analytics, and exports. Flightdeck turns it into
a recognizable key name and public prefix where a person needs to read it. The
plaintext bearer credential is not written to these surfaces; a request made
without an authenticated key is attributed to anonymous.
Per-workspace vs platform scoping
Almost every analytic in the tables above respects DVARA's multi-tenant scoping rules:
- Platform-role callers (
owner/policy-admin/billing-admin) see cross-workspace data by default. Most Flightdeck list pages accept an optional?workspace_id=filter to narrow. - Workspace-role callers (
admin/developer/viewer) only ever see their own workspace's rows. Passing another workspace's id returns HTTP 403 with aWORKSPACE_SCOPE_VIOLATIONaudit event.
Flightdeck enforces the same rule at the request boundary. Changing a URL cannot move a workspace user outside the workspaces their team membership allows. See Multi-Tenancy for the full isolation model.
What's deliberately not in the box
A few things the platform deliberately leaves to your existing tools:
- Long-term metric retention beyond the Prometheus scrape window — point a Mimir / Thanos / Datadog at the scrape endpoint.
- Custom dashboards beyond the built-in Flightdeck views — Grafana on top of the Prometheus and OTLP feeds covers the long tail.
- Alerting — Prometheus AlertManager or Grafana alerts on the same metrics; the platform emits the data, not the alert routing.
- Cross-system correlation (joining DVARA spans against your service's app spans) — handled by your APM via the
traceparentpropagation DVARA already emits.
Where to go next
- DVARA Flight Deck — the full Flightdeck tour, including every dashboard listed above with screenshots.
- Cost Management — cost dashboard, forecasts, anomaly alerts, budget caps, chargeback reports.
- Observability — audit event taxonomy, Prometheus metrics list, OpenTelemetry span names, SIEM configuration.
- Multi-Tenancy — the scoping rules that govern who sees what data.
- Operate DVARA with Flightdeck — where operators and workspace teams find each supported analytics workflow.