AI Agent Production-Readiness Scorecard

Can you produce a current list of every app, service, and agent calling an LLM in production — and which model each one uses?

Is every LLM and tool call authenticated, so you can name which app or agent made it (not just “traffic from the cluster”)?

When an agent calls another agent or a tool, is that invocation identified and authorized — not just an open internal hop?

Is there a hard spend cap enforced in the request path — one that actually blocks calls — not just a dashboard alert after the fact?

Are budgets scoped per app / team / agent, so one runaway can't drain the shared bill?

Can an agent that loops or retries be auto-killed before the invoice lands, rather than discovered on next month's statement?

Is there a tamper-evident record of every AI call — prompt, model, caller, cost, outcome — that a reviewer could trust six months from now?

Given any past request, can you reconstruct who authorized it and what it was permitted to do at that moment?

Are these logs held outside the reach of the teams and services that generate them, so the record can't be quietly edited?

Do agents hold only the permissions their task needs — read-only where reading is the job — rather than broad credentials that also let them write, move money, or delete?

Do consequential actions (payments, data mutations, external sends) pass a human approval gate before they execute?

If an agent is confused or compromised, is its blast radius bounded by design — caps, gates, kill switch — rather than “we’d catch it in review”?

Do you know, per call, whether customer or regulated data leaves your perimeter — and where it lands when it does?

Are model keys and sensitive data kept inside your boundary (self-hosted / BYOK) rather than living in a third party's logs under their retention policy?

Can you map your AI controls to the regime you answer to (EU data residency, sector rules, OWASP LLM Top 10) well enough to survive an audit or a security questionnaire?