Agent Inventory Is a Pipeline, Not a Spreadsheet

Ask your platform team how many AI agents are running in production right now. Then ask your security team, your data team, and whoever owns the Salesforce org. You’ll probably get four different numbers, and each person will be sure theirs is right. That’s the agent inventory problem. Nobody is hiding anything. Agents now get created in at least six different places: cloud agent services, data platforms, SaaS admin consoles, low-code builders, coding assistants, and custom code. Each of those places keeps its own partial list.

The usual fix is a registration process. You make a form and a policy, add an approval step, and require every new agent to be registered before it ships. It looks tidy on a governance slide, and it breaks down in production for a simple reason. Registration only captures the agents whose builders chose to register them, at the moment they registered. It misses the Copilot Studio agent a finance analyst built last Tuesday. It misses the Agentforce agent that got three new tool permissions after go-live. It misses the LangGraph service whose model changed from one provider to another in a config push. An inventory built on what people tell you is out of date the day after you finish it.

My argument in this post is that agent inventory is a data engineering problem, not a documentation problem. You build it the way you build any system of record that has to stay true while its sources change: continuous discovery from several independent signals, a canonical schema, reconciliation that surfaces disagreements, and evidence attached to each record. This week gave us several reasons to do it now.

The Confidence Gap Is the Real Finding

Two surveys published this week measure the same blind spot from different angles.

Guild.ai’s AI Agent Management Gap report was fielded by Morning Consult in early August with 362 U.S. IT decision-makers. It found that 96.4% are confident their organization has a complete and accurate agent inventory. Yet 66.7% of organizations running agents had an agent-related operational incident in the past twelve months. Only 42.7% have a centralized monitoring dashboard, only 39.8% have logging or audit trails, and only 35.9% require agents to be registered before deployment. Meanwhile 60.6% think it’s likely employees are deploying agents without formal approval.

Dataiku’s Global AI Confessions Report: CIO Edition, released at its Succeed conference on September 24, surveyed 685 CIOs and found the same shape. Nine in ten said they had complete tracking of their agents. When the question turned to agents built outside approved channels, 81% admitted they lacked complete oversight, and 84% said employees are building agents faster than IT can govern them. IBM research cited in the launch puts the share of organizations with a complete, current AI inventory at under one in five.

The architectural reading matters more than the headline numbers. Nearly everyone is confident, and nearly everyone is also missing the basic instruments (logs, dashboards, registration gates) you’d need to verify that confidence. When an organization is sure of an inventory it has no way of checking, that isn’t a governance posture. It’s an unreconciled data set that nobody has queried.

Why Registration-Based Inventories Decay

There are four distinct ways a registration-based inventory drifts from reality, and each one needs its own detection mechanism.

Creation drift is the obvious one. Someone builds an agent in a tool that makes it easy, like Copilot Studio, Agentforce, a Bedrock console, or a Snowflake Cortex notebook, and never files the form. Guild’s 60.6% shadow-agent estimate is about this category.

Capability drift is quieter and more dangerous. The agent was registered correctly, but since then it has picked up new tools, new MCP server connections, broader OAuth scopes, or access to a new data source. The inventory entry still describes the agent as it was on approval day. From a risk perspective this is a different agent under the same name.

Dependency drift is when the model, prompt, retrieval corpus, or downstream API changes under a registered agent. A model version bump or a prompt edit is a behavior change, and in a regulated setting it’s often a material change to a validated system. Most registration flows never see it.

Lifecycle drift is the zombie problem. The project ended, the owner left or changed teams, but the agent’s credentials, schedule, and tool grants are still live. Guild found that 61.2% of engineering teams lose time to duplicate agent work. Many of those duplicates exist because nobody could find the original, or because the original is orphaned and nobody is sure it’s safe to reuse.

A form catches only the first kind, and only when people comply. The other three can only be caught by watching what agents actually are and do.

Inventory as a Discovery Pipeline

The better pattern treats agent inventory like a CMDB that is continuously reconciled from telemetry, not from declarations. That’s roughly the approach Dataiku is taking with Agent Management. It connects to AWS Bedrock, Databricks Agents, Google Vertex, Microsoft Copilot Studio and Azure Foundry, Salesforce Agentforce, and Snowflake Cortex, and accepts OpenTelemetry from custom environments. It scans them into one inventory that maps each agent’s tools and models and assigns a risk tier based on autonomy, data sensitivity, and business impact. You don’t need that specific product to use the pattern. What matters is the architecture underneath it.

Layer 1: Multiple independent discovery sources. No single source gives you complete coverage, so you triangulate. Platform control-plane APIs tell you which agents each vendor’s service knows about. Your LLM gateway or AI proxy tells you which workloads are calling models, and with which credentials. Your identity provider tells you which non-human identities and OAuth grants exist. OpenTelemetry GenAI spans (invoke_agent, execute_tool) tell you which agents are actually running and what they touch. Code and infrastructure repositories tell you which agent frameworks are deployed and where. MCP gateway logs tell you which clients are connecting to which tool servers. Each source is incomplete, but the gaps are different, and that difference is what makes reconciliation possible.

Layer 2: A canonical agent record. Every discovered agent resolves to one record with a stable ID. At minimum that record needs: accountable business owner and technical owner; hosting platform and runtime; model(s) and version; tools, MCP servers, and data sources it can reach; the identity it acts as (its own, or a borrowed human one); autonomy level (suggest-only, act-with-approval, act-autonomously); the reversibility class of its highest-impact action; risk tier; validation status and date; and last-observed-active timestamp. If you’ve built a feature store or a model registry, this will feel familiar. It’s an entity-resolution problem across heterogeneous sources.

Layer 3: Reconciliation and drift detection. This is where the value comes from. Every discovery run compares observed state against the recorded state and raises typed exceptions: observed but not registered (shadow agent), registered but not observed in N days (possible zombie), observed capability exceeds recorded capability (capability drift), model or prompt hash changed since last validation (dependency drift), owner no longer in directory (orphan). Each exception type gets a routing rule and an SLA. This is the same discipline that makes data-quality pipelines work. The number to watch is not how many agents you have. It’s how many unresolved discrepancies you have.

Layer 4: Risk tiering that drives controls. A flat inventory is just a list. Tiering turns it into a control plane. An agent that summarizes internal wiki pages and an agent that initiates payment reversals should not go through the same governance workflow. The practical tiering inputs are autonomy level, data sensitivity (PII, NPI, PHI, MNPI), action reversibility, and blast radius (internal users vs. customers vs. counterparties). The tier then determines validation depth, monitoring frequency, human-checkpoint requirements, and how often tests rerun.

Layer 5: Evidence attached to records. For high-tier agents, the record should carry its own evidence: certification status, named risks, test results from scheduled evaluation runs, and links to the relevant trace samples. When an auditor asks, you export evidence that already exists. You don’t spend three weeks rebuilding it.

Why Banking Teams Should Care This Quarter

The regulatory pressure is now concrete, and it points straight at inventory. On September 16, the Conference of State Bank Supervisors released its AI Supervisory Framework for state examiners. State agencies supervise 3,355 of the country’s 4,233 FDIC-insured banks, so most U.S. banks may see these questions at their next state exam. The framework creates no new legal obligations. What it does is make AI use visible within existing examination practice, and the document list examiners can request is essentially an inventory spec: AI policies, use-case inventories, risk assessments, management reports, vendor contracts, and testing records. Examiners are prompted to ask whether each AI use has an assigned owner, whether uses are ranked by risk, and whether controls are revisited as systems change.

For agentic systems specifically, examiners are directed to look at the boundaries on an agent’s actions, human checkpoints, logs, reversibility, and the ability to stop it. The framework also covers AI embedded in third-party software. An institution that isn’t sure whether it uses AI may get follow-up questions about its vendor and software inventories. Your Salesforce and Microsoft 365 tenants are in scope whether or not your AI team built anything there.

Add NYDFS’s September 10 industry letter, which reminds regulated entities to refresh cybersecurity risk assessments when technology changes, AI and frontier models included, materially alter cyber risk. The pattern is clear. “We have a spreadsheet” will not survive a sample-based examination in which the examiner picks three agents and asks for each one’s owner, risk tier, change history, and last test result.

Common Failure Modes

Inventory by survey. Teams send a questionnaire to business units every quarter and call the results an inventory. Response rates fall, answers get copied from the last quarter, and the capability fields are never accurate. Surveys can supplement discovery. They can’t replace it.

Vendor-scoped visibility. Each platform’s native governance console shows its own agents well and everyone else’s agents not at all. Five consoles do not make one inventory. Dataiku is explicitly positioning against single-vendor monitoring for this reason. Whether you buy or build, the inventory has to sit above the platforms.

Identity-blind records. If agents act through borrowed human credentials, your identity provider can’t tell them apart from people, and your inventory can’t link an agent to what it did in the audit logs. Inventory quality is capped by identity quality.

Tiering at registration only. Risk tier gets assigned once on approval day and is never recomputed. An agent that gains a payments tool should move up a tier automatically. If recomputation isn’t triggered by capability drift events, the tier is just decoration.

Counting instead of reconciling. Dashboards that show “247 agents” feel like progress, but the number means nothing without the discrepancy count next to it. A system of record you can’t reconcile isn’t one.

Architecture Impact

What changes in system design? Agent inventory moves from a governance artifact to a platform service with ingestion, entity resolution, and a reconciliation loop. That means the LLM gateway, identity provider, MCP gateway, and OpenTelemetry collector become required inventory feeds, not just operational tools, and each agent runtime has to emit a stable agent ID on every model call and tool span. The inventory itself becomes an upstream dependency for policy enforcement: gateways and approval workflows query the agent’s tier at runtime instead of hardcoding rules per agent.

What new failure mode appears? The main new risk is confident staleness: an inventory that looks authoritative but was reconciled against incomplete sources, so it declares coverage it doesn’t have. This is exactly the 96%-confident / 67%-incident pattern in the survey data. A second failure is tier lag, where an agent’s capabilities expand faster than its risk tier gets recomputed, so it runs high-impact actions under low-tier controls until the next manual review.

What enterprise teams should evaluate:

• Platform engineering: whether every agent runtime (vendor-hosted and custom) emits a stable agent identifier in OpenTelemetry spans and gateway headers, and what percentage of model traffic can be attributed to a known agent record today.
• Identity / IAM: how many agents authenticate as themselves versus through a human’s delegated token, and whether non-human identities have owners and expiry dates that can be joined to the inventory.
• Model risk management / compliance: whether the inventory can produce, for any sampled agent, its owner, tier, current model and prompt version, change history, and most recent test evidence within one business day.
• Security operations: whether “observed but not registered” agents generate a triaged alert with an SLA, and what the median time from shadow-agent detection to disposition is.

Cost / latency / governance / reliability implications: The pipeline itself is cheap. Discovery runs are batch API calls and span aggregation, typically a few hundred dollars a month in compute for a mid-size enterprise, well under the cost of one duplicated agent build. Latency impact is limited to runtime tier lookups at the gateway, which should be cached, adding single-digit milliseconds. The governance payoff is large: examiner document requests turn into exports instead of multi-week projects. On reliability, reconciliation brings orphaned and zombie agents to the surface, and those are the long-lived credentials and forgotten schedules behind a meaningful share of the incidents Guild’s respondents reported.

Implementation Guide

Start with attribution, not with a catalog UI. The highest-leverage first move is to make every model call and tool call traceable to a stable agent ID. Route model traffic through a gateway that requires an agent identifier header, and adopt the OpenTelemetry GenAI conventions for agent and tool spans in your custom runtimes. Once you can answer “which agent made this call” for most of your traffic, discovery is a query rather than an investigation. Measure attributed traffic as a percentage of total model spend. That one number tells you more about inventory completeness than any registration count.

Next, stand up connectors for the two or three platforms where most of your agents actually live. For most enterprises that’s Microsoft (Copilot Studio and Azure Foundry), Salesforce Agentforce, and one hyperscaler agent service. Pull their agent lists, tool bindings, and model configurations nightly into a canonical record, and resolve them against what the gateway and identity provider see. Resist the urge to model every possible field on day one. Owner, platform, model, tools, data sources, identity, autonomy level, and last-active timestamp will get you through most examiner questions and most incident investigations.

The mistake to avoid is building a beautiful registry that depends on people updating it. Registration gates are still worth having for new agents in high-risk tiers, but treat registration as one input to reconciliation, not as the source of truth. The other common trap is tiering by platform (“everything in Copilot Studio is low risk”). Tier by capability and data access, and recompute automatically whenever discovery detects a new tool, scope, or data source.

You’ll know it’s working when the discrepancy queue has a stable, declining shape: shadow agents are found within days, not quarters; zombie agents are decommissioned instead of lingering; and when someone asks “which agents can move money or touch customer NPI,” the answer comes from a query in minutes, not a Slack thread. A good practical test is to have someone from model risk pick five agents at random each month and try to produce their full evidence package. Record the time it takes and what’s missing. That drill is your examination rehearsal.

Over six to twelve months, teams that get this right end up with the inventory as the policy backbone for their whole agent estate. Gateways enforce tier-based controls by looking up the record. Scheduled evaluations run against every high-tier agent and write results back to it. Cost and value metrics get attributed per agent, so portfolio questions like “where is risk concentrated” and “which agents cost more than they return” become answerable. At that point the inventory stops being a compliance cost and becomes the control plane that lets you let more agents act autonomously, because you can show exactly what each one is, what it can touch, and when it last passed its tests.

Sources

• The AI Agent Management Gap Is Growing — Guild.ai via GlobeNewswire (Sept 22, 2026)
• Dataiku debuts cross-platform Agent Management, expands Cobuild building agent — SiliconANGLE (Sept 24, 2026)
• Dataiku Agent Management reveals unmonitored AI agents — Help Net Security (Sept 25, 2026)
• CSBS Announces AI Supervisory Framework — CSBS
• The CSBS Artificial Intelligence Supervisory Framework — CSBS
• State Bank Regulators Give Examiners a Road Map for AI Oversight — PYMNTS (Sept 23, 2026)
• CSBS Releases Artificial Intelligence Supervisory Framework for State Examiners — Consumer Finance Monitor (Sept 18, 2026)
• NYDFS Industry Letter on Cybersecurity Risk Assessments (Sept 10, 2026)