FDA Wants to Credential AI Like a Doctor. The Infrastructure Doesn't Exist.

On August 18, 2026, FDA’s Center for Devices and Radiological Health released a 30-page discussion paper proposing a new regulatory framework for generative AI-enabled medical devices. The headline concept is intriguing: credential AI devices the way you credential physicians — demonstrate competency before deployment, monitor performance in the field, and re-evaluate when capabilities change. The comment window closes October 19, 2026. That’s seven days from now.

Most manufacturers haven’t read it carefully. Most health systems haven’t read it at all. And the governance infrastructure the framework assumes — clinical adjudicators, monitoring workflows, foundation model accountability chains — doesn’t exist yet in any operational form. That gap is not a detail. It is the central problem this framework will spend the next several years trying to solve, and the decisions being made right now in the comment record will determine whether it ends up workable or unenforceable.

If you build, deploy, or govern generative AI in a clinical context, this is the regulatory framework that will define your next decade. Here is what it actually requires, where the infrastructure breaks down, and what happens when FDA comes back with a final rule and your products still haven’t been mapped to it.

Model Risk Reality

FDA’s proposed framework introduces a two-axis risk matrix as the entry point for regulatory scrutiny. One axis scores functional independence: from non-directive informational output at the low end, through action-directing outputs and supervised actions, to fully autonomous action at the top. The other axis scores the severity of consequence if a user acts on an incorrect output. Where your product lands on this grid determines the evidence package required to support a regulatory submission.

This sounds like a clean abstraction. In practice, it is not. Generative AI products don’t have a single position on either axis — they have a distribution of behaviors across use cases, user populations, and deployment contexts. A diagnostic AI that flags imaging findings for physician review may sit low on the autonomy axis in a supervised radiology workflow and much higher when deployed to primary care settings where the “physician review” step is abbreviated under time pressure. The framework offers no guidance on how to handle this variance, and it doesn’t specify whether the regulatory position is set at product design or at deployment context. That ambiguity is a validation gap that FDA reviewers and manufacturers will argue about for years.

The premarket competency benchmark breaks into ten elements: S.1 through S.3 cover safety, E.1 through E.4 cover clinical proficiency, R.1 and R.2 cover generalizability, and A.1 covers agentic AI conduct. Each element maps to a set of test configurations, but the framework is deliberately non-prescriptive about methodology. This is the FDA acknowledging that it cannot define specific benchmarks for a technology category that doesn’t exist in final form yet. That’s honest. It also means that until FDA publishes follow-on guidance, every manufacturer is writing its own validation protocol and hoping it survives review.

The clinical confirmation ladder has five rungs of rising rigor: retrospective chart review, shadow deployment with no patient exposure, standardized patient or simulation studies, clinician adjudication of real-world outputs, and prospective studies. A prospective trial is not required in every case — FDA is explicit that risk scaling determines which rung is required. But for action-directing functions in high-severity consequence scenarios, the framework will clearly push toward prospective evidence. Most manufacturers building agentic clinical AI have designed their go-to-market around retrospective validation and clinical expert opinion. That approach may not survive the formal framework.

The deeper model risk problem is what happens after deployment. The paper proposes accepting more premarket uncertainty in exchange for robust continuous postmarket monitoring. That tradeoff requires a functioning monitoring infrastructure: real-world output logging, drift detection, sample-based clinician review, and incident escalation pathways back to the manufacturer. The question FDA explicitly asks in its discussion paper — Question 20 — is whether machine-based supervisory agents could perform some of this monitoring. Framing the answer to that question correctly in the comment record matters, because if FDA writes monitoring requirements that assume automation, health systems that can’t build it will be out of compliance on day one.

The Governance Gap

The framework introduces “shared ecosystem responsibility” as its model for postmarket accountability. The idea is that accountability for monitoring real-world AI performance is distributed across manufacturers, deploying health systems, clinicians, payors, and clinical societies. This is conceptually reasonable — no single party has full visibility into how a generative AI device performs across the range of clinical environments where it gets deployed. But “shared responsibility” without clearly defined obligation tiers and enforcement points is not a governance model. It is a description of the current state, which is exactly what the framework is supposed to fix.

The foundation model layer illustrates the problem clearly. FDA proposes voluntary “Foundation Model Device Master Files” — confidential submissions in which model developers would document architecture, training data provenance, safety testing, and update schedules. Manufacturers building on top of foundation models could reference these files in their own submissions. This would be valuable if foundation model developers actually participated. The paper asks directly whether there is “limited incentive to disclose,” which is a polite way of acknowledging that the answer is almost certainly yes. Why would a foundation model developer voluntarily submit confidential architecture information to FDA before any legal requirement to do so? The answer is that they generally won’t, and the manufacturers who depend on them will be required to characterize their foundation models without the information needed to do so rigorously.

The agentic AI provisions make the governance gap most visible. Benchmark element A.1 covers agentic conduct: planning within a defined safety envelope, recognizing tool-call errors, and maintaining human checkpoints before irreversible actions. This is one element out of ten — it receives the same structural weight as E.2, which covers basic clinical proficiency accuracy. For a multi-step autonomous clinical AI system executing a treatment protocol, “planning within a safety envelope” and “maintaining human checkpoints” are entire system design disciplines, not single validation checkboxes. The framework does not define what a safety envelope is, how checkpoints should be implemented, what counts as an irreversible action in a clinical context, or how to validate that the agent will stop when it encounters an error it doesn’t recognize. These are not details FDA can leave to manufacturers to define independently.

The health system side of the governance gap receives the least attention in the paper. Health systems are described as potential sites for retrospective evaluation, shadow deployment, and clinical adjudication of outputs. They are mentioned as likely monitors of real-world device performance. But their specific obligations — what data they must log, what incidents they must report, what operational changes they must make when a vendor updates a foundation model — are completely undefined. This matters because monitoring costs are real costs. A clinical AI product that generates clinician review requirements every N outputs at scale creates workload that doesn’t show up in any current deployment budget.

What the Examiner Will Find

When FDA reviewers eventually examine submissions for generative AI-enabled medical devices under a final framework, the documentation gaps that are already predictable from the current discussion paper are not small ones.

The two-axis placement will require written justification. Reviewers will look for documentation showing that the manufacturer evaluated the full distribution of deployment contexts and autonomy levels, not just the best-case scenario. Products placed low on the autonomy axis because the manufacturer designed a nominal “physician review” step — without evidence that the step actually constrains decision-making in practice — will face scrutiny. FDA has seen this pattern before in clinical decision support submissions. The new framework’s risk matrix makes it more explicit, but the underlying temptation to under-scope regulatory exposure is unchanged.

Benchmark selection will be checked against intended use. If a product’s intended use statement covers a patient population with significant demographic variation, and the R.1 and R.2 generalizability benchmarks were only evaluated on a narrow validation cohort, that gap will surface in review. Generative AI products trained on data from large academic medical centers will be asked how their performance generalizes to community hospitals, rural health settings, and underrepresented patient populations. This is not a new concern in FDA review, but the framework makes it a mandatory element of the validation package.

The clinical confirmation approach will be scrutinized for independence. A pattern that will draw attention immediately: manufacturers using LLM-based systems as clinical adjudicators for their own products’ outputs. The framework paper notes that independence expectations for human expert adjudicators “would still be applicable” when the adjudicator is an LLM. What this means operationally is that a manufacturer cannot validate a generative AI diagnostic tool by having a related AI system grade its outputs as correct. Clinical validation has to remain clinically grounded, with genuine independence between the device being tested and the adjudication process.

Postmarket monitoring commitments will be examined carefully, and the ones that are unachievable at stated scale will create problems. If a manufacturer commits in a premarket submission to sample-based clinician review of 5% of real-world outputs and then deploys at 100,000 outputs per day, that’s 5,000 clinician reviews daily. Either the commitment was not made seriously, or the deployment scale was not anticipated. FDA reviewers will ask what happens to the monitoring program when volume exceeds the committed capacity, and “we’ll figure it out then” is not an acceptable answer.

The SuperML Take

The FDA’s competency-based framework for generative AI medical devices is the most serious attempt yet to bring regulatory rigor to a technology category that desperately needs it. The two-axis risk framework is a real improvement over the binary CDS exemption logic that allowed manufacturers to avoid FDA oversight by adding a disclaimer that a physician makes the final decision. The ten-element benchmark structure is more systematic than ad hoc validation protocols. The clinical confirmation ladder provides a proportionate evidence structure that doesn’t require a prospective RCT for every use case.

None of that changes the fundamental problem: the framework assumes governance infrastructure that does not exist. Clinical adjudicators with the independence and expertise to validate generative AI outputs at scale don’t have a credentialing system, an oversight body, or a fee structure. Health system monitoring obligations are described in aspirational terms with no defined data model, reporting cadence, or enforcement mechanism. Foundation model developers have no legal obligation to disclose anything and no economic incentive to start voluntarily. The framework distributes responsibility across the ecosystem while leaving the question of who enforces compliance against each party genuinely open.

The comment deadline of October 19 is the moment to push on these gaps. Not to slow the framework down — clinical AI needs regulatory structure, and the absence of a framework is not protecting patients. The push should be specific: define what “shared ecosystem responsibility” means as a legal matter, not a governance aspiration. Require that foundation model developers who supply medical device manufacturers document their model versions, update schedules, and known limitations as a condition of their commercial relationship — even if they don’t submit to FDA directly. Specify what health system monitoring obligations look like in operational terms that can be budgeted and built.

For manufacturers, the strategic position right now is not to wait for the final framework. The two-axis placement question is knowable today. The product teams building clinical AI should map every use case to the grid before the final guidance arrives — not because FDA is watching yet, but because the gaps that surface in that mapping will take 18 months to close, and the final rule won’t give you 18 months. The manufacturers who have done this work already will have a genuine regulatory advantage over the ones who haven’t.

For health systems deploying generative AI from third-party vendors: the monitoring obligations the framework describes will eventually become your operational responsibilities. That means your vendor contracts need to address model version control, change notification timelines, and incident reporting before those obligations are legally mandated — not after. A vendor who won’t commit to notification when they update a foundation model is one who is implicitly planning to deploy changes you can’t validate. Get the contractual commitments now, while you have leverage.

The October 19 comment deadline is not a formality. The decisions FDA makes in response to the comment record will determine whether this framework becomes workable or collapses under the weight of its own governance assumptions. The framework is on the table. The question is whether the people who have to operate under it show up to shape it.

Sources

• FDA Discussion Paper: Considerations for the Regulation of Generative AI-Enabled Medical Devices (Docket FDA-2026-N-7874)
• Cooley: Regulating AI Like a Doctor — FDA Floats Competency-Based Path for Generative AI-Enabled Devices
• Innolitics: FDA Generative AI Medical Devices Discussion Paper — What’s Inside
• Maynard Nexsen: FDA Releases Discussion Paper on Generative AI-Enabled Medical Devices
• Arnold Porter: FDA Seeks Public Feedback on Regulatory Approach for Generative AI-Enabled Medical Devices
• MassDevice: FDA Considers Competency-Based Evaluations for GenAI-Enabled Medical Devices
• Akin Gump: FDA’s Generative AI Discussion Paper Signals a Break from the Past
• Applied Clinical Trials: FDA Seeks Public Input on Regulatory Framework for Generative AI-Enabled Medical Devices