Nature: AI Oversight Must Shift From Model Inputs to Real-World Capabilities
A Nature article argues that traditional AI oversight focused on training data, prompts, or model architecture is no longer enough. As large language models become more capable and more widely deployed, the key question is what they can do in practice and how those capabilities should be monitored over time.
The governance debate around medical AI is moving beyond static model reviews. If a system can be updated, fine-tuned, connected to tools, or embedded in clinical workflows, then the risk profile changes faster than a one-time evaluation can capture.
That is why capability-based monitoring matters. Rather than asking only whether a model was trained safely, regulators and health systems increasingly need to ask what tasks it can now perform, how reliably it performs them, and whether those abilities expand in ways that were not originally approved.
For healthcare, the stakes are especially high because capability can outpace visibility. A model that appears to be a documentation assistant today may become a quasi-decision-support layer tomorrow if it gains access to records, retrieval systems, or agentic tools.
The practical implication is that oversight must become continuous, not episodic. Hospitals, vendors, and regulators will need shared methods for logging behavior, defining capability thresholds, and triggering re-review when systems cross into new clinical risk territory.