Seven Major Language Models Tested on Radiology Exam Show Uneven Clinical Readiness
A Cureus study compared seven mainstream large language models on the 2022 American College of Radiology Diagnostic Imaging In-Training Examination. The results offer a useful reality check on how far general-purpose AI still is from dependable radiology support.
Benchmark studies like this are valuable because they move the discussion from broad claims to specific performance against a known standard. In radiology, that matters: the field demands precision, and even small errors can have outsized consequences.
The larger point is that “doing well on an exam” is not the same as being deployable in practice. LLMs can appear competent on multiple-choice tests while still lacking the consistency, domain grounding, and contextual judgment required in clinical workflows.
Comparative studies are especially helpful because they show that not all foundation models behave the same way. For health systems, that means procurement decisions should be based on measured task performance rather than brand recognition or hype.
The likely near-term use case is not autonomous diagnosis, but assistance in education, triage, and structured support. This kind of evidence helps define the boundary between a promising tool and a clinically credible one.