🎧 Part I Podcast free on Apple Podcasts and Spotify.
🎧 Part II Podcast episode for paid subscribers only. Also available on Apple Podcasts and Spotify.
To listen to paid episodes in Apple or Spotify, link your Substack subscription via the show settings on those platforms (instructions inside the Substack app under Subscriptions → Podcast).
Table of Contents
What the evidence actually measures
Radiology has evidence that reaches beyond image accuracy
Cardiology connects measurements to clinical action
Gastroenterology finds more and inherits the consequences
Ophthalmology and dermatology show what screening leaves unfinished
Pathology and oncology need different kinds of proof
Emergency care and neurology expose the cost of an alert
General medicine still needs better human and AI teamwork
Ambient documentation has scaled faster than clinical autonomy
Mental health has promising results and unresolved safety questions
Surgery and diabetes clarify what autonomy requires
What changes for specialists and healthcare businesses
Abstract
Clinical AI has produced measurable gains in selected screening, diagnostic, and workflow applications, with a smaller set of studies demonstrating improved patient outcomes. Breast imaging, cardiac assessment, diabetic retinal screening, and prostate pathology offer concrete evidence of progress. Colonoscopy exposes the trade-offs between additional detection and additional intervention. Sepsis alerts and language-model deployments show how clinical usefulness can fall short of technical performance. Ambient documentation has achieved substantial adoption, while evidence for broad autonomous diagnosis, treatment selection, mental healthcare, and surgery remains more conditional. Across specialties, the emerging effects include changes in referral volume, review burden, training, and the distribution of clinical work. The commercial question is whether an implementation improves a complete care process at an acceptable cost, with benefits that survive outside the original study setting.
What the evidence actually measures
A hospital can install an impressive clinical AI product and still get almost nothing useful from it. The prediction can be right, the physician can agree, and the patient can receive exactly the same care as before. Alternatively, the system can improve a fairly ordinary task, such as getting a diabetic eye exam completed during an existing appointment, and produce a meaningful change in access. Those differences explain much of the uneven progress across specialties. Clinical AI now includes useful deployed products, credible randomized trials, expensive workflow experiments, and demonstrations that remain several operational steps away from treating anybody. Putting all of them in the same market forecast makes the spreadsheet look terrific.
The relevant question is what changed after the software entered the clinical workflow. Retrospective discrimination establishes whether a model separates cases in a dataset. Prospective validation tests whether that performance survives current patients and actual data feeds. A deployment study can establish whether clinicians use the output and whether turnaround times improve. A randomized clinical trial can test whether assigning care teams to the intervention changes outcomes. Even then, the endpoint matters. More detected lesions, better documentation, fewer admissions, and lower mortality represent very different achievements. A treatment recommendation judged sensible by specialists remains a recommendation until someone evaluates what happens when it guides care.
The practical inference from the stronger studies is that AI succeeds most reliably when the task has a defined input, a manageable set of outputs, and somebody responsible for acting. The software also needs a way to recognize when the case falls outside its competence. That combination is easier to build around screening images or a measurement than around an entire patient with contradictory records, multiple diseases, and a medication list last reconciled sometime during the previous presidential administration.
Radiology has evidence that reaches beyond image accuracy
Breast screening provides one of the clearest examples of clinical progress. The Swedish MASAI randomized trial assigned more than one hundred five thousand women to AI-supported screening or standard double reading. AI helped route examinations to single or double reading and highlighted suspicious findings. Cancer detection increased from five to six point four cancers per thousand screened, while the number of screen readings fell by about forty-four percent. The reduction applied to reading workload within that protocol. It didn’t mean a radiology department could remove forty-four percent of its staffing, because acquisition, diagnostic assessment, procedures, and other work remained.
The subsequent interval cancer analysis, published in two thousand twenty-six, made the result more informative. Interval cancers occurred at rates of one point five five versus one point seven six per thousand participants. The trial met its noninferiority criterion, but it didn’t establish a statistically significant reduction in interval cancers. Sensitivity improved from roughly seventy-four to eighty-one percent, with specificity unchanged. That supports a more efficient screening strategy without evidence of a worse interval cancer rate. It still leaves longer-term mortality and overdiagnosis questions open. Germany’s observational PRAIM deployment added scale: more than four hundred sixty thousand women, higher cancer detection with AI support, and no increase in recall. Because radiologists chose whether to use the system, its results carry different causal weight from randomization.
For breast imagers, the plausible near-term change is a redistribution of attention. Routine negative examinations consume less reading capacity, while additional findings create diagnostic work and potentially more biopsies. The economics depend on the starting workflow. Replacing part of a European double-reading process offers a different labor opportunity from adding another reader to an American practice already using a single radiologist. Across other imaging applications, the same diligence question applies: does the product eliminate a task, shorten it, or add another finding that somebody must investigate? A pulmonary nodule alert may be clinically valuable while increasing total imaging utilization. The payer and the radiology practice can look at that outcome and arrive at different conclusions about return on investment.
Cardiology connects measurements to clinical action
Cardiology has several advantages for AI deployment: repeated digital measurements, established confirmatory tests, and treatments that can follow a positive finding. In a blinded randomized echocardiography trial, cardiologists made substantial changes to initial AI estimates of ejection fraction less often than to initial sonographer estimates, and the AI workflow saved review time. This is a useful form of automation because the output is bounded and reviewable. The EAGLE randomized trial went further downstream. Providing clinicians with an AI interpretation of routine ECGs increased new diagnoses of ejection fraction at or below fifty percent from one point six to two point one percent. That is a modest absolute increase across the population, with potentially meaningful consequences for the patients found. It establishes improved detection; the trial did not establish that this screening strategy reduced heart failure admissions or prolonged survival.
There is also randomized evidence involving mortality. A trial of AI-enabled ECG alerts in almost sixteen thousand hospitalized patients reported ninety-day mortality of three point six percent with the intervention versus four point three percent with usual care. The hazard ratio was zero point eight three, with a confidence interval that only narrowly excluded no difference. This deserves attention precisely because patient survival was measured. The tested intervention included alert delivery and the clinical response, however, and the mechanism of benefit was not fully resolved. It would be a stretch to transfer that mortality estimate to another algorithm, patient population, or response protocol.
For cardiologists, screening creates a downstream capacity problem. An inexpensive risk score can generate echocardiograms, ambulatory monitoring, and specialist referrals much faster than a health system can add appointment slots. The useful product therefore needs to identify which additional evaluations are worthwhile and track whether treatment follows. Otherwise, the organization has automated the creation of a waiting list. The specialist’s work increasingly includes adjudicating discordant results and deciding whether a signal warrants intervention, especially when the patient has competing risks or limited ability to tolerate treatment.
Gastroenterology finds more and inherits the consequences
Colonoscopy AI has a credible detection story, but the clinical value depends on what gets detected. A multicenter randomized trial published in two thousand twenty-six evaluated a cloud-based system in more than eight hundred patients. Adenoma detection was about forty-three percent with assistance versus thirty-six percent without it, with additional gains in large polyps and sessile serrated lesions. That is encouraging because much of the debate concerns whether extra detections are confined to small lesions with limited incremental benefit. Yet another randomized trial in high-performing settings did not establish superior overall adenoma detection, despite finding more adenomas per examination. Its additional findings were largely diminutive lesions, and surveillance recommendations became more intensive. Baseline endoscopist performance and lesion mix clearly affect the marginal benefit.
This creates an awkward financial situation. Detecting additional lesions can mean more polypectomies, pathology specimens, and follow-up colonoscopies. Some of that activity prevents cancer; some adds cost and procedural exposure for small gains. Long-term benefit cannot simply be calculated by applying the historical relationship between physician adenoma detection rates and cancer incidence to every additional tiny lesion an algorithm finds. That relationship may not transport unchanged when the composition of detected lesions shifts. Evaluation needs to follow advanced neoplasia and interval cancers, alongside unnecessary resections and surveillance burden.
There is also an early warning about skill maintenance. A multicenter observational study found that adenoma detection during unaided colonoscopies fell from about twenty-eight to twenty-two percent after endoscopists had been exposed to AI assistance. The design doesn’t prove that AI caused deskilling, and temporal or behavioral differences could contribute. It does justify monitoring performance when assistance is unavailable. For gastroenterologists, the enduring skills include exposing the mucosa properly, recognizing difficult lesions, deciding what to remove, and managing complications. A detection overlay cannot rescue anatomy that never entered the camera’s view. Training programs will need to assess those capabilities directly instead of assuming that good assisted performance demonstrates preserved independent skill.
Ophthalmology and dermatology show what screening leaves unfinished
Autonomous diabetic retinal screening offers a practical answer to a common access failure: the patient is already in a diabetes clinic, but the separate eye appointment never happens. In the ACCESS randomized trial involving one hundred sixty-four young people with diabetes, screening completion reached one hundred percent with an AI examination at the point of care, compared with twenty-two percent after referral and education. That is an unusually large process improvement. Among participants with an abnormal AI result, however, only sixty-four percent completed follow-up with an eye care provider. Screening completion and completed treatment remain separate achievements. The study also evaluated a specific pediatric research population; it should not be read as a blanket statement about every product’s authorized age range or intended use.
For ophthalmologists, this approach could reduce routine screening visits while producing more referrals for actual disease. The referral pathway needs to handle ungradable images, urgent findings, and patients who disappear after receiving an abnormal result. Camera placement and staff training matter, but so does having retinal appointments available. A screening company that measures only completed scans can report excellent performance while the clinical program still loses patients before definitive assessment.
Dermatology makes the trade-offs especially visible. A prospective evaluation of AI in an urgent skin cancer pathway reported sensitivity of about ninety-five percent, compared with roughly eighty-nine percent for dermatologists, but a lower positive predictive value: about forty-seven versus sixty-two percent. Four cancers were found among lesions the AI had classified as benign and that could otherwise have been discharged without human review. Those results support careful triage evaluation, with attention to the selected referral population and verification process. They do not establish that autonomous assessment is safe across all skin presentations.
A dermatologist still has to turn a suspicious image into a diagnosis and a management decision. Earlier research using the Diverse Dermatology Images dataset also demonstrated performance problems on darker skin and uncommon conditions. Local validation therefore needs enough examples of the patients actually served. Average accuracy can hide poor performance for a smaller subgroup, while high sensitivity can produce enough false positives to consume the very appointments the deployment was supposed to free.
Pathology and oncology need different kinds of proof
Digital pathology is beginning to produce operational evidence that goes beyond retrospective slide competitions. The prospective Articulate Pro study evaluated prostate biopsy assistance across three English specialist centers. In the staged second-read workflow, AI prompted changes in diagnosis or grade group for about five percent of patients, although only about one percent had changes judged potentially relevant to management. Concurrent assistance reduced mean turnaround time by roughly thirty hours at one site, and the need for immunohistochemistry fell across all sites. The variation matters. The center with the clearest turnaround improvement had mature digital infrastructure and full integration, while a site’s existing double-reporting process could constrain the effect on release times.
For pathologists, the immediate opportunity is more efficient review and better targeting of ancillary work. Whether that produces savings depends on scanners, image quality, storage, laboratory systems, and how results move through sign-out. A laboratory still operating largely on glass faces a substantially different investment from one that already has a functioning digital workflow. The algorithm’s license fee is only the beginning of that budget.
Oncology then adds a harder question: can AI identify which treatment will help? A model that predicts recurrence is prognostic. A model that identifies differences in treatment benefit is predictive. Those claims are often blurred in commercial presentations. A multimodal prostate cancer model combined clinical information with pathology images and was validated using tissue and outcomes from a completed randomized radiotherapy trial. The locked model identified a subgroup with a greater apparent benefit from short-term androgen deprivation, with a statistically significant treatment interaction. This is stronger evidence for treatment selection than showing that an image score correlates with survival.
The treatment comparison came from randomization, but use of the biomarker did not. Patients weren’t prospectively assigned to an AI-guided treatment strategy. A nonsignificant benefit within a model-negative subgroup also does not prove that every patient in that subgroup can safely skip therapy. Contemporary treatment patterns, available tissue, and subgroup uncertainty affect applicability. For oncologists and pharmaceutical companies, the commercial opportunity is substantial if a validated test improves selection or avoids toxicity. Procurement should demand evidence that the test changes a decision and improves its consequences. Predicting a molecular feature from a slide, on its own, cannot establish equivalence to the assay used to select a drug.
Emergency care and neurology expose the cost of an alert
Sepsis prediction illustrates how a model can improve statistically while remaining demanding operationally. A multicenter evaluation of the updated Epic Sepsis Model, published in two thousand twenty-six, included more than two hundred twenty-seven thousand encounters. Encounter-level discrimination ranged from zero point eight two to zero point nine two across institutions. At thresholds chosen for sixty percent sensitivity, positive predictive values were only thirteen to twenty-six percent. Repeated predictions over short time horizons performed differently again. The hospital still needs to know how many alerts arrive during a shift, how early they arrive relative to clinical recognition, and how many require evaluation before a useful intervention follows. A strong area under the curve does not answer those questions.
COMPOSER offers a more encouraging example. At two emergency departments, deployment was associated with an absolute reduction in sepsis mortality of about one point nine percentage points and improved bundle compliance. The study used a before-and-after design with adjustment, so changes in staffing, practice, or other unmeasured factors remain possible explanations for part of the effect. The result supports further evaluation of that care process. Claiming the same mortality benefit for the algorithm wherever it gets installed would go well beyond the evidence. For emergency physicians, intensivists, and nurses, the clinical workload includes examining the patient, deciding whether infection is plausible, and avoiding unnecessary treatment of sepsis mimics.
Stroke AI has a more concrete workflow target: get a potentially treatable large-vessel occlusion to the right team faster. In a retrospective implementation study involving more than two hundred thrombectomy patients, median door-to-groin puncture time fell from one hundred nine to seventy-five minutes after deployment. Mortality or hospice discharge did not differ significantly. That combination supports a workflow benefit while leaving uncertainty about the patient outcome attributable to the software. The imaging flag, mobile communication, transfer coordination, and team activation were part of a service.
For vascular neurologists and neurointerventionalists, faster recognition can expand the number of patients reaching assessment in time. It also creates pressure on transport and procedural capacity. An automated flag still needs clinical interpretation, particularly when imaging artifacts, an uncertain last-known-well time, or substantial baseline disability complicate the decision. Measuring the entire pathway through treatment and functional recovery provides a much more useful account of progress than timing how quickly the notification appeared.
General medicine still needs better human and AI teamwork
Language models have made progress on clinical reasoning, but their value changes with the task and the way clinicians use them. In a randomized study of fifty physicians, access to a language model did not significantly improve diagnostic reasoning scores on clinical vignettes, even though the model performed better on its own. A separate randomized study involving ninety-two physicians found an improvement of six point five percentage points on management reasoning tasks, accompanied by roughly two additional minutes per case. Both studies used simulations. Together they suggest that access to a capable model is insufficient, and that better deliberation may sometimes take longer.
More recent subspecialty work is encouraging. A randomized evaluation using retrospective data from one hundred seven patients with suspected genetic cardiovascular disease involved nine general cardiologists. AI-assisted assessments were preferred by blinded subspecialists and contained fewer clinically significant errors, about thirteen percent versus twenty-four percent. This demonstrates a useful form of assistance on complex case review. It does not establish better outcomes in prospectively treated patients, and the small physician sample limits generalization. The implication for specialists is worth exploring: some expertise may become more accessible to generalists, while subspecialists concentrate on uncertain cases, unusual phenotypes, and decisions where competing risks dominate.
Actual deployment can be less accommodating. A prospective emergency department evaluation of a language-model decision-support system followed more than one thousand patients over four weeks. Use declined from sixty-eight to thirty percent, although expert review judged nearly all sampled outputs clinically appropriate. Length of stay was unchanged. As workload increased, clinicians disengaged. The answers looked good; getting busy clinicians to keep using them was another matter.
Then there’s the unglamorous work of reconstructing what was actually known when a decision was made. A discharge diagnosis is unavailable at admission. A medication mentioned in an old note may have been stopped. A missing allergy entry is different from a documented absence of allergies. Retrieval can supply current evidence, but the system still has to preserve dates, reconcile contradictions, and identify uncertainty. Internists and primary care physicians may benefit most when the product removes specific information gaps or prepares a reviewable decision, without requiring them to investigate another long answer during an already crowded visit.
Ambient documentation has scaled faster than clinical autonomy
Ambient scribes offer one of the clearest examples of enterprise adoption because the task is frequent and the burden is immediately recognizable. The Permanente Medical Group reported 2,576,627 encounters involving 7,260 physicians during a 63-week evaluation from October 2023 through December 2024. The program estimated 15,791 hours of documentation-time savings, equivalent to 1,794 eight-hour workdays. Usage and benefit were concentrated: the highest-using third of physicians accounted for nearly ninety percent of activations, and heavy users achieved substantially greater time savings per note than occasional users. Those figures establish substantial real-world use, but they also show why purchased seats and activated clinicians are weak measures of success. Specialty, encounter type, and baseline documentation habits affect whether the software becomes part of daily practice.
The evidence has since broadened beyond the Permanente rollout. A randomized trial involving 238 outpatient physicians across 14 specialties compared two commercial ambient-scribe tools with usual workflow and found vendor-specific effects: one tool reduced time-in-note by about 9.5%, while the other produced an approximately 1.7% reduction that was not statistically significant versus controls. That result reinforces the article’s larger point: enterprise adoption is not the same as a uniform productivity gain, and the product, workflow, and user population matter.
Randomized evidence gives a more restrained view of the productivity claim. In a trial of two hundred thirty-eight outpatient physicians across fourteen specialties and roughly seventy-two thousand encounters, Nabla reduced time spent writing notes by about nine and a half percent relative to control. Microsoft DAX Copilot reduced time-in-note by about one point seven percent, a difference that was not statistically significant. Both products showed signals of improved clinician experience, although secondary findings needed confirmation. Clinicians reported occasional clinically significant inaccuracies with either tool. A clinician can value less typing and better eye contact even when the measured time saving is small.
The trial’s practical lesson is narrower than the adoption numbers suggest. Ambient documentation can reduce a defined documentation burden, but the effect varies by product and does not demonstrate improved diagnostic accuracy, patient outcomes, or broad clinical autonomy. Enterprise buyers should therefore evaluate actual note time, adoption over repeated encounters, correction burden, accuracy, privacy, and downstream workflow rather than treating deployment volume as proof of clinical value.
The boundary between documentation and clinical action deserves attention. Drafting a narrative is different from populating an active medication list, suggesting a diagnosis, or generating an order. Errors become harder to contain as outputs move from reviewable text into structured fields that trigger other workflows. Attribution also matters: a family history of a condition must not become the patient’s diagnosis, and a medication considered during discussion must not become an active prescription.
For specialists, the benefit may be less evening documentation rather than additional billable visits. Finance teams should value that honestly. Retention, reduced exhaustion, and a less distracted consultation can justify spending, but they are different from cash savings. Turning every saved minute into another scheduled patient may also remove the benefit that encouraged clinicians to adopt the tool.
Mental health has promising results and unresolved safety questions
The Therabot randomized trial provides a credible signal that a purpose-built generative system can help some patients. Two hundred ten adults with clinically significant depression or anxiety symptoms, or elevated eating-disorder risk, were assigned to the intervention or a waitlist. Symptoms improved more with the chatbot, and participants reported substantial engagement over the study period. The trial evaluated a carefully developed intervention with a defined research and safety structure. Its comparator was a waitlist, rather than an active course of psychotherapy. Positive ratings of therapeutic alliance therefore cannot establish equivalence to a therapist, and short follow-up leaves relapse and sustained benefit unresolved.
The safety problem extends beyond factual errors. A system can respond fluently while reinforcing a harmful interpretation of events. Experimental evaluations of general-purpose language models have documented stigmatizing responses and inappropriate reinforcement of delusional thinking. Those studies identify failure modes; they do not measure the incidence of harm across all deployed mental health products. Product-specific assessment remains necessary. Performance during a short, cooperative exchange also says little about behavior during repeated conversations with someone whose symptoms are changing.
For psychiatrists and therapists, a plausible clinical role is support between appointments: structured exercises, symptom tracking, and help practicing skills already established in treatment. That opportunity depends on whether deterioration reaches a responsible care team and whether someone can respond. An automated escalation message won’t help much if it lands in an unattended inbox. The business model therefore needs to account for clinician coverage and continuity, not merely the cost of generating text. Trials should examine functioning, adverse events, treatment engagement, and durable symptom change alongside satisfaction. A product can be pleasant to use and still be poorly equipped for the moments when clinical judgment matters most.
Surgery and diabetes clarify what autonomy requires
Surgical AI demonstrations deserve careful description because the experimental setting carries so much of the safety burden. A hierarchical robotic system reported fully autonomous performance of studied cholecystectomy steps across eight ex vivo gallbladders. That is a meaningful engineering achievement in planning and error recovery. It did not demonstrate independent surgery in living human patients. Bleeding, variable anatomy, physiological deterioration, unexpected pathology, and the need to abandon the original plan create a very different problem. Surgeons should evaluate autonomous capabilities step by step, with explicit boundaries and rescue requirements, instead of treating a successful laboratory sequence as a forecast for replacing an operating team.
Diabetes provides a useful comparison because bounded therapeutic automation has stronger clinical evidence. In a randomized trial of more than three hundred adults with insulin-treated type two diabetes, automated insulin delivery improved glycated hemoglobin by an adjusted six-tenths of a percentage point relative to the control strategy, with both groups receiving continuous glucose monitoring. Time in the target glucose range improved by fourteen percentage points. These are meaningful metabolic outcomes over a thirteen-week trial, although they do not directly establish reductions in long-term complications. One severe hypoglycemia event occurred in the automated-delivery group.
Not every automated insulin controller uses machine learning, so those results should not be repackaged as proof that generative AI can prescribe safely. They illustrate the value of frequent measurements, a constrained action space, and defined operating limits. The device receives feedback about the consequences of its actions, although sensors and infusion systems can still fail. For endocrinologists, work shifts toward patient selection, initiation, troubleshooting, and management when physiology or circumstances exceed the controller’s assumptions. For surgeons, the analogous challenge is deciding which steps can operate within similarly testable limits. In both specialties, supervising automation is an additional clinical skill that requires its own training.
What changes for specialists and healthcare businesses
The evidence supports a task-level view of specialist impact. Reading, measuring, drafting, and identifying candidates for review can become faster. More detected disease can then increase demand for consultations, confirmatory testing, and procedures. A reasonable near-term expectation is that many specialists will see a different mix of work, with less routine production and more exceptions, verification, and treatment decisions. Whether that improves the working day depends on how organizations staff the remaining tasks. Removing easy cases can increase the complexity of every hour without reducing the hours themselves.
For healthcare businesses, a signed enterprise contract is the beginning of the operational test. Evaluation should track use across eligible encounters, the amount of correction required, completed downstream actions, and performance across patient groups. Time savings need to survive review, rework, integration support, and software downtime. A hospital may gain capacity without reducing payroll, which can still be valuable when access is constrained. A payer may see higher near-term spending from additional detection before any benefit appears in avoided complications. A laboratory may save on ancillary testing while absorbing new scanning costs. These are specific economic mechanisms that need measurement.
Contracts should make ongoing evaluation possible. Buyers need access to relevant outputs and audit data, clear notice of material model changes, and a practical way to suspend the tool. Regulatory authorization is tied to a device’s intended use and supporting evidence; it doesn’t establish that every workflow built around the device will improve care. Local assessment should include a period of prospective observation before outputs influence care, followed by monitoring of both errors and clinical consequences. Automated performance reports will not discover missed cancers or failed referrals unless the organization collects the downstream outcomes.
The strongest commercial position is likely to belong to companies that can repeatedly deliver a useful clinical result within those constraints. That may involve less glamorous work than building another demonstration: handling an ungradable image, getting a positive result to the right clinic, or making sure an updated model still behaves properly on a particular scanner. Specialists have good reason to welcome tools that remove real burdens. They also have good reason to ask exactly which burden disappeared, who inherited the new work, and whether patients ended up better off.
