🎧 Part I Podcast free on Apple Podcasts and Spotify.
🎧 Part II Podcast episode for paid subscribers only. Also available on Apple Podcasts and Spotify.
To listen to paid episodes in Apple or Spotify, link your Substack subscription via the show settings on those platforms (instructions inside the Substack app under Subscriptions → Podcast).
Table of Contents
The leaderboard needs a medical chart
A correct answer can still produce bad medicine
Doctors and models need a better working arrangement
The strongest evidence comes from narrower jobs
The medical record is part of the diagnostic engine
The regulatory boundary runs through the product
The payment model decides who buys
Where a defensible diagnostic business can be built
The physician role changes case by case
Abstract
Artificial intelligence is getting much better at diagnostic reasoning, but the commercial implications depend on what happens after the answer appears. Benchmark design, patient selection, physician behavior, record quality, regulatory classification, and reimbursement can each change whether an impressive model produces useful care. Clinical studies already support adoption in specific screening workflows, while general diagnostic assistance has produced mixed results. The strongest business opportunities involve defined patient populations, verifiable recommendations, and responsibility for completing the next clinical step. Falling model costs help, although physician review and unnecessary downstream testing can easily overwhelm those savings. The likely outcome is substantial redistribution of clinical work, with different levels of automation across individual tasks.
The leaderboard needs a medical chart
The current diagnostic AI conversation could use someone checking the arithmetic before declaring another medical revolution. The November RareBench leaderboard places Gemini Four Hydrogen at forty-nine point two percent and Claude Opus Five Point Five at fifty-one point six percent. Across the benchmark’s one hundred twenty-two cases, that means sixty versus sixty-three cases meeting the selected scoring standard. Hydrogen leads Opus on LabBench, at forty-six percent versus forty point two percent, but LabBench evaluates experimental reasoning. Combining those results into a blanket claim about superior clinical diagnosis mixes different tasks and different scoring systems. RareBench’s selected metric measures whether all required causal findings appear within a ranked list sized to the number of required findings. That’s useful information about a system’s ability to identify and prioritize findings in rare genetic disease cases. It doesn’t establish the probability that an unselected patient walking into urgent care will receive the right diagnosis.
Apple’s collaboration with OpenEvidence requires a similar distinction. Its newly announced Apple Health AI Usage Study, launched November 15, 2026, can enroll up to two hundred thousand eligible U.S. adults aged twenty-one or older. It examines how people understand and evaluate AI-generated health information, what makes them trust it, and how it influences everyday health actions, including whether it helps them prepare for appointments with their doctors. Participants can optionally share certain HealthKit data. Apple describes the app as a research tool for everyday health questions, not a doctor or a substitute for professional diagnosis or treatment. That makes the announcement interesting as a study of consumer behavior and health information delivery. It provides no evidence yet that the app improves diagnostic outcomes. Still, distribution deserves attention. A tool available where patients already manage their daily lives could influence which symptoms they mention, which records they assemble, and which questions enter an appointment. Those effects could improve care or generate additional clinical work. The study has to distinguish between them.
Benchmark results also describe a particular configuration. The model operates with a prompt, tools, retrieval sources, execution environment, and time allowance. Change those inputs and the product changes. A system that searches variant databases for half an hour has a different job from one expected to help during a short office visit. Published clinical cases can contain clues selected precisely because they explain the eventual diagnosis; production records contain abandoned hypotheses, copied errors, and information that arrived after the decision under evaluation. Useful diligence therefore starts with the encounter being modeled. What information existed at that moment? What could the clinician reasonably obtain? What action did the system recommend? A leaderboard becomes commercially informative when those details resemble the work a buyer needs done.
A correct answer can still produce bad medicine
Diagnosis is a sequence of decisions under uncertainty, with consequences attached to each additional test. A model can name the eventual disease and still recommend an expensive, invasive, or unnecessarily prolonged route to confirmation. Conversely, a system might omit the final diagnosis initially while recommending the appropriate referral that gets the patient there quickly. A hospital should care about that distinction. Evaluations need to measure diagnostic yield alongside time to appropriate action, unnecessary testing, missed urgent conditions, and clinician workload. They also need to distinguish a useful differential from a long list containing the answer somewhere near the bottom. Naming enough diseases eventually becomes a strategy for winning the grading rubric while losing the afternoon.
Base rates make deployment especially unforgiving. Consider a hypothetical disease affecting one patient in a thousand. A screening system with ninety percent sensitivity and ninety-nine percent specificity sounds excellent. Applied to ten thousand people, it would identify about nine true cases and generate roughly one hundred false positives. Fewer than one in ten positive results would represent actual disease. Apply the same hypothetical performance to a carefully selected population with ten percent prevalence, and more than nine in ten positive results would be true positives. The model hasn’t improved. Patient selection has changed the meaning of its output. Actual sensitivity and specificity can also change across populations, which adds another reason to validate locally. A rare disease detection business needs a credible way to identify the population in which further investigation makes sense.
The testing decision should turn on whether new information can change management enough to justify its cost and risk. That requires knowing what happens after the result. A suggested genetic test has different value when counseling, confirmatory interpretation, and a treatment pathway are available. An imaging recommendation has different value when the patient has already undergone the relevant study elsewhere. Negative findings matter as well, although their meaning depends on whether something was examined, tested, denied by the patient, or simply never documented. A model that treats all four as equivalent can produce a beautifully explained mistake. The practical evaluation question is whether the complete workflow improves decisions at an acceptable cost, including the burden imposed on patients who turn out to be healthy.
Doctors and models need a better working arrangement
Evidence on physician assistance has already complicated the assumption that putting a capable chatbot beside a doctor automatically improves performance. A randomized study published in JAMA Network Open in two thousand twenty-four enrolled fifty physicians working through challenging clinical vignettes. Median diagnostic reasoning scores were seventy-six percent with access to the model and seventy-four percent with conventional resources, a difference that wasn’t statistically significant. The model tested alone achieved a median score of ninety-two percent. These were rubric scores on a small set of cases, not comprehensive measures of clinical competence. Nevertheless, the result identifies a product problem: strong standalone performance didn’t reliably transfer into better physician performance under the study’s conditions. Access to useful information and effective use of that information are separate achievements.
A randomized trial published in Nature Health in two thousand twenty-six found a much larger benefit among physicians in Pakistan who had completed an AI literacy curriculum. Among fifty-eight participants completing the study, those with model access achieved mean diagnostic reasoning scores of seventy-one point four percent, compared with forty-two point six percent using conventional resources. The adjusted difference was twenty-seven point five percentage points, without a significant difference in case review time. That supports the possibility of substantial assistance in the studied setting. It doesn’t establish that training caused the improvement, because training itself wasn’t the randomized intervention. Nor does it settle performance in routine practice. The contrast with the earlier American trial argues for examining users, cases, baseline resources, and interaction design before attaching a universal productivity assumption to a model subscription.
Training also doesn’t eliminate susceptibility to bad advice. Another randomized study published in two thousand twenty-six involved forty-four physicians who had completed twenty hours of AI literacy training. Participants receiving model suggestions containing deliberately introduced errors performed worse than participants receiving error-free suggestions, with an adjusted fourteen percentage point reduction in diagnostic reasoning accuracy. This wasn’t a measurement of the natural frequency of model errors, and its comparator wasn’t unaided practice. It tested what happened when misleading suggestions entered the workflow. The lesson for product design is uncomfortable but useful: a clinician’s ability to reject an answer cannot be assumed from credentials or training alone.
A better interface would make disagreement specific. It could show which finding supports a recommendation, which finding contradicts it, and which missing datum would most change the proposed next step. Some workflows may benefit from obtaining the clinician’s initial assessment before displaying the model’s suggestion. Others may benefit from early assistance when the clinician lacks relevant expertise. Those designs need testing against the actual users and task. Agreement rates are a poor substitute. A system that increases confidence without improving decisions can look successful in a satisfaction survey while making supervision harder.
The strongest evidence comes from narrower jobs
Mammography provides a useful example because the intervention can be defined and evaluated within an established screening pathway. In the Swedish MASAI randomized trial, involving more than one hundred five thousand women, AI supported triage to single or double reading and helped with detection. An earlier analysis found twenty-nine percent higher cancer detection, with rates of approximately six point four versus five cancers per thousand screened participants, a similar false-positive rate, and roughly forty-four percent less screen-reading workload. That is an operationally meaningful combination. It also describes a specific screening system with a particular comparator. The workload reduction refers to screen reading, rather than total breast imaging labor, and transferring the result into another country requires understanding how its reading practices and staffing differ.
The subsequent interval cancer analysis strengthens the clinical case while preserving important limits. Interval cancer rates were approximately one point five five versus one point seven six per thousand participants, meeting the trial’s noninferiority criterion. Sensitivity was eighty point five percent with AI support versus seventy-three point eight percent with standard reading, while specificity was ninety-eight point five percent in both groups. The numerical reduction in interval cancers didn’t establish statistically significant superiority for that outcome. These results support a particular screening workflow; they don’t establish a mortality benefit or the replacement of radiologists. The implementation question is how to reproduce the measured improvements while maintaining reliable image acquisition, recall assessment, and follow-up.
Autonomous diabetic eye screening illustrates a different mechanism of value. In the ACCESS randomized trial involving young people with diabetes, point-of-care autonomous screening produced eye examination completion of one hundred percent in the intervention group, compared with twenty-two percent after conventional referral and education. This was a pediatric research setting, so the result shouldn’t be casually generalized to every population or device indication. Its practical significance is that the service was available during an appointment the patient was already attending. Even then, only sixty-four percent of participants with abnormal intervention results completed the indicated eye care follow-up. Completing a screening test and completing the resulting care remain distinct problems.
These examples suggest where adoption may advance fastest: tasks with defined inputs, a manageable set of outputs, and a clear destination for exceptions. General diagnostic assistance has a much larger search space and less predictable follow-through. A screening workflow can specify what an unreadable image means, who repeats it, and where an abnormal result goes. A general diagnostic chatbot can recommend a specialist who has no appointments, a test the insurer won’t cover, or a workup already performed under a different name. The commercial advantage of a narrow task is that the company can identify those failure points and take responsibility for fixing them.
The medical record is part of the diagnostic engine
Uploading an entire chart sounds straightforward until the chart contains several competing versions of reality. A medication may appear active because nobody discontinued the order. A suspected diagnosis may have been copied forward until it resembles an established condition. A symptom may have begun months before its first documented appearance. A scanned specialist letter may contain the decisive finding while the structured problem list contains little of value. For a diagnostic product, reconciliation is part of the clinical work. The system needs to preserve who reported a fact, when it was observed, whether it was confirmed, and how confident the source was. Compressing that history into a tidy summary can remove precisely the uncertainty the clinician needs to see.
Interoperability standards help move information but leave substantial interpretation work to the application. Fast Healthcare Interoperability Resources, commonly called FHIR, can represent observations, diagnostic reports, and associated context. CDS Hooks provides a standard pattern for invoking decision support at selected points in a clinical workflow. Neither capability guarantees that a connected application receives a complete longitudinal record or understands the clinical meaning of an imported result. A diagnostic report can arrive successfully while the underlying images remain unavailable. A laboratory result can be present without the surrounding treatment history needed to interpret it. Integration quality therefore needs clinical acceptance criteria alongside technical ones. Successfully receiving data is an intermediate milestone, even if the integration team has already ordered celebratory pizza.
The record also creates an evaluation trap. Retrospective testing must reconstruct what was available before the decision, excluding later diagnoses, pathology, and explanatory notes. Otherwise, the model may appear to detect a disease early because someone already documented it late. Production systems face the reverse problem: their own suggestions can enter subsequent documentation and return as apparently independent evidence. A proposed diagnosis copied into a note should retain its status as a proposal. The product needs to distinguish original findings from repeated interpretations and preserve the model version behind consequential recommendations. That makes review possible when performance changes or an adverse event requires reconstruction.
Patient-directed data sharing adds another distinction. Health information entered into a consumer application doesn’t automatically receive the same federal privacy protections that apply when a covered provider or its business associate maintains it. The applicable obligations depend on the entities and relationships involved, with other laws potentially applying outside HIPAA. A diagnostic business should make its data handling understandable before inviting patients to upload years of sensitive records. The practical questions include retention, secondary use, deletion, and which parties can access the information. A reassuring privacy screen is useful only if the operating arrangements support it.
The regulatory boundary runs through the product
The Food and Drug Administration evaluates software functions and intended uses, which makes product boundaries commercially consequential. The statutory exclusion for certain clinical decision support requires, among other conditions, that the healthcare professional can independently review the basis for recommendations and isn’t intended to rely primarily on them. The agency’s January two thousand twenty-six guidance also describes enforcement discretion for certain functions that provide only one clinically appropriate recommendation while otherwise meeting the relevant criteria. That is different from being outside the device definition. Software that analyzes medical images or signal-level inputs raises a different classification question from software working with established medical information. Likewise, supporting an urgent decision can limit the clinician’s opportunity for independent review. A physician approval button doesn’t settle any of these issues.
This puts pressure on how a diagnostic product presents its work. A plausible explanation generated after an answer doesn’t, by itself, establish that the recommendation is sound. The user needs access to relevant inputs, important omissions, and evidence appropriate to the intended use. If a recommendation depends on a finding that the product cannot verify, that limitation should be visible where the decision occurs. For a regulated device, updating the underlying model also needs deliberate change management. The agency’s predetermined change control plan framework allows specified modifications to be reviewed in advance, including how they will be developed, validated, and implemented. It doesn’t authorize an unrestricted sequence of model substitutions merely because the next release performs better on a public benchmark.
Certified health information technology introduces another set of obligations. Under the applicable decision support certification framework, developers supplying predictive interventions as part of their certified modules face requirements concerning source information and intervention risk management. These include relevant performance, validation, maintenance, and risk characteristics. The scope matters: this is neither a universal certification requirement for every chatbot nor an approval of an intervention’s clinical effectiveness. For buyers, the information can support a more concrete procurement discussion about where a model was evaluated and how its performance will be monitored after deployment.
Liability remains a separate negotiation. In November two thousand twenty-six, the American Medical Association called for legislation aligning responsibility with the parties able to prevent harm, including vendors and organizations mandating particular tools. That is a policy position, not an enacted nationwide allocation of liability. Contracts still need careful treatment of intended use, product failures, mandated workflows, and the evidence available to clinicians. A company cannot credibly sell greater autonomy while describing every failure as inadequate human supervision. The review workload promised in the sales process should match the authority and responsibility assigned in the contract.
The payment model decides who buys
A diagnostic system can improve care while producing an awkward business case for the organization expected to buy it. Earlier recognition may help a health plan avoid an expensive admission, while requiring a medical practice to perform additional work it cannot separately collect for. A specialty group may value shorter chart review if the time can support more appointments. A hospital may value detection that brings patients into an appropriate service line, although that benefit looks different to the payer financing the additional care. The relevant question is who receives enough measurable value to fund the workflow, including the labor required to act on its recommendations. The answer often determines the initial customer more clearly than the model’s clinical breadth.
Cheap inference helps, but the economically useful unit is the completed episode. Consider an explicitly hypothetical service charging fifty dollars for a diagnostic review. The model might cost one dollar to run, yet the case could require ten minutes of physician review. At an assumed physician cost of two hundred forty dollars an hour, that review alone costs forty dollars. Record retrieval, scheduling, support, and cases requiring escalation would consume the remainder quickly. The example isn’t an estimate of any company’s economics. It illustrates why token prices reveal little without the distribution of human work. A small proportion of difficult cases can dominate cost if each requires extensive review, additional records, and repeated patient contact.
Deployment evidence also counsels against assuming that a billing pathway creates rapid adoption. A Medicare study published online in two thousand twenty-six found that observed autonomous diabetic retinal screening services in the analyzed public files increased from one hundred forty-three in two thousand twenty-one to one thousand four hundred twenty-seven in two thousand twenty-three. Billing providers increased from five to thirty-nine. The authors characterized this as early uptake with small absolute use. Those files describe a limited population and period, rather than the entire market, but the finding is still useful. A service can have authorization and a dedicated billing code while facing substantial implementation friction. Cameras, staff training, eligible patient volume, collection workflows, and referral arrangements still need to work together.
Payment design should reward a clinical result the buyer can verify. Depending on the service, that might be an appropriately completed screening episode, a resolved referral, or reduced specialist preparation time without deterioration in quality. Shared savings requires credible attribution, adequate follow-up, and adjustment for changes in patient mix. Paying for additional diagnoses creates a different incentive from paying for better decisions. Particularly in risk-bearing populations, an increase in documented disease may improve revenue before anyone has established that care improved. Investors should separate those mechanisms when assessing growth.
Where a defensible diagnostic business can be built
Rare disease evaluation is a plausible opportunity because the work often involves assembling a longitudinal account across specialties and identifying which findings deserve another look. A company could choose a defined referral population, reconcile prior investigations, organize phenotype information, and prepare a reviewable case for genetics specialists. The defensible service would include access to appropriate confirmation and a process for communicating unresolved findings. It would also preserve the distinction between a candidate explanation and an established diagnosis. Genetic interpretation makes that especially important: professional guidance states that a variant of uncertain significance should not drive clinical decision-making. Fluent biological storytelling cannot upgrade uncertain evidence into a treatment indication.
Specialty intake may offer a more manageable starting point than general autonomous diagnosis. A renal genetics service, for example, could help identify missing family history, locate previous testing, and organize records before specialist review. The service could measure preparation time, incomplete referrals, appointment utilization, and time to an appropriate next step. Some functions would be administrative; others could involve clinical decision support or regulated activity depending on their design and intended use. The company would need to define those boundaries honestly. The opportunity comes from improving a specific bottleneck where the customer already experiences delay and expense. Expanding into adjacent tasks becomes easier once the company understands the exceptions, referral patterns, and clinical review burden in that setting.
A separate opportunity exists in following abnormal findings through to resolution. Software could identify a potentially actionable result, establish who owns follow-up, track whether the patient was contacted, and surface unresolved cases for review. This requires reliable interpretation and operational coordination, with clinical validation appropriate to the task. Its commercial appeal is that failure can often be described concretely: a recommendation existed, but the expected action didn’t occur. The customer can evaluate whether the service improves completion without flooding teams with duplicate alerts. The company would need access to outcomes and enough workflow control to distinguish a completed outside evaluation from a genuinely missed follow-up.
Pharmaceutical sponsorship could support some diagnostic services where delayed recognition prevents appropriate treatment, but the incentive requires scrutiny. A sponsor-funded program needs clinically defensible selection criteria, independent assessment, and a clear separation between diagnostic evaluation and promotion of the sponsor’s therapy. Otherwise, the system risks producing a disproportionate number of explanations that happen to end at the same drug. Health plans and providers will also want to know whether additional testing improves management or primarily expands a commercial funnel. A durable business needs evidence that remains persuasive when the sponsor’s preferred product is removed from the equation.
Across these opportunities, the strongest assets are likely to include dependable record access, specialist participation, distribution into an existing workflow, and rights to measure subsequent outcomes. A collection of conversations is less useful than a well-governed record linking recommendations to review decisions and clinical results. Even then, labels need care. A diagnosis code isn’t always confirmation, and an uncompleted test doesn’t establish that the suggested disease was absent. Companies that invest in outcome verification may develop useful operational knowledge competitors cannot obtain simply by switching to the newest model.
The physician role changes case by case
Predicting which specialty disappears first is an entertaining pastime with a poor unit of analysis. A specialty contains many tasks with different requirements. In radiology, detecting a finding on an image is one part of work that can also include choosing protocols, resolving uncertain findings, communicating urgent results, and performing procedures. Pathology involves specimen quality, additional testing, interpretation, and discussion of what a result means for treatment. Some tasks may support substantial automation before others do. Buyers therefore need to estimate labor changes at the task level and account for what happens when the system cannot handle a case. An automated first pass can save time, create additional review, or change the skill required for the remaining work.
Physician review deserves the same specificity. Checking whether a system retrieved the correct laboratory value is different from independently reconsidering a complex diagnosis. The latter can require much of the original work. If every case needs that depth of review, a business should include it in staffing and pricing assumptions. If only selected cases need it, the company must validate the selection process. Confidence expressed in fluent prose is insufficient. Useful escalation depends on observable conditions such as missing records, contradictory findings, unsupported inputs, or a case outside the evaluated population. A system also needs a safe operational response when it abstains. Sending the case to an already overwhelmed inbox doesn’t complete the handoff.
The near-term investment case is therefore strongest when the company can identify a particular clinical decision, demonstrate an improvement in the setting where it will be used, and organize the work required afterward. Better models widen the range of tasks worth attempting. They also make superficial differentiation harder, because a competitor can buy access to similar reasoning capabilities. The harder achievement is delivering a reliable service around those capabilities at a price someone will pay. For a patient waiting on an unexplained finding, the useful result is an appropriate next step that actually occurs. Someone still has to retrieve the missing report, review the exception, secure the appointment, and make sure the patient gets there.


