AI is rapidly creating a parallel, consumer-driven medical system outside traditional clinics. While recent studies show that advanced AI models can perform impressively on selected clinical reasoning tasks, the authors warn that these results are increasingly being interpreted as evidence that doctors themselves are becoming unnecessary. At the same time, consumers can now combine AI chatbots, diagnostic testing, health-monitoring devices, and online prescription services to manage parts of their health without seeing a physician.
The article highlights how this ecosystem is already taking shape. Oura, for example, offers a 50-biomarker blood panel through Quest Diagnostics, while Function Health provides extensive testing and allows members to have ChatGPT analyze results. Ro and Hims can provide certain prescriptions following online intake processes, and Doctronic says it has conducted millions of AI consultations. These services can make health care more accessible and convenient, but they also shift medical decision-making toward consumers who may not have the expertise to judge the quality or significance of what AI tells them.
A central warning is that performing well on a completed clinical case is different from practicing medicine from the beginning of a patient's encounter. The authors describe their evaluation of 21 frontier AI models, where models identified the correct diagnosis more than 90% of the time when given a complete case, but failed to produce a comprehensive differential diagnosis more than 80% of the time when given only the information normally available at the beginning of a clinical visit. Real patients provide fragmented symptoms, and doctors must determine what information matters, what questions to ask, and which possibilities should not be overlooked.
The broader issue, according to the authors, is accountability rather than raw intelligence. A doctor remains responsible when a clinical decision causes harm, whereas current AI systems have no comparable professional responsibility. The authors argue that AI could instead strengthen medicine by helping patients understand conditions, integrating long-term health information, supporting clinicians between visits, and reducing administrative work. Their concern is that commercial incentives could turn AI into a “shadow medical system” that adopts the authority of medicine without accepting the responsibilities that come with it.
