The most interesting part of Microsoft’s new medical AI is not the accuracy number. It’s that the system finally gets tested the way medicine actually works: no tidy summary, no multiple-choice answers, just a patient and a list of things you could ask or order.
Microsoft researchers reported in Science that their tool, which diagnoses sequentially, was more than twice as accurate as general physicians on more than 300 highly unusual and confusing cases. The physicians were allowed textbooks and the internet. The tool also beat human doctors on scenarios drawn from a major database of emergency room admissions.
The mechanism is what Microsoft calls an orchestrator. It runs a simulated panel of clinical experts that collaborate on the case. The system first decides what to ask about the patient’s history and what matters in the physical exam. Then it picks tests, reads the results, and orders more if needed. All the while it updates its working hypotheses and plans.
That is a break from most medical AI, which takes an all-at-once approach. You hand it the symptoms, labs and imaging, and it returns a diagnosis. Those systems are usually graded on multiple-choice questions, and as Dr. Dominic King, who leads the health unit at Microsoft AI, put it, AI has become very good at getting a 100% pass rate on those. His next line is the real critique: no patient ever came to him with a nice summary.
Cost is the other finding worth watching. The researchers say diagnostic work-ups cost less when the orchestrator designs them than when human doctors do. Harsha Nori, head of AI at Microsoft AI Health, said the team measures how judiciously the system goes through a work-up, not only whether it lands on the right answer. Hospitals and insurers care about that second metric at least as much as the first.
Context matters here. Google has pushed its own conversational diagnostic research, and OpenAI and Anthropic keep promoting medical benchmark results for their general models. Most of those claims lean on static question sets. An agent that chooses its own next step is a harder and more honest test, and it points to where clinical AI products are heading: multi-agent workflows, not single prompts.
Still, keep the caveats in view. These were rare, difficult cases, and the comparison group was general physicians, not specialists. Published results on curated cases are not the same as performance in a crowded ER. Microsoft knows that. With Beth Israel Deaconess Medical Center in Boston, it will next test the tool on triaging patients in emergency department waiting rooms.
So is this noise? No. A Science paper and a live hospital pilot are more than a demo video. But the verdict comes from the triage trial, where errors have consequences and the cases are not hand-picked.
