Microsoft AI outdiagnoses generalist doctors in complex case simulation
Microsoft says its MAI-DxO system diagnosed complex medical cases more accurately than generalist doctors in a simulation, a result the company sees as progress towards advanced AI for healthcare. The findings matter because AI companies are increasingly claiming their systems can match or exceed doctors, while healthcare services face a shortage of medical expertise.
In a study published in *Science*, MAI-DxO was correct in 83% of complex cases drawn from the *New England Journal of Medicine*, compared with 36% for primary care doctors and general internists using reference materials. The comparison did not include specialists or doctor teams, who would usually handle cases of this complexity. Microsoft also reported that results varied by AI model and case type: its orchestration system helped most with complex cases and less capable models, while adding little on emergency department cases. The company is working with Mayo Clinic on a healthcare foundation model it plans to release next year.
In short
- MAI-DxO scored 83% on simulated complex cases.
- Generalist doctors scored 36% in the comparison.
- Specialists and physician teams were not tested.
Steelman: both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
The results suggest AI could help clinicians reason through rare, complicated cases, especially where specialist expertise is scarce. If further studies confirm the benefit in real clinical settings, such tools might support more accurate diagnoses and make medical knowledge more accessible.
The case against
The simulation used unusually complex published cases and compared the system with generalist doctors, not specialists or clinical teams who would typically assess them. Performance on selected cases does not establish that AI will improve patient care in practice, where decisions depend on examination, context and accountability as well as diagnostic reasoning.
Explained simply
Microsoft is developing artificial intelligence (AI) tools for healthcare. MAI-DxO is a system designed to work through medical information and suggest a diagnosis. The company hopes such tools could help services facing shortages of medical staff.
Doctors who provide general care usually assess common health problems and refer patients to specialists when needed. Cases described as complex may involve unusual symptoms or several possible explanations, and are often handled by specialist doctors or teams. AI systems are being tested to see how well they can support this kind of diagnostic work.
The results of a simulation can show how a system performs on selected cases, but do not by themselves show how it would work with patients in everyday care. The comparison also matters because AI systems are increasingly being assessed against doctors, though their performance can differ depending on the model and type of case.
AI Research Science Technology
Read the full article at the source →
Originally published by TechCrunch as “Microsoft’s Mustafa Suleyman says its AI outperforms doctors on hard cases as he chases ‘medical superintelligence’”.