Microsoft reports higher diagnostic accuracy from AI in historical case tests
Microsoft says its Diagnostic Orchestrator AI diagnosed conditions more accurately than doctors in tests using past medical cases. The results suggest that AI systems coordinating several agents could support medical diagnosis, but they do not show that the tool is ready to replace clinicians.
In cases drawn from published medical reports, the system scored about 83 per cent accuracy, compared with 36 per cent for generalist doctors using references and 22 per cent without them. For emergency department cases, it scored 80 per cent, while doctors averaged 47 per cent; Microsoft also estimated lower diagnostic costs. The system uses OpenAI’s GPT-5 and has not yet been validated in real clinical settings, though Microsoft plans to test it with a Boston hospital.
In short
- Microsoft’s AI tool scored higher than doctors in tests using past cases.
- It achieved 83 per cent accuracy on published medical reports.
- Real-world clinical validation is still needed.
Steelman: both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
The results suggest that an AI system coordinating several agents could help clinicians by identifying diagnoses doctors might miss, while potentially reducing the time and cost of working through complex cases. If further testing confirms these gains in real clinical settings, such tools could support more timely and consistent care, especially where specialist expertise is limited.
The case against
The comparisons use past published cases, which may not reflect the uncertainty, time pressure and patient interaction of real clinical practice. Until the system is tested prospectively and shown to be safe and reliable across different patients and settings, its reported accuracy and cost estimates are not enough to justify relying on it for clinical decisions.
Explained simply
Microsoft’s Diagnostic Orchestrator is an artificial intelligence system designed to help work out what might be causing a patient’s symptoms. It uses OpenAI’s GPT-5 and coordinates several AI agents to assess medical information.
Doctors make diagnoses by weighing symptoms, test results and medical history, sometimes consulting reference material or specialists. The reported tests used cases from published medical reports, rather than patients being assessed in everyday care.
The results matter because diagnosis can affect which treatment a patient receives and how quickly. Testing on past cases cannot establish how well the system would work with real patients, so further clinical evaluation is needed to assess its role alongside doctors.
AI Business Companies Technology
Read the full article at the source →
Originally published by The Verge as “Microsoft says its AI tool is better than doctors at diagnosing conditions”.