Post says medical AI scores barely drop when test images are removed
An AI researcher reported in a social media post that GPT-5's accuracy fell only slightly when medical images were removed from a test. The claim is preliminary and has not been peer reviewed.
Source: Claim: LLM scores barely fall on a medical benchmark without images (x.com). Summary written with AI from the source and reviewed by rules; read the original.
What the research shows
According to a social media post by an artificial intelligence researcher, GPT-5's accuracy on a multimodal test based on New England Journal of Medicine cases fell from 86.59 to 82.91 when the medical images were removed. The difference is about 3.7 percentage points. The author suggests that some medical AI tests may not be measuring the ability to interpret images, which could overstate how well these systems really perform.
This kind of claim is preliminary. It is a social media post, not a study reviewed by other experts, and the figures cannot be checked from the text alone. Medical AI tests often combine text and images, and understanding what is really being measured requires a close look at the methods. Even so, the discussion matters for anyone following how these systems are evaluated before being compared with health professionals.
What it means
The claim suggests that part of the score on some medical AI tests may not depend on reading images. It is not yet known whether the drop would repeat on other tests, other models, or under independent analysis.
How to read it with care
This result does not replace a medical consultation and should not be read as a reason to trust or distrust AI tools for diagnosis. Anyone considering using these tools, or changing habits, diet, supplements or treatment, should talk to a doctor or another qualified professional.