Siddharth Vohra: What Happens When Medical AI Loses the Evidence It Was Supposed to Use

His newly accepted MICCAI workshop paper expands earlier medical AI research into a larger audit of what happens when structured diagnoses become separated from the evidence they are supposed to represent.

A medical AI system can return a perfectly formatted diagnosis even when the image it was supposed to inspect was never actually attached. Siddharth Vohra has been studying what happens at that boundary, where medical evidence can become separated from the request and the structured output produced from it.

His latest paper, “Absent-Byte Diagnoses: Auditing Structured Medical VLM Interfaces,” has been accepted to the Agentic AI for Medicine Workshop at the Medical Image Computing and Computer Assisted Intervention conference, known as MICCAI. The expanded study asks whether software can treat a diagnosis as grounded in medical evidence when that evidence disappeared before the model produced its response.

The problem Vohra examines occurs around the model rather than solely within it. In the tested condition, the prompt states that a medical image is attached, but no image was actually included. The correct result in that state is no diagnosis at all. Across a 3,000-call baseline involving five hosted models, however, the study found 617 cases in which a diagnosis field was filled anyway. When the prompt instead stated plainly that no image was attached, none of 19,350 control calls produced a diagnosis, which points to the mismatch itself, rather than the models, as the problem.

These models do a lot of things well, and I want to see them used in medicine. What I am interested in is what the downstream system can actually act on," Vohra said. "A response can look structurally correct and still be disconnected from the evidence the task required. If software only checks whether the field exists and has the expected format, it can miss the more important question of whether the image was ever attached.

Structured outputs make that problem especially difficult to detect. Every response in the 3,000-call baseline satisfied the requested schema, yet 20.6 percent contained a diagnosis despite the missing image bytes. In 114 of the 617 filled diagnoses, another field in the same response said diagnosis was impossible. A person reviewing the complete response could notice that contradiction, while software extracting only the diagnosis field could preserve the disease name and discard the warning.

“Schema validity is not evidence validity,” Vohra said. “You can have a machine-readable object that passes formatting checks and still contains a conclusion that should never have been released. The interface has to know what evidence the field is supposed to be tied to.”

The paper expands Vohra’s earlier study, “Hearsay: Vision-Language Medical Diagnoses Without an Image.” That peer-reviewed project, presented at the TrustVLM workshop held with ACM ICMR 2026, analyzed 11,700 responses from Claude, GPT, and Gemini after the expected medical image was deliberately withheld. The new research moves further into the software interface itself, examining not only whether diagnoses appear without images, but also how the remaining request can influence what appears in the structured field.

One part of the expanded study examines demographic wording after the image evidence has already disappeared. Researchers made 288 direct comparisons in which only age, race wording, or s*x wording changed while the imaging question, attachment claim, response format, and text-only construction remained fixed.

A GPT-5.4 chest X-ray comparison illustrates the effect. For a profile described as a white man, Pneumothorax appeared as the leading diagnosis in 54 of 100 calls. Changing only the word “white” to “Black” shifted the leading result to Sarcoidosis in 77 of 100 calls. No medical image was supplied in either condition.

Vohra stresses that the comparison does not measure diagnostic accuracy between demographic groups. The demographic descriptions function as controlled wording probes, and there is no image evidence to diagnose in either condition. The result instead shows how other text in a request can influence a structured diagnosis once the evidence that should anchor the task is missing.

“The concern is not which group receives a better prediction,” Vohra said. “There is no image to evaluate in these comparisons. The important point is that once the evidence disappears, other text in the request can begin steering a field that downstream software may still treat as medically grounded.”

The paper also looks at how such separation can happen before a model is called. It examines public software regressions in which images or task text were lost while information moved through client software and agent handoffs. These examples do not establish how often the problem occurs in deployed medical systems, but they demonstrate plausible paths through which the intended task and the completed request can diverge.

Vohra’s proposed safeguard is an evidence-binding mechanism controlled by the calling software. The caller retains a record of the task and the images that should accompany it. Before dispatch, the completed request is checked against that record, and the connection is verified again before the diagnosis field can move downstream. A mismatch can therefore stop the process before an unsupported diagnosis is released.

The researchers tested the approach in a 6,165-case offline suite containing 1,118 valid controls and 5,047 specified violations. Three complete implementations passed every control and blocked every listed violation. The study does not claim that this covers every possible software failure, but it provides a concrete test of whether the proposed rule can detect the fault patterns included in the evaluation.

“That changes where the safety check happens,” Vohra said. “Instead of asking the model to tell us whether it saw the right evidence, the calling software keeps its own record. If the task or image has been lost or changed, the system can stop before an unsupported diagnosis moves forward.”

The paper is explicit about its limits. It does not estimate how frequently these failures occur in real deployments or claim measured medical impact. The study uses synthetic prompts, generated outputs, and cited public software revisions, without patient records or clinical images.

For Vohra, those limits reinforce the larger engineering question rather than weaken it. A medical agent is a chain of components, and each handoff creates another place where the intended evidence can become separated from the result. His work argues that safeguards should therefore verify the connection between a diagnosis and its evidence before downstream software is allowed to act on the field.

"The models are capable. The job of the surrounding software is to make sure they are working from the right evidence," Vohra said. "A medical agent is more than the model producing the answer. If evidence can disappear between components, safety must include the interfaces that connect them. Before a diagnosis moves forward, the system should be able to verify that the evidence it depended on traveled with it."