Skip to Content
You are currently on the new version of our website. Access the old version .
  • Article
  • Open Access

7 November 2026

40 Pages

Probabilistic Coherence of Stated Diagnostic Beliefs in Large Language Models: Anchored Tests on a Synthetic Clinical Corpus

,
and
1
College of Computer, Mathematical, and Natural Sciences, University of Maryland, College Park, MD 20742, USA
2
DiceMed, Cuttack 753007, Odisha, India
3
School of Information, Florida State University, Tallahassee, FL 32306, USA
*
Author to whom correspondence should be addressed.
This article belongs to the Section Data

Abstract

Clinical evidence arrives in sequence, yet a coherent diagnostician’s belief should depend on what was found, not on the order of finding it. Bayesian belief revision is order-invariant as a theorem, which yields an exact, annotation-free test, with target zero, of the distribution a clinical language model reports. Three further axioms cover update fidelity, redundancy insensitivity, and positional anchoring. Eleven open-weight configurations of 4B to 32B parameters are evaluated on 1.12 million responses over 1956 patients and 49 pathologies of the synthetic DDXPlus corpus. Every divergence is anchored to the model’s own test–retest floor and between-patient ceiling, and to a corpus replicate floor of 0.049. On the Qwen3 ladder, raw permutation divergence rises from 0.128 to 0.291 with scale, while the order effect as a fraction of each model’s floor-to-ceiling range falls from 0.215 to 0.127. Belief updates do not track the corpus evidence: direction agreement is 0.454 to 0.549, even on findings the corpus moves by more than two log odds. Every model names the correct diagnosis 1.8 to 5.6 times more often than its own stated posterior ranks it first, locating the deficit in the probabilistic interface, not in clinical knowledge.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.