CALIBRATION PARADOX

Someone who forecasts events with a probability (such as a weather forecaster) may be more or less well calibrated in the following sense. Consider the sequence of days for which he predicts rain with 0.1 probability. It may rain on 0.1 of them, or more, or less. If the frequency of rain on such days corresponds to the probability he offers, for every probability, then he is perfectly calibrated. This would be a remarkable success, and it would be quite irrational for anyone to have such confidence in their own abilities that they expect to be so successful. But the paradoxical theorem (discovered in 1962 by Pratt, a Harvard statistician, and rediscovered by Dawid, in London) states that an agent with a view about his own calibration must assign 1 (corresponding to certainty) to the proposition that he is perfectly calibrated, on pain of having an incoherent set of probabilities. The situation bears some analogy to that of the paradox of the preface, where modesty induces inconsistency.