in partnership with | ||

Providing Outreach in Computer Science | Bringing Bayesian solutions to real-world risk problems |

Consider the following two statements:

- "There is a 50% chance that the next toss on a fair coin will be a Head"
- "There is a 0.0000001% chance of Martians landing on earth this year"

Statement 1 can be explained by a 'frequentist' arguments: if you toss a fair coin many times it will land as a Head roughly 50% of the times. So it makes sense to use this knowledge to inform your assessment.

Statement 2 has no such frequentist argument. We cannot 'play' this year over and over again counting the number of times in which Martians land. We can only provide a subjective measure of uncertainty based on our current state of knowledge.

Some people (including even clever ones) feel comfortable with the frequentist approach but so uncomfortable with the subjective approach that they reject it as invalid. Their primary objections are that:

- the subjective measure cannot be validated
- different experts will give different subjective measures

If you still believe that the frequentist approach is somehow pure and superior then consider the following two statements which are less extreme versions of the frequentist versus subjective examples of statements 1 and 2 above:

3. "There is a 50.9% chance that a baby born in the UK is a girl"

4. "There is a 5% chance of Spurs winning the FA Cup next year"

There is no doubt that statement 3 is explained by a frequentist argument: Over the last 100 years 50.9% of all births recorded in the UK have been girls.

There is also no doubt that statement 4 has no such frequentist explanation (and hence must be subjective) since there is only one FA Cup next year and we cannot somehow play the tournament many times in the same year and count the number of occasions on which Spurs win.

But if we dig a little deeper here, things get rather murky. The 50.9% figure in statement 3 is actually based on many years of data that may disguise crucial trend information. Suppose we discover that the percentage of girls born is increasing; say a hundred years ago 48.5% of babies were girls compared with 51.2% last year. Then surely the probability of a randomly selected new born baby being a girl now is higher than 50.9% (and higher than 51.2% if the figures have been steadily increasing). And what exactly do we mean by a 'randomly' selected baby. Surely what we are most interested in are specific babies such as "the next baby born to Mrs Roberts of 213 White Hart Land, London N17". In that case the frequency data may need to be 'adjusted' to take account of specific factors relevant to Mrs Roberts. Both the general trend adjustments and the case specific adjustments here clearly require the subjective judgment of relevant experts. But that means, according to the frequentists, that their own approach is no longer valid since, as we saw above:

- the measure cannot be validated
- different experts will give different subjective measures

Whatever anybody's objection to subjective measures, like it or not, they are used so extensively that the fabric of modern society would break down without them. Hence bookies will provide 'odds' on events (such as Spurs winning the FA Cup) based on subjective measures, while insurance companies will do the same in determining policy premiums and governments the same when determing economic policies.

The frequentist approach for measuring uncertainty is all well and good providing that we have been able to record accurate information about many past instances of the event. However, most uncertain events of interest do not have such historical databases associated with them, and even where relevant historical data does exist it must still usually be informed by subjective judgements before it can be used for measuring uncertainty. Hence, generally we cannot rely on the frequentist approach to measure uncertainty.

The subjective approach accepts unreservedly that different people (even experts) may have vastly different beliefs about the uncertainty of the same event. Hence Norman's belief about the chances of Spurs winning the FA Cup this year may be very different from Daniel's. Norman, using only his knowledge of the current team and past achievements may rate the chances at 10%. Daniel, on the other hand, may rate the chances as 2% based on some inside knowledge he has about key players having to be sold in the next two months.

Hence the subjective approach is always based on some prior body of knowledge. In this sense subjective measures of uncertainty are always conditional on this prior knowledge. The subjective approach is also called the Bayesian approach, because only in the Bayesian approach is there a rigorous way of reasoning about such conditional knowledge, and only the Bayesian approach can provide a rational way of revising your beliefs in the light of evidence (such as when you have seen 100 consecutive Heads tossed on an apparently fiar coin).

Norman Fenton

Return to Main Page Making Sense of Probability: Fallacies, Myths and Puzzles