Statistical pundit William M. Briggs has written a piece for ‘Significance’ on Why Do Statisticians Answer Silly Questions That No One Ever Asks?.
Briggs is right to object to instances where statisticians (or more often users of statistics) respond to silly questions with the answers to different (and often equally silly) ones without making it clear enough that they are not answering the original question. But he is wrong in his presumption that the questions asked usually make sense.
In fact it is common to talk of probability in quantitative terms in situations where it is by no means clear what a specific numerical probability would mean. Such talk is what gives rise to many well known “paradoxes” which can only be resolved by clarifying the interpretation.
But although Briggs alludes to recent advances in Bayesian analysis, he doesn’t seem to understand them well enough himself – at least not well enough to answer a simple question about what he means when he says “a civilian needs little or no maths to understand what ‘the probability that A is better than B is 80%’ means”.
Briggs response to the question of what that understanding might be is just “It means the evidence is such that the probability ‘A is better than B’ is 80%. Which is greater than 0% but less than 100%. Nothing more.”
When challenged that this is like claiming to explain what “the hoy is gerflumptive” means by saying that it means “the evidence is such that the hoy is gerflumptive”, he responds with “I wasn’t being glib. Probability (see above) is a measure of truth, or closeness to truth. 80% is closer than 70% and less close than 90% to being true. What you do with this number is different than what the number is.“
Well, I’m sorry, but giving “closeness to truth” as a definition of probability *is* glib.
(It’s also more than 75% wrong in that I can think of at least three measures of closeness to truth that are more common than anything to do with probability.)
He asks for examples and I say:
For example, in common language (as per my claim):
1. an approximate answer is often referred to as close to the truth
2. a false statement is sometimes referred to as close to the truth if its error arises from a fairly common misuse of terminology
3. a detective may be said to be getting close to the truth if he has a good idea of where to look for the deciding piece of evidence
etc.
Briggs responds to these with:
But two of these examples are non-probabilistic.
1. Given our background knowledge, an approximate answer is likely true
3. Ditto
2. You’ll have to clarify this. A falsity is not close to a truth; a mistake is still a mistake.
Cooper:
They were intended to be non-probabilistic as I was giving them as examples of why “closeness to truth” is not a good definition of probability.
1. The statement that the circumference of a circle is six times its radius has zero probability of being true but it is close to the truth.
3. Knowledge of the fact that the murdered duke wrote a deathbed note which will tell me whether it was Colonel Mustard or Professor Plum who poisoned him brings me closer to the truth without increasing the probability of either hypothesis.
2. Your attempt to define probability as “closeness to the truth” may be close to the truth but it has zero probability of actually providing a useful definition.
Briggs:
Alan,
I assume you meant your “2″ as a joke, but it has backfired on you. In a useful way, however. Let’s see.
1. A = “The circumference of a circle is six times is radius.” Now, there is no such thing as
Pr(A).
But we can calculate:
Pr(A | E) = 0
where E = “My knowledge of geometry as might be found in any high school or higher text”. Notice that this is completely different than B = “A is a good approximation”. We still cannot calculate
Pr(B).
But we can calculate:
Pr(B | E & F) = 1
where we have the same E plus information F = “A good approximation is being within plus or minus 20% of the radius” or some other F (different F might change the probability, of course).
3. A = “Duke says M or P killed him”. If B = “M killed the Duke” then
Pr(B | A) = 1/2
and similarly for C = “P killed the Duke.” The probability
Pr(Duke was murdered | evidence of dead body & foul play) = 1
which is the same as
Pr(Duke was murdered by somebody | evidence of dead body & foul play) = 1.
But we cannot compute
Pr(Duke was murdered by M | evidence of dead body & foul play) = unknown,
unless we condition on something more, namely a list of suspects.
2. I could write this out, but you’ll get the idea. The probability that I have provided you the true definition, given all this (and other information on the blog) is 1.
Cooper:
All I can say is that I think you must have missed my point – which was that there are common language senses of “closeness to truth” which have nothing to do with probability, and so that “closeness to truth” is not a good definition of probability.
This all started when I asked you what you would say ‘the probability that A is better than B is 80%’ means, and so far I haven’t seen anything not glib in response.
Briggs:
Alan,
I haven’t; you have failed to make yours. In order to disprove my thesis, you need to show an example that can’t be written in the forms (for example) that I’ve given.
Cooper:
Thanks for trying, but I don’t understand what you are saying. If you have given an intelligible answer to my question about the meaning of probability then I guess I’ll just have to accept that the subject is beyond me.
Seriously, am I nuts or is this guy cuckoo?