NaClhv

Here is yet another example from which we can empirically derive the Bayes' factor for a human testimony.

The September 11 terrorist attacks killed about 3000 people. It is the worst terrorist attack in world history to date. As such, it caused a great deal of shared grief and an outpouring of sympathy for the survivors and the families of its victims.

Of course, human being being what they are, some people falsely claimed that a close loved one had perished in the attacks. This got them a lot of sympathy - and more importantly, it got them a great deal of aid money, exceeding a hundreds of thousand of dollars in some cases.

This naturally leads us to ask - how reliable was a person's claim that they had lost a loved one in the 9/11 attacks? What was the Bayes' factor for such a claim? The numbers for this calculation are readily available. We just have to assemble them.

First, let's calculate the prior probability that someone really did lose a close loved one in the 9-11 attacks. We will assume that every one of the 3000 victims had about 4 loved ones (father, mother, sister, son, etc) whom we can consider "close", and that all of these loved ones lived in New York City. This gives 12,000, or about 1e4, close relation of the victims in a city with a population of 1e7. Therefore, the prior odds for a random person in New York City actually having a close loved one as a victim is about 1e-3.

Now, if someone claims that they had a close loved one die, what is the posterior odds that this person is actually telling the truth? One may assume that a vast majority of the 1e4 actual close relations of the victims made that claim. But how many false claims were mixed in with those? The specific number is not possible to determine (as someone could have lied so well that they were never suspected), but the article I previously linked mentions numbers like "dozens", "two dozen", or "37 arrests". Taking these numbers into account, let us be generous here and assume that there were 100, or 1e2, false claimants. The posterior odds are therefore 1e4:1e2, which is equal to 1e2.

Therefore, the Bayes' factor for someone claiming to have lost a loved one in the September 11th terrorist attacks is sufficient to take the odds from an empirically calculated prior value of 1e-3 to an empirically calculated posterior value of 1e2 - so it must be given a value of 1e5.

Nearly all of the numbers here are from Wikipedia or the New York Times. You can follow up on their sources and verify the values yourself. In the few places where I had to make assumptions, they have a definitive bias towards reducing the Bayes' factor - for example, the people who lost loved ones are not all confined to New York City, and 100 false claimants are a good deal more than two dozen. There's probably also a greater tendency for the truth-tellers to communicate their loss to more people in cases like these. Therefore, 1e5 is an underestimate of the true Bayes' factor. The actual value is greater - 1e6 seems like a reasonable guess.

Consider what this means: even when there was a clear reason to lie - that is, even when there was cold, hard cash at stake as a tangible reward for lying - people turned out to be fairly reliable overall. The Bayes' factor for their earnest claim about the personal tragedy of losing a loved one turned out to be about 1e6. Now, the general case would not have the explicit possibility of fraud as a precondition, and we would not be constrained to only consider the minimum value. Therefore a value of 1e8 for the general case is quite appropriate. That is a good estimate of the Bayes' factor for an earnest, insistent, personal testimony.

These Bayes' factor calculations will be summarized in the next post.