Absence of Evidence is Evidence of Absence

[DuB]

Absence of evidence is not evidence of absence. This assertion has long puzzled me, since its falsehood seems so obvious. Nevertheless, we hear it all the time: "No evidence that homeopathic treatments are more effective than placebo? That's okay, absence of evidence is not evidence of absence! No evidence for alien abductions...?" Most reasonable people cringe when they hear statements like these... but they nevertheless concede the point that absence of evidence is not--indeed, cannot be--evidence of absence. This thread is about why this is nonsense. The statement above really has two interpretations, a weak version and a strong version. The weak version is trivially true, and the strong version is simply wrong. The reason that so many reasonable people make the above concession is that they hear someone make the strong version of the statement, but they hear it as the weak version, and then in intending to concede the weak version, they unintentionally concede the strong version. The weak version goes like this: absence of evidence is not proof of absence. Notice the subtle but important difference here: in this case we are concerned with establishing proof of absence. It is notoriously difficult to prove anything, regardless of the state of the evidence, which is what makes this version of the statement weak. The strong version is strong because it is concerned with evidence rather than proof: "absence of evidence of not evidence of absence." The strong version is wrong because it defies the basic axioms of probability theory. To demonstrate this, we will view probability theory through the lens of Bayes' Rule, which is a normative and deductively valid interpretation of probability. Although Bayes' Rule is at its heart a mathematical theorem, it can be perfectly well understood on an intuitive level through a simple example. I will not be the first person by far to bring Bayes' Rule to bear on this annoyingly persistent misbelief, but hopefully my explanation will serve as a clear first introduction. For a superb introduction to Bayesian reasoning in general, see the link above. Let's say that I have misplaced my cell phone. Consider two hypotheses concerning where my cell phone is. The first hypothesis, H, represents the hypothesis that my cell phone is somewhere in my bedroom, and is true with some probability P(H). The competing hypothesis, that my cell phone is anywhere but in my bedroom (i.e., that H is false), is true with probability P(~H) or 1 - P(H). Note that these are exhaustive and exclusive. In order to gather evidence for and against these two hypotheses, I decide to call my cell phone from my house phone and see if there is a ringing coming from my bedroom. We will call this evidence E, and it will obtain with probability P(E) or not obtain with probability P(~E). I call my cell phone from my house phone. No ring comes from my bedroom. The evidence E did not obtain. What does this mean for our hypotheses H and ~H? We are ultimately interested in two conditional probabilities: the probability that my cell phone is in my bedroom given that no ring came from the bedroom, P(H|~E), and the probability that my cell phone is not in my bedroom given that no ring came from my bedroom, P(~H|~E). (From this point on it is crucial to recognize that in this format for representing conditional probabilities, the events following the | symbol are assumed to have obtained. What we are considering is the probability of a certain event conditional on a different event being true. Keeping this format in mind will allow me to save a lot of space in the following paragraphs and will save you a lot of confusion.) These two probabilities of ultimate interest are in turn dependent on the two conditional probabilities: P(~E|H), the probability that no ring comes from my bedroom given that my cell phone is in my bedroom, and P(~E|~H), the probability that no ring comes from my bedroom given that my cell phone is not in my bedroom. That these latter two probabilities make a difference makes sense, because it's basically saying that the reliability of our particular evidence has an important bearing on what kinds of conclusions we can make about the hypotheses. If my cell phone has a faulty speaker and only rings about half of the times that it's called, then failing to hear a ring coming from the bedroom will be less convincing evidence against hypothesis H. On the other hand, if my cell phone is in perfect condition and has never been observed not to ring when called, this will be much more damning evidence against H. Bayes' Rule precisely specifies how to combine these probabilities in order to get the exact values for P(H|~E) and P(~H|~E), but we don't need to be precise for the purposes of this example. We need only think about the matter intuitively. We saw above that as the reliability of my cell phone's speaker increases, it becomes stronger and stronger evidence against H in the event that we fail to hear its ring. In other words, as P(~E|H) decreases, P(H|~E) increases and P(~H|~E) therefore decreases, since the two hypotheses are exclusive and exhaustive. On the other hand, higher values of P(~E|~H) tip the odds in favor of P(~H|~E) and against P(H|~E). What we need to do then is compare P(~E|H) and P(~E|~H). I should point out here that we are making simplifying assumptions in order to keep things intuitive, but we are not fundamentally altering the nature of the problem. Recall that P(~E|H) represents the probability of not hearing the ring coming from my bedroom given that the cell phone is actually in my bedroom, while P(~E|~H) represents the probability of not hearing the cell phone's ring given that it is not in my bedroom. There are various reasons to believe that P(~E|H) is not terribly low. The phone's speaker may be unreliable, the battery may have run out, I could have poor hearing, etc. But it should be obvious that P(~E|~H) is incredibly high, almost 1. If the cell phone is not actually in my bedroom, the odds of hearing a cell phone ring which sounds exactly like mine coming from my bedroom at the exact instant that I happen to be calling it are impossibly low, whereas it is a near certainty that I will not hear such a ring. No matter how many reasons we might dream up that would increase P(~E|H), it will never be as high as P(~E|~H). Thus, failing to hear the ring coming from my bedroom will always decrease the probability that H is true and increase the probability that H is false. In other words, absence of evidence for my cell phone being in my bedroom is always evidence that my cell phone is not in my bedroom. This is true for any set of hypotheses where the correlation between E and H is different from 0, in other words, when evidence is actually evidence. Therefore, absence of evidence is evidence of absence.

[Spartiate]

I agree 100%. I like how you differentiate proof and evidence. It seems that believers in paranormal phenomena or what have you are satisfied with that phrase, as if not being able to 100% disprove their theory is better than providing evidence in favour of their theory.

[Xedan]

I think the phrase should be changed more to something like: "Absence of overwhelming evidence is not proof of absence" or "Absence of insignificant evidence is not evidence of being incorrect" as these seem to be the two things normally implied by the phrase. Though as is, the phrase is pretty much a back up for EVERY crack-head theory. Since you linked me to this from an argument with Xei, let me explain also that I use the first alternate version against him because in numerous threads of things considered supernatural, but by nature visible (namely psi) I will present lots of evidence. Some he will say is wrong and he's correct, some he will say is wrong and he'll be incorrect, and some he'll totally ignore. And even after all that he'll say there's no evidence whatsoever.

[DuB]

I think the phrase should be changed more to something like: "Absence of overwhelming evidence is not proof of absence" or "Absence of insignificant evidence is not evidence of being incorrect" as these seem to be the two things normally implied by the phrase. Though as is, the phrase is pretty much a back up for EVERY crack-head theory. Those are just two different ways of phrasing the weak and strong versions of the statement, respectively, which I addressed in the OP. As I wrote above, the first is trivially true (given the well-known problem of induction, evidence of any quality whatsoever can never be proof), and the second is wrong (I'm not sure what you mean by "insignificant" evidence, but the point is that the existence or nonexistence of evidence, regardless of its quality, is ipso facto a piece of evidence). Since you linked me to this from an argument with Xei, let me explain also that I use the first alternate version against him because in numerous threads of things considered supernatural, but by nature visible (namely psi) I will present lots of evidence. Some he will say is wrong and he's correct, some he will say is wrong and he'll be incorrect, and some he'll totally ignore. And even after all that he'll say there's no evidence whatsoever. You're bringing your personal beef with Xei into this thread too now? Let's just talk about the actual topic, okay?

[Oneironaut Zero]

I've always thought of the saying as pretty hokey, myself.

[Xaqaria]

Although I do agree with everything you've said, there are situations in which the logic still applies. I don't have a lot of background in the math of probability so forgive me for not using such technical language. Your example is a good one because there are certain conditions that it ignores that, if present, would completely change the probabilities. Say, for instance your ringer is off but you still decide to call it. Obviously it won't ring and there is an "absence of evidence" but still you know that it really is not evidence of absence since it won't ring no matter where it is. Another good example would be a situation in which evidence is thought to be absent only because it is not recognized as evidence. Say with your cellphone example, someone else had recently changed your ring tone to some fruity bird sound and when you call you don't immediately associate the sound with a ringing phone. In this case you would simply be not recognizing the evidence because you are looking specifically for evidence of a ringing phone sound. Therefore, absence of evidence of a ringing phone sound is not evidence of the absence of the phone.