Absence of Evidence is Evidence of Absence

Quote:

---

Originally Posted by purveyors of woo

Absence of evidence is not evidence of absence.

---

This assertion has long puzzled me, since its falsehood seems so obvious. Nevertheless, we hear it all the time:

"No evidence that homeopathic treatments are more effective than placebo? That's okay, absence of evidence is not evidence of absence! No evidence for alien abductions...?"

Most reasonable people cringe when they hear statements like these... but they nevertheless concede the point that absence of evidence is not--indeed, cannot be--evidence of absence. This thread is about why this is nonsense.

The statement above really has two interpretations, a weak version and a strong version. The weak version is trivially true, and the strong version is simply wrong. The reason that so many reasonable people make the above concession is that they hear someone make the strong version of the statement, but they hear it as the weak version, and then in intending to concede the weak version, they unintentionally concede the strong version. The weak version goes like this: absence of evidence is not proof of absence. Notice the subtle but important difference here: in this case we are concerned with establishing proof of absence. It is notoriously difficult to prove anything, regardless of the state of the evidence, which is what makes this version of the statement weak.

The strong version is strong because it is concerned with evidence rather than proof: "absence of evidence of not evidence of absence." The strong version is wrong because it defies the basic axioms of probability theory. To demonstrate this, we will view probability theory through the lens of Bayes' Rule, which is a normative and deductively valid interpretation of probability. Although Bayes' Rule is at its heart a mathematical theorem, it can be perfectly well understood on an intuitive level through a simple example. I will not be the first person by far to bring Bayes' Rule to bear on this annoyingly persistent misbelief, but hopefully my explanation will serve as a clear first introduction. For a superb introduction to Bayesian reasoning in general, see the link above.

Let's say that I have misplaced my cell phone. Consider two hypotheses concerning where my cell phone is. The first hypothesis, H, represents the hypothesis that my cell phone is somewhere in my bedroom, and is true with some probability P(H). The competing hypothesis, that my cell phone is anywhere but in my bedroom (i.e., that H is false), is true with probability P(~H) or 1 - P(H). Note that these are exhaustive and exclusive. In order to gather evidence for and against these two hypotheses, I decide to call my cell phone from my house phone and see if there is a ringing coming from my bedroom. We will call this evidence E, and it will obtain with probability P(E) or not obtain with probability P(~E).

I call my cell phone from my house phone. No ring comes from my bedroom. The evidence E did not obtain. What does this mean for our hypotheses H and ~H? We are ultimately interested in two conditional probabilities: the probability that my cell phone is in my bedroom given that no ring came from the bedroom, P(H|~E), and the probability that my cell phone is not in my bedroom given that no ring came from my bedroom, P(~H|~E).

(From this point on it is crucial to recognize that in this format for representing conditional probabilities, the events following the | symbol are assumed to have obtained. What we are considering is the probability of a certain event conditional on a different event being true. Keeping this format in mind will allow me to save a lot of space in the following paragraphs and will save you a lot of confusion.)

These two probabilities of ultimate interest are in turn dependent on the two conditional probabilities: P(~E|H), the probability that no ring comes from my bedroom given that my cell phone is in my bedroom, and P(~E|~H), the probability that no ring comes from my bedroom given that my cell phone is not in my bedroom. That these latter two probabilities make a difference makes sense, because it's basically saying that the reliability of our particular evidence has an important bearing on what kinds of conclusions we can make about the hypotheses. If my cell phone has a faulty speaker and only rings about half of the times that it's called, then failing to hear a ring coming from the bedroom will be less convincing evidence against hypothesis H. On the other hand, if my cell phone is in perfect condition and has never been observed not to ring when called, this will be much more damning evidence against H.

Bayes' Rule precisely specifies how to combine these probabilities in order to get the exact values for P(H|~E) and P(~H|~E), but we don't need to be precise for the purposes of this example. We need only think about the matter intuitively. We saw above that as the reliability of my cell phone's speaker increases, it becomes stronger and stronger evidence against H in the event that we fail to hear its ring. In other words, as P(~E|H) decreases, P(H|~E) increases and P(~H|~E) therefore decreases, since the two hypotheses are exclusive and exhaustive. On the other hand, higher values of P(~E|~H) tip the odds in favor of P(~H|~E) and against P(H|~E).

What we need to do then is compare P(~E|H) and P(~E|~H). I should point out here that we are making simplifying assumptions in order to keep things intuitive, but we are not fundamentally altering the nature of the problem. Recall that P(~E|H) represents the probability of not hearing the ring coming from my bedroom given that the cell phone is actually in my bedroom, while P(~E|~H) represents the probability of not hearing the cell phone's ring given that it is not in my bedroom. There are various reasons to believe that P(~E|H) is not terribly low. The phone's speaker may be unreliable, the battery may have run out, I could have poor hearing, etc. But it should be obvious that P(~E|~H) is incredibly high, almost 1. If the cell phone is not actually in my bedroom, the odds of hearing a cell phone ring which sounds exactly like mine coming from my bedroom at the exact instant that I happen to be calling it are impossibly low, whereas it is a near certainty that I will not hear such a ring. No matter how many reasons we might dream up that would increase P(~E|H), it will never be as high as P(~E|~H).

Thus, failing to hear the ring coming from my bedroom will always decrease the probability that H is true and increase the probability that H is false. In other words, absence of evidence for my cell phone being in my bedroom is always evidence that my cell phone is not in my bedroom. This is true for any set of hypotheses where the correlation between E and H is different from 0, in other words, when evidence is actually evidence. Therefore, absence of evidence is evidence of absence.

I agree 100%. I like how you differentiate proof and evidence. It seems that believers in paranormal phenomena or what have you are satisfied with that phrase, as if not being able to 100% disprove their theory is better than providing evidence in favour of their theory.

I think the phrase should be changed more to something like: "Absence of overwhelming evidence is not proof of absence" or "Absence of insignificant evidence is not evidence of being incorrect" as these seem to be the two things normally implied by the phrase. Though as is, the phrase is pretty much a back up for EVERY crack-head theory.

Since you linked me to this from an argument with Xei, let me explain also that I use the first alternate version against him because in numerous threads of things considered supernatural, but by nature visible (namely psi) I will present lots of evidence. Some he will say is wrong and he's correct, some he will say is wrong and he'll be incorrect, and some he'll totally ignore. And even after all that he'll say there's no evidence whatsoever.

Quote:

---

Originally Posted by Xedan

I think the phrase should be changed more to something like: "Absence of overwhelming evidence is not proof of absence" or "Absence of insignificant evidence is not evidence of being incorrect" as these seem to be the two things normally implied by the phrase. Though as is, the phrase is pretty much a back up for EVERY crack-head theory.

---

Those are just two different ways of phrasing the weak and strong versions of the statement, respectively, which I addressed in the OP. As I wrote above, the first is trivially true (given the well-known problem of induction, evidence of any quality whatsoever can never be proof), and the second is wrong (I'm not sure what you mean by "insignificant" evidence, but the point is that the existence or nonexistence of evidence, regardless of its quality, is ipso facto a piece of evidence).

Quote:

---

Originally Posted by Xedan

Since you linked me to this from an argument with Xei, let me explain also that I use the first alternate version against him because in numerous threads of things considered supernatural, but by nature visible (namely psi) I will present lots of evidence. Some he will say is wrong and he's correct, some he will say is wrong and he'll be incorrect, and some he'll totally ignore. And even after all that he'll say there's no evidence whatsoever.

---

You're bringing your personal beef with Xei into this thread too now?
:facepalm:
Let's just talk about the actual topic, okay?

I've always thought of the saying as pretty hokey, myself.

Although I do agree with everything you've said, there are situations in which the logic still applies. I don't have a lot of background in the math of probability so forgive me for not using such technical language. Your example is a good one because there are certain conditions that it ignores that, if present, would completely change the probabilities. Say, for instance your ringer is off but you still decide to call it. Obviously it won't ring and there is an "absence of evidence" but still you know that it really is not evidence of absence since it won't ring no matter where it is.

Another good example would be a situation in which evidence is thought to be absent only because it is not recognized as evidence. Say with your cellphone example, someone else had recently changed your ring tone to some fruity bird sound and when you call you don't immediately associate the sound with a ringing phone. In this case you would simply be not recognizing the evidence because you are looking specifically for evidence of a ringing phone sound. Therefore, absence of evidence of a ringing phone sound is not evidence of the absence of the phone.

Quote:

---

Originally Posted by Xaqaria

Although I do agree with everything you've said, there are situations in which the logic still applies. I don't have a lot of background in the math of probability so forgive me for not using such technical language. Your example is a good one because there are certain conditions that it ignores that, if present, would completely change the probabilities. Say, for instance your ringer is off but you still decide to call it. Obviously it won't ring and there is an "absence of evidence" but still you know that it really is not evidence of absence since it won't ring no matter where it is.

---

This objection (in a slightly different form) has been pointed out elsewhere, and in response I should point out that a disclaimer near the end of my original post handles this point. Bayesian-style inference (or any kind of inference, really) is only appropriate when there is some non-zero correlation between the evidence in question and the hypothesis. Evidence is by definition some event whose presence or absence allows us to discriminate to some degree between two or more hypotheses. If an event is completely uninformative toward any of the hypotheses of interest (e.g., if not hearing a ring is equally likely regardless of the location of my phone because I know that the ringer is turned off), it makes little sense to call the occurrence or nonoccurence of that event "evidence" in the first place, and not Bayes' Rule nor any other rule can help you.

As a more amusing example of this same principle, the absence of evidence for the existence of unicorns is not evidence that my cell phone is absent from my bedroom, but this is only because there is zero correlation between my cell phone's location and the existence of unicorns. Similarly, If I know that my cell phone's ringer is off, attempting to use its ring as evidence makes about as much sense as attempting to use the existence of unicorns as evidence. They're both irrelevant.

Quote:

---

Originally Posted by Xaqaria

Another good example would be a situation in which evidence is thought to be absent only because it is not recognized as evidence. Say with your cellphone example, someone else had recently changed your ring tone to some fruity bird sound and when you call you don't immediately associate the sound with a ringing phone. In this case you would simply be not recognizing the evidence because you are looking specifically for evidence of a ringing phone sound. Therefore, absence of evidence of a ringing phone sound is not evidence of the absence of the phone.

---

This brings up an important point which I should have made clear in the original post. The point is that these probabilities and inferences are completely dependent on the judge's personal knowledge. The probabilities at hand are not technically the objective probabilities that the events will or will not occur, because those probabilities are necessarily unknowable--only "mother nature" can ever know them, so it is not useful to talk about the real, objective probabilities at all. Rather, the probabilities we are considering are degrees of subjective personal belief that the events will occur--quantified.

In the case of your example, the absence of evidence for the cell phone actually would still be evidence for the cell phone's absence, because the judge has no way of knowing that the friend had tampered with the phone's ring tone. It does not affect the probability P(E), the judge's degree of belief that the phone will ring when called, and thus the result of the inferential process is the same. For the judge, absence of the cell phone's ring is evidence of the cell phone's absence. This is completely rational given the information that the judge has. We can't hold against him what he can't possibly know.

Quote:

---

Originally Posted by Xaqaria

Another good example would be a situation in which evidence is thought to be absent only because it is not recognized as evidence. Say with your cellphone example, someone else had recently changed your ring tone to some fruity bird sound and when you call you don't immediately associate the sound with a ringing phone. In this case you would simply be not recognizing the evidence because you are looking specifically for evidence of a ringing phone sound. Therefore, absence of evidence of a ringing phone sound is not evidence of the absence of the phone.

---

Exactly. That's my main reason for using it in any given situation involving widely unknown things. We draw on prior knowledge to form theories, which is not in itself a bad thing. But the fact that we don't know what we're looking for doesn't mean it isn't there.

And, DuB, just to clarify, by insignificant evidence I mean like how fundies always point to missing links as a roundabout way of saying their theory is better, even though theirs takes no place in know fact and would in fact break the laws of physics. "Pointing out unavoidable flaws in a theory doesn't weaken existing evidence, and therefor only makes you Captain Obvious while still gaining no logical high ground" (I may have gone a little out of the existing frame with this statement) And by unavoidable flaws, I mean things like missing links, where finding one only makes two more missing links.

I really should have used "proof" in the second statement in my earlier post too, which would of course make it just as trivial, but that's the point. "the absence of evidence isn't evidence of absence" isn't supposed to be a phrase to base your whole argument off of. Though, given how unclear it is, I will try to refrain from using it in the future without clearing up the many things it leaves to interpretation.

Right. I brought up both of those points because of how they relate to the topics that this argument is usually used for. Say for instance the example of ghosts. One may assume that since they don't see floating transparent apparitions, there is evidence of absence of ghosts. This is a sort of fallacious narrowing of what is considered evidence. The existence of a ghost may not be evident in any way that one would normally expect and so absence of evidence may only be evidence that we don't know what the hell we are looking for.

Quote:

---

Originally Posted by Xaqaria

Right. I brought up both of those points because of how they relate to the topics that this argument is usually used for. Say for instance the example of ghosts. One may assume that since they don't see floating transparent apparitions, there is evidence of absence of ghosts. This is a sort of fallacious narrowing of what is considered evidence. The existence of a ghost may not be evident in any way that one would normally expect and so absence of evidence may only be evidence that we don't know what the hell we are looking for.

---

And this can even go as far as religion, explaining why people become agnostic. We don't know what god looks like. We don't know what he's doing. We don't know what he did. So to try and pull together any positive evidence of that without relying on holy text is completely pointless. But of course, holy text brings up an entirely new challenge. You don't know which one is right. You can't even infer from how much fact is involved in the writing, because most are written as post-diction in the first place.

Quote:

---

Originally Posted by Xaqaria

Say for instance the example of ghosts. One may assume that since they don't see floating transparent apparitions, there is evidence of absence of ghosts. This is a sort of fallacious narrowing of what is considered evidence. The existence of a ghost may not be evident in any way that one would normally expect and so absence of evidence may only be evidence that we don't know what the hell we are looking for.

---

I thought you were right there with me, but I'm starting to think you're misunderstanding the argument. This example clearly falls within the scope of the earlier discussion in that it either (a) is evidence of absence, or (b) there is no correlation between E and H, so no inferences can be drawn and it doesn't make sense to speak of evidence in the first place. (And the fact that this is not clear to you as well raises the probability P(H) that I have not explained myself well enough... :boggle:)

Let's consider the proposed piece of evidence for (or against) ghosts: me seeing a ghost. Whether or not the occurrence of nonoccurence of this piece of evidence places us in the categories (a) or (b) from above depends on what we believe about the visibility of ghosts. Perhaps we think ghosts are invisible. In this case, there is no correlation between whether we can visibly see a ghost and whether a ghost is actually there. This puts us in (b): no inferences can be drawn, and seeing or not seeing a ghost is not evidence. But perhaps we think that ghosts often appear as translucent, floating spooky things. In this case there is a correlation between seeing a ghost and the existence of a ghost, so we are in category (a): my failure to observe translucent, floating spooky things is evidence that there are not any ghosts (in my personal vicinity, anyway).

The fact that you even mentioned floating apparitions is indicative of the fact that many people clearly do believe that there is some non-zero correlation between the visibility of ghosts and their presence--in other words, ghosts are at least sometimes visible--so in this case, absence of visual evidence for ghosts is evidence of the absence of ghosts.

This last point is crucial. There doesn't have to be a perfect correlation of 1 (or -1) between the evidence and the hypothesis in order for us to draw rational inferences toward the hypothesis. It only has to be non-zero. Xedan, I'd prefer not to steer this discussion toward religion, so I'll use the example of Santa Claus. On some arguments, we could say that the absence of evidence for Santa Claus (such as my failure to ever catch him in the act of present-leaving) is not evidence of Santa Claus's absence, because Santa Claus is sly and elusive. However, given that under our hypothesis he actually does come down my chimney every year, there must be some probability--however vanishingly small--that somebody would have seen him at least once. Compare this to the probability of seeing Santa Claus under the hypothesis that he doesn't exist: zero. By Bayes' Rule, the latter hypothesis becomes more likely every time the second piece of evidence (non-observance) obtains.

We might also say that Santa Claus is actually invisible, like a ghost. In this case whether or not we see him is irrelevant toward our hypotheses. We don't even have an absence of evidence in this case, because that would require that there be potential evidence in the first place.

Maybe it's only me, but I don't see what's different about what you two are saying, DuB. The only thing is that he used the term "is evidence", though the statement as a whole led to the conclusion that we cannot make a conclusion.

DuB, my position is essentially that Bayes' rule cannot be applied to the type of situations that you seem to want to apply it to for the reason that you yourself have mentioned; it is entirely dependent on the judge's personal knowledge, which is often sorely inadequate when judging these topics. We can use another somewhat cliche statement to illustrate this,

Quote:

---

Originally Posted by thank you, Donald Rumsfeld

there are known "knowns." There are things we know that we know. There are known unknowns. That is to say there are things that we now know we don't know. But there are also unknown unknowns. There are things we do not know we don't know.

---

So in other words, we may be able to calculate some sort of perceived probability based on what we think we know, but how useful is this information really, when we have no way of judging the degree of accuracy of our knowledge?

I have to ask in order to be sure, are you saying that perceived evidence and legitimate evidence are the same?

An example I personally like is gravity/relativity. While I don't know the whole theory of relativity, I know that it implies how things with more mass seem to bend space-time (or just space, I don't really remember), effecting gravity, or just causing it in the first place. While both had to do with the same subject, neither Newton's or Einstein's theories are considered wrong. One was simply made in the absence of evidence used in the other, which, while it didn't give it a huge picture to work with, made very precise as fundamental laws.

Quote:

---

Originally Posted by Xedan

An example I personally like is gravity/relativity. While I don't know the whole theory of relativity, I know that it implies how things with more mass seem to bend space-time (or just space, I don't really remember), effecting gravity, or just causing it in the first place. While both had to do with the same subject, neither Newton's or Einstein's theories are considered wrong. One was simply made in the absence of evidence used in the other, which, while it didn't give it a huge picture to work with, made very precise as fundamental laws.

---

I don't quite understand, how does this apply?

You have a good point, Dub. I guess the saying should be, "Absence of proof is not proof of absence."

Quote:

---

Originally Posted by Xaqaria

DuB, my position is essentially that Bayes' rule cannot be applied to the type of situations that you seem to want to apply it to for the reason that you yourself have mentioned; it is entirely dependent on the judge's personal knowledge, which is often sorely inadequate when judging these topics.

---

I don't really know which situations you think this is. Bayes' Rule is applicable in any situation which involves evidence, period. Furthermore, Bayes' Rule can never be wrong in the same way that 2 + 2 can never equal 5. It is true as a matter of a priori knowledge. (Hence this thread's placement in the Math forum.)

Quote:

---

Originally Posted by Xaqaria

So in other words, we may be able to calculate some sort of perceived probability based on what we think we know, but how useful is this information really, when we have no way of judging the degree of accuracy of our knowledge?

I have to ask in order to be sure, are you saying that perceived evidence and legitimate evidence are the same?

---

I'm not sure what you mean by "legitimate evidence," but I'm going to assume you're referring to the objective, unknowable probabilities with which events will occur. If this is the case, obviously the two are not the same, but it makes no sense to speak of the objective probabilities because these are, well, unknowable. "Perceived" probabilities are all that we can ever have. Even if God himself stepped down from Heaven and revealed to us the objective probability that some event will happen, we paradoxically still wouldn't have the objective probability; we would only have our personal perception of what He said. We will have a perceived probability, plus some nonzero probability that we misheard or misremembered Him. Focusing on subjective probabilities is not some peculiarity that is unique to Bayes' Rule, it is a basic fact of epistemology.

What you refer to as "unknown unknowns" refers directly to the possibility that our subjective probabilities do not match up with the objective probabilities. Since the latter are unknowable, this will always be possible, regardless of the particular situation we're considering. However, it seems rather silly to say that our subjective probabilities are therefore uninformative, since this is equivalent to saying that it is impossible to have "true" knowledge about anything, ever. It is not a situation-specific position at all which you seem to be taking, it is rather one which precludes the possibility of knowing anything. To the extent that it is actually possible to know things, and therefore to evaluate our beliefs in the light of known evidence, Bayes' Rule is a normative description of how this should be done. And Bayes' Rule tells us that absence of evidence is always evidence of absence.

Xedan, I understand your confusion. It's hard to syntactically convey the difference between the two positions, but semantically they differ completely. "Absence of evidence" in the case from the OP refers to the situation in which evidence would be entirely possible, but has for whatever reason not obtained. In other words, we have looked but have not found. These cases are subject to Bayes' Rule. In the other sense of the phrase, the sense in which I have said the very question of evidence is meaningless, there could never be evidence--that is to say, the "evidence" in question would not affect our beliefs about the hypotheses regardless of whether or not it obtained, and therefore this "evidence" isn't actually evidence at all. As they say, it's "not even wrong," it's irrelevant. In my unicorn example, for instance, the very question of how the evidence for or against unicorns bears on the location of my cell phone is entirely meaningless. The two are completely independent, so it doesn't make sense to call unicorns "evidence" at all. However, this is not the case for any of the interesting questions, such as the existence of (Santa Claus). If we ever saw Santa Claus, we would take this as damn strong evidence that he actually exists. This implies that the correlation between seeing Santa Claus and the existence of Santa Claus is different from zero, which means that Bayes' Rule applies, which means that failure to observe Santa Claus is evidence of his nonexistence.

I disagree that failure to observe something means that it doesn't exist and I can state several examples.

Black holes were theoretical physics for 200 years, but no one had ever seen one until about 50 years ago.

Neutrinos were theoretical and the only evidence of them was the the universe seemed more massive than it looked, and were proven only recently.

Single pole magnets still elude physicist, and they've been looking at them for hundreds of years. The mathematics say that you should be able to create a single pole magnet, but it as of yet, doesn't work.

Quote:

---

Originally Posted by DuB

Furthermore, Bayes' Rule can never be wrong in the same way that 2 + 2 can never equal 5.

---

2 plus 2 can equal 5. Haven't you read 1984? My sig quote deals directly with that part of the book.

Quote:

---

Originally Posted by ninja9578

I disagree that failure to observe something means that it doesn't exist and I can state several examples.

Black holes were theoretical physics for 200 years, but no one had ever seen one until about 50 years ago.

Neutrinos were theoretical and the only evidence of them was the the universe seemed more massive than it looked, and were proven only recently.

Single pole magnets still elude physicist, and they've been looking at them for hundreds of years. The mathematics say that you should be able to create a single pole magnet, but it as of yet, doesn't work.

---

Just like experimental observations, theoretical results constitute valuable information. We do value concrete evidence more than theoretical, but all information works in a way that it makes one hypothesis more likely than the other.

Furthermore, if it wasn't for theoretical results, no one would know how to look for experimental evidence that can show whether your examples exist or not.

Quote:

---

Originally Posted by ninja9578

I disagree that failure to observe something means that it doesn't exist

---

That is clearly not what I've been saying with this thread. Be honest, ninja: did you actually read the OP?

Quote:

---

Originally Posted by Xedan

2 plus 2 can equal 5. Haven't you read 1984? My sig quote deals directly with that part of the book.

---

This has nothing to do with anything we have discussed here.

I'm not sure we're on the same page, DuB.

By legitimate evidence, I mean evidence that is known to have a connection to phenomenon it is addressing. With your phone, you 'know' that a ring is evidence of the presence of a phone because you have a long case history of those two phenomena accompanying each other.

The only evidence we have to connect to ghosts is steeped in anecdote, myth and mysticism. We have no accurate way to judge the quality of this sort of evidence and so not only can we not be sure that such evidence should be present, we can't even really be sure that witnessing this kind of evidence would correlate to what we are trying to predict (the presence of ghosts). We also may ignore pertinent evidence because it appears unrelated.

Let me use an example. You've probably heard of the theory of spontaneous generation, but I'll tell it for those who haven't. For hundreds of years before 1668, a commonly held belief was that creatures could come in to existence spontaneously. This was the idea that non-living objects can give rise to living organisms. (s.) In 1668, an experiment was done by Francesco Redi in Italy to determine if meat gave rise to flies. It was commonly observed that when meat was left out, fly larvae would appear in it and so a correlation was drawn between the two. Francesco Redi was able to show that if the meat is covered, flies do not appear and that new flies only come from meat that other flies have landed on.

Still though, after this experiment it was still common to believe that smaller organisms could spontaneously generate, that is how deeply some false correlations were ingrained in people.

Now, more than 300 years later we know that nothing springs up from nothing, but we have realized that it is actually possible for non-living things to give rise to living organisms. We have even observed non living things assembling into the in between building blocks; things that are not exactly alive but that are essential for living things to exist. What it took to make these realizations was not proper logic, but proper tools of observation, something we still may lack to explain other even more subtle phenomena. Remember that many people believe they have received evidence, but we live in a society that trusts the judgement of the tool over the unaided observer. I'm not arguing that this position should change necessarily, I'm just stating it for what it is. Many of the topics (not all) that I am under the impression we are addressing here ("woo") have not been subjected to scientific rigor, so any attempt to apply judgement to them is basically starting from scratch. Its not exactly as easy as watching an apple fall from a tree, in other words.

I don't think we're on the same page either, but I think we're getting closer to figuring out exactly where the disagreement lies. You seem to be emphasizing the fact that not all evidence is of equal reliability, and further that we can't always necessarily tell how reliable the evidence actually is. These are actually two very different points. They basically refer to two different kinds of uncertainty: for this discussion, I'll call the former "relative uncertainty" and the latter "absolute uncertainty." Relative uncertainty refers to the fact that not all evidence is created equal, and we therefore can't depend on every piece of evidence to the same extent. Absolute uncertainty refers to the fact that we can't ever be truly certain that we've even correctly evaluated the degree of relative uncertainty in the situation; that is, we may be placing too much weight on the wrong evidence or too little weight on the right evidence. I acknowledge that these are both true facts that we have to deal with, but disagree that these uncertainties preclude Bayesian reasoning. In short, the very purpose of Bayes' Rule is to deal with relative uncertainty, and so this sort of uncertainty can of course never be problematic. Absolute uncertainty, on the other hand, is always present in any situation, whether we're discussing ghosts, cell phones, or whatever, so to fall back on this "argument from absolute uncertainty" in a truly even-handed way is to appeal to a form of radical skepticism which robs us of any possible epistemic traction.

Let's reconsider some of our examples first in terms of relative uncertainty and then in terms of absolute uncertainty.

Clearly ghosts are harder to find than my cell phone (on most nights), but the two problems can be approached through the same inferential steps. It is absolutely true that the evidence we have for determining the location of my cell phone (i.e., the fact that the phone will dutifully ring when called) is very reliable; it is "legitimate" in your words. It is equally true that the evidence for ghosts is very unreliable; our evidence for ghosts depends mostly or entirely on hearsay and myth. It is not true that this latter form of evidence is "illegitimate" and that the question of ghosts is therefore untouchable.

To illustrate this, consider what would happen if, as I walked into my kitchen, the cupboards suddenly flew open and my silverware started flying out and spinning around the room in circles, and that there was a family of translucent, spooky people in old-fashioned clothes seated at my dinner table. What would happen to my personal belief about the existence of ghosts? If we are truly taking these events to be "illegitimate evidence," nothing at all should happen to my belief. The connection between these events and the existence of ghosts is merely "steeped in anecdote," and should therefore be ignored. On the other hand, if my belief in ghosts raises even a little in response to observing these events, we can only conclude that this evidence has some degree of legitimacy after all.

Both seeing a ghost and hearing my cell phone are valid forms of evidence; they differ only in their relative uncertainty. Both are "legitimate" in the sense that they are potentially informative toward their respective hypotheses--they are both in principle capable of affecting the hypotheses in positive or negative directions--but the evidence for or against the location of my cell phone is precise and relatively certain, while the evidence for or against ghosts is imprecise and relatively uncertain. Capturing this relative uncertainty is a fundamental aspect of the Bayesian reasoning process. We simply adjust the probabilities P(E|H) and P(~E|H) accordingly, and this in turn alters the extent to which observing or not observing evidence affects the hypotheses.

The stronger part of your argument seems to be that these situations differ in terms of absolute uncertainty as well. That is, we are more likely to be putting our money on the better evidence in the case of cell phones than in the case of ghosts. However, this argument is fundamentally incoherent because the degree of absolute uncertainty in a situation is, by definition, unknowable! It cannot be quantified, and even estimating it is dubious. Whether or not the questions have been subjected to scientific rigor is completely irrelevant. Just as there are infinitely many potential reasons why looking for visual evidence for ghosts will never turn up ghosts, there are infinitely many potential reasons why listening for my cell phone's ring will never turn up my cell phone. We looked at some of these latter reasons above: a mischievous friend could have tampered with the ring tone, the phone's battery could have run out, etc. And those are just the obvious, likely reasons; it is a possibility, however small, that an invisible gnome ate my cell phone and its little belly is muffling the ring. The fact is that not even our imaginations can put a cap on these potential reasons. It is therefore incoherent to say that there is less absolute uncertainty when looking for my cell phone than when looking for ghosts because absolute uncertainty is, by its very nature, boundless and unknowable.

It's a very short logical step from there to the hard truth that we can never be completely certain of any external knowledge, ever. Absolute uncertainty is a fact of life, and it's one that holds equally true for all empirical questions, whether it be spontaneous generation or Santa Claus. From here, there are two basic choices we can take. One is to throw up our hands and declare any empirical inquiry, scientifically based or otherwise, as ultimately a waste of time. Your position, if applied in an even-handed manner, appears dangerously close to this stance. The second possibility is to say that our subjective degrees of belief in events are all that matter. They are our only tools for estimating the ultimately unknowable features of reality, so let us not worry about the inescapable possibility that they are objectively wrong, because there is no improving upon them. And if our subjective degrees of belief are all that matter, we must recognize that Bayes' Rule is the very calculus of updating them. To the extent that we can believe anything about the world, Bayes' Rule describes what our beliefs should be.

It's not like I ever needed this to falsify a claim like UFOs before, but it's nice to know we have even more options xD It's almost like a menu.

----

Quote:

---

Originally Posted by Universal Mind

"Absence of proof is not proof of absence."

---

Hmm.. Just to avoid misinterpretations, this should be:
"Absence of proof is not necessarily proof of absence."

There is a crucial difference between the 'not' and the 'not necessarily' logical operators. 'Not means negation, 'not necessarily' means no implication.

So absence of proof can be proof of absence?

Quote:

---

Originally Posted by Universal Mind

So absence of proof can be proof of absence?

---

I was just saying your statement could be misinterpreted.

In mathematics, absence of proof can be proof of absence.

How?

So, based on your reply to the changed ring tone problem (and I'm not used to reading through such technical speak, especially right before bedtime, so forgive me if I'm wrong) an unsaid piece to your interpretation of this theory is that evidence of abscence is subjective to the observer. For example, if we took two people, gave person A knowledge of the alphabet from A-M and gave person B knowledge of the letters N-Z and had them stare at a wall with an O drawn on it, person A would rightfully say there is evidence of abscence of the alphabet from the wall whereas person B would (also correctly) say there is none? Because person A didn't have a knowledge base encompassing the part of the alphabet including O he would see evidence of abscence when in reality there is proof of existance of the alphabet?

Would I be correct in making that assumption, that according to what you've been saying evidence of abscence is evidence only to the observer and people with applicable knowledge base similar to that of the observer?

I'm not saying that evidence of ghosts in unreliable, or that the uncertainty is necessarily greater in this case. If one were to see a ghost as you describe that would be a matter of positive evidence. What I am saying is that from the personal perspective of someone who has never seen a ghost, or from the broader scientific perspective that there has never been documented evidence of a ghost, one cannot legitimately claim that evidence for ghosts is indeed absent, since we cannot be sure what that evidence would even look like if it were present.

I'm not saying that the existence of ghosts is untouchable either, only that barring the perspective of someone who believes they have in fact seen a ghost, this phenomena has not yet been touched at all.

I don't know, I think we've reached the point of a run around. I'm not even really questioning the fact that one can gleen certain probabilities from this exercise, only the deeper philosophical efficacy of the whole matter.

Quote:

---

Originally Posted by rose_red

Would I be correct in making that assumption, that according to what you've been saying evidence of abscence is evidence only to the observer and people with applicable knowledge base similar to that of the observer?

---

Yes, that is exactly correct. Although I should point out that I haven't left this point "unsaid" at all; it's been a major topic of discussion in many of the posts posts following the OP. But given the sheer length of many of the posts in this thread (particularly my own), you can be forgiven for not having read them all. :wink:

Quote:

---

Originally Posted by Xaqaria

I don't know, I think we've reached the point of a run around. I'm not even really questioning the fact that one can gleen certain probabilities from this exercise, only the deeper philosophical efficacy of the whole matter.

---

Fair enough.

Dub, is the following statement true or false? Absence of proof is not proof of absence.

Given that we are talking about inductive reasoning, it is clearly true. In logic, inductive conclusions are rendered more probable if they follow from the premises, but they are never logically entailed by those premises. Proof need not apply. In mathematics as well, failure to provide a proof is not a proof to the contrary; you have to actually prove the contrary (a refutation). Interestingly, even proving that a certain property is unprovable is not the same as refuting that property--hence the apparent paradox of the incompleteness theorems.

Very well written thread.

Quote:

---

Originally Posted by DuB

Given that we are talking about inductive reasoning, it is flatly false. In logic, inductive conclusions are rendered more probable if they follow from the premises, but are not logically entailed by those premises. Proof need not apply. In mathematics as well, failure to provide a proof is not a proof to the contrary; you have to actually prove the contrary (a refutation). Interestingly, even proving that a certain property is unprovable is not the same as refuting that property--hence the apparent paradox of the incompleteness theorems.

---

You seem to agree with the statement. Are you sure you noticed the "not" in it?

Quote:

---

Originally Posted by Universal Mind

You seem to agree with the statement. Are you sure you noticed the "not" in it?

---

:doh: Fixed...

Okay, we're on the same page then. So while you agree with my statement concerning proof, you disagree with the statement when the word "proof" is replaced with the word "evidence" because evidence only has to serve the probability of the truth of a negative claim while proof has to show it to be definately true. I agree with you. Absence of evidence of x does serve the probability of not x. So apparently the saying should involve the word "proof" instead of "evidence".

Quote:

---

Originally Posted by Universal Mind

Okay, we're on the same page then. So while you agree with my statement concerning proof, you disagree with the statement when the word "proof" is replaced with the word "evidence" because evidence only has to serve the probability of the truth of a negative claim while proof has to show it to be definately true. I agree with you. Absence of evidence of x does serve the probability of not x.

---

Exactly.

Quote:

---

Originally Posted by Universal Mind

So apparently the saying should involve the word "proof" instead of "evidence".

---

Well, it's not exactly that it should refer to proof rather than evidence, it's that people need to be clear about which version of the claim they're really asserting. I call the version involving "proof" the weak version (because it is obviously true, but neither interesting nor relevant) and the version involving "evidence" the strong version (because it is both interesting and relevant, but flat wrong). They are both bad arguments, but for very different reasons.

Unfortunately, people very often tend to get the two versions mixed up, typically such that they speak explicitly of the strong version but interpret it implicitly as the weak version (i.e., they speak of "evidence of absence" but really are talking about "proof of absence"). This gives the statement the illusion of being interesting, relevant, and true all at once. It isn't.

I think too many people really do believe that absence of proof is proof of absence. To them, the nature of the principle is not so obvious. The claim that not finding WMD stockpiles in Iraq proves that the WMD stockpiles were never there is a classic example, and people more commonly use the idea in regard to God's existence and his nonexistence. Of course, people also use reasonable theological arguments that sound a lot like the statements that miss the nature of proof but are different.

I certainly believe Abscence of evidence is NOT evidence of absence.

Allthough indeed alot of commercial mystics, doctors and healers( read hustlers) abuse this line to provide some credibility for their fraudulent businesses.

There are, however, many humanly experienced phenomena that science cannot explain. Not even remotely. Scientists are often so bold to then conclude this is evidence of falsehood/absence. However that seems rather bold and self absorbed to me. Instead of simply rediculing theories and phenomena, why do so many scientists never question their own methods of investigation?

You could off course study hallucinatory drug induced and psychotic/schizophrenic visions and the hearing of voices and, after having found no conclusive evidence of ther true objective existance, pass it off as false/illusionairy.

But why not wonder wether the instruments and the method of investigation you used were adequate and suitable for the "realm" of investigation?

Far too many scientists never doubt their own methods or instruments and are only analytical towards the subjkects of investigation. That is off course blind science.

When we keep this in mind, the line "Abscence of evidence is not evidence of absence" indeed holds much truth.

How many of you people here believe in Darwin's evolution theory? I for one do. We all know there are MANY missing links; absence of evidence, but I think it's safe to say that we have enough evidence to at least conclude that Evolution is a very credible theory.

We m,ay have many missing links, but the evidence of quite obvious links are out there: Fish > Frogs> Salamanders> Lizzards.... seems to reveal the truth of the Evolution theory. Even though there is still many conclusive evidence missing.

I think this is very much the same for any other subject that blind science seeks to declare as false. We have many pieces of evidence that suggest the existance of Telepathy, foresight, remote viewing, prophetic visions..etc Like the Evolution theory there is alot of conlusive evidence missing; Absent. But far too many people have experienced these phenomena to simply render them false/illusionairy.

Quote:

---

Originally Posted by SKA

I certainly believe Abscence of evidence is NOT evidence of absence.

---

I'm sure that you don't.

I've already covered the general objection regarding objective (or absolute) certainty--scientific or otherwise--earlier in this thread in some detail, so rather than lapsing into repeating myself, I invite you to read those posts. I know it's a lot to read; I can be a bit long winded, but I invested quite a bit of time into addressing these exact points and I hardly think it's fair to me for you to simply skip over all that and demand that I explain myself over again from scratch. Let me know if you come back with specific rebuttals to what I've written, otherwise we're talking past each other.

I don't even know what you're trying to get across with your evolution example. As you say yourself, there is an absolute abundance of evidence in support of the theory in its broad form, so it's hardly a candidate for these "absence of evidence" type of arguments in the first place.

And please, don't make this about Big Bad Science. This thread has very little to do with science. It's about a neat little formula that tells us how to evaluate and revise our beliefs in a normative and personally consistent fashion. Nothing more, nothing less. An interesting and necessary consequence of this personal consistency happens to be that, if presence of evidence is going to be taken as evidence of presence, then absence of evidence must be taken as evidence of absence.

DuB is right -- this is not science, this is logic alone.

This stuff is really confusing. I think I get the gist of it, though I'll probably have a hard time explaining it to other people :P

It may help (both for understanding and for explanation purposes) if you simply skip over the probability terms I kept referring to and just think of the situation in terms of the cell phone example. I think it's a good example because it's one that is very familiar for most of us, and this familiarity nicely illustrates that, on some level, people really do intuitively understand that absence of evidence has to be evidence of absence. If they really didn't believe this was the case, why would they bother to call the cell phone at all? The fact that people do call their phones when in this situation reveals that they believe, implicitly or explicitly, that the presence or absence of the ring (the evidence) is going to be informative one way or another as to where the phone is (the hypothesis). You can think of it as "cell phone logic." All empirical knowledge is a matter of cell phone logic.

The purpose of referring to the probability terms and to Bayes' Rule at all is partly for the sake of brevity (it's a lot easier and, I think, less confusing to say "P(~E|H)" than it is to say "the probability of not observing the evidence, given that the hypothesis is true") and partly to demonstrate that there is a mathematical and axiomatically true basis to the argument. Even while referring to these terms, I still actually simplified quite a lot. I didn't plug in any actual values--indeed, I didn't even present the formula itself!--and I gave only superficial treatment to the issue of base rates (priors), which is a highly significant factor in this sort of analysis, but as I mentioned in the OP, not one that fundamentally alters the nature of the conclusions. Anyway, if you can wrap your head sufficiently around that part of the argument, more power to you, but as I noted above, the idea simply has intuitive appeal to most people. Even if they don't explicitly realize it or even acknowledge it, they do abide by it.

That does clear things up a bit DuB, thanks for taking the time to explain it to me again :)

I'll have to take another look at the explanation, that involves symbols sometime later.

DuB: good points as always, but I don't think your conjecture is true in all instances.

What about when you can't really assign a probability to something?

For instance,

P = universe has existed forever
¬P = universe has existed for infinite amount of time

Consider the situation before we had the equipment to check for evidence. You can't assign P a probability because you have no information or logical argument either way. Obviously though it would be wrong to say the absence of evidence for P is evidence that P is wrong. By symmetry you could say the same for ¬P, which is clearly absurd.

Quote:

---

In mathematics, absence of proof can be proof of absence.

---

I don't think this applies to anything in mathematics actually. Proof of absence is proof of absence, and that alone, as it were.

The absence of solutions to Fermat's equation were not proven by the failure of anybody to find some, they were proven by Andrew Wiles via algebra.

Absolutely it's not true in every instance, which I made sure to mention in the OP (near the end: "This is true for any set of hypotheses where the correlation between E and H is different from 0, in other words, when evidence is actually evidence"). There are some situations for which there are no justifiable logical inferences to be made about the hypotheses, regardless of whether we view it from a Bayesian perspective or any other. This is another inadequacy of the "absence of evidence" line: it may be a clever turn of phrase, but it's terribly misleading because it conflates two very different ways in which evidence can be "absent."

On the one hand, we can have a reasonably clear idea about what the evidence might look like and which hypothesis would have given rise to the evidence in the most straightforward way, but when we go to look for this evidence, it isn't there. Then whichever hypothesis would have most straightforwardly caused the evidence to be absent becomes more probable at the expense of the other hypotheses. This is what the cell phone example illustrates, and this is almost always the type of situation to which people are referring when they invoke the "absence of evidence" line.

But on the other hand, evidence can be absent because (a) we don't have even an inkling as to what the evidence might look like, so we aren't sure what we should even look for, or (b) we do have an idea about what it may look like, but we haven't looked yet! Essentially, there isn't even any evidence to be absent. This is apparently the situation for the hypotheses concerning the existence of the universe. Before proper measurement was possible, evidence was "absent" in that there didn't seem to be any empirical observations that would support one hypothesis or the other, but this is clearly very different from the previous situations. Drawing the analogy back to the cell phone example, this would be like trying to draw an inference about the phone's location before I even tried calling the phone. Clearly absence of a ring is not informative in that case, because I haven't even looked for the ring yet!

The difference is that in this latter class of situations, the correlation between the "evidence" and the hypothesis is 0. Not hearing a ring coming from my bedroom is equally likely under both of the phone location hypotheses in this case because I haven't called the phone yet. Similarly, if we don't have a proper means of measurement that would allow us to infer things about how long the universe has existed, then not observing those measurements is equally likely regardless of how old the universe actually is; E and H are not correlated. The phrase about "absence of evidence" should not be applied to these cases in either its negative or its positive form, because absence of evidence implies that there is possible evidence which happens to be absent. However, if a certain piece of "evidence" is equally likely regardless of which hypothesis is true, it doesn't make any sense to call it "evidence" in the first place, and in fact I would say that it is by definition not evidence. It's simply irrelevant.

You are right that at least for me the terms evidence and proof are really interchangable. I see that in logic they arnt. In everyday speech it seems most people do see them as such. Im gonna seem like a grammer nazi if I start correcting people on that LOL