Thread: Are Some Things Not Testable?

In lieu of the recent surge of claims about science ("science can't do this" and "this is beyond the scope of science" and so forth) I was considering how one might approach testing something that, hypothetically for our purposes, exists but is difficult to reproduce the effects of.

Extra Sensory Perception and aspects of magical practice have been somewhat popular around here, so we can use one of these as an example. If we say that, for instance, I can actually effect the reality around me by being conscious of it and "feeling things at it" (this would be the unproven but true reality that we wouldn't know is true), but that the effects of imposing my will were unpredictable, can it or can it not be tested and verified through any method we know about? Furthermore if the very nature of the ability was to be somewhat unpredictable (as in, based on many variables beyond our control) what does that imply about our capacity to verify real things with complex, unpredictable behavior? How does that effect our perceptions of what we consider possible or probable?

Have at me.

The essential ingredient for a statement to be theoretically testable is that it be reducible to a statement of the form "A implies B." Then it is tested by creating the condition A and verifying that B holds whenever A does. For it to be practically testable, we of course need to be able to create A.

So testing something like ESP would essentially boil down to finding an A and a B. It seems that most of experiments to date have focused on B being something along the lines of "The subject is able to acquire information at a rate higher than a guess." That I know of, not one of these experiments that has shown positive results has been repeatable. It seems to me that people wanting to test it should focus on the A part and tighten up their understanding of what preconditions are necessary for B to manifest itself.

EDIT: As for the question about being able to test complex things, I would just point at QM. It is complex and we end up having to measure its effects based on probabilities but it still got figured out. If the effect that we are assuming is real is so unpredictable that eventually a set of testable predictions for it could not be worked out then I would have to question the meaning of even asserting its reality. It would seem to be synonymous with the assertion that the universe is too random for anything to be testable simply because somebody could think at whatever we were testing. Yet on the other hand, if there are limits to it that leave some things meaningfully testable, then one could work that out and at least be further along in ones quest to understand the mechanisms at play and hence being able to make meaningful predictions.

If the statement was (Attempt to do X) => (Weird Shit), then I would not consider the statement to fail if in one lab, it started raining elephants and in another lab, a pride of lions appeared out of thin air. That would obviously not be a satisfying prediction but it would be something to start with.

Well you really need to break it down into smaller parts. If there are to many variables that are causing it to become very unpredictable, you need simplify it. Ideally in science you only test one variable at a time. Luckily in this example we can in fact simplify the problem, and narrow it down to one variable.

What you do is create a situation. Say you stand in the middle of the room and you try to effect something with your mind. As a control, someone stands in the middle of the room and does nothing. You test this a 100 times. If nothing happens the same frequencies of times as the control, then this should prove that your will isn't effecting anything. If something happens more often, then this should prove that you are doing something that actually effects your surroundings.

You can test this with any of the things to be honest. Compare the results to doing nothing. If any of its true, the results should show something happening. Even if the results are different and unpredictable, the results should be constant in that they show something happens on a regular basis.

Originally Posted by Invader

If we say that, for instance, I can actually effect the reality around me by being conscious of it and "feeling things at it" (this would be the unproven but true reality that we wouldn't know is true), but that the effects of imposing my will were unpredictable, can it or can it not be tested and verified through any method we know about?

I'm not sure that I fully understand your question. If we don't even have an idea about what an effect might look like (nevermind the underlying mechanisms causing the effect), why would we even be talking about it, much less trying to verify or falsify it?

To say that "when I impose my will, I have no idea what will happen" is not only a basically meaningless statement, it doesn't seem to have anything to do with ESP or related phenomena, where there are some ideas about what the effects might look like--however clear or fuzzy, "objective" or "subjective," etc.

Allow me to specify then. Let's imagine a scenario in which you live out a day of your life doing whatever it is you do, and nothing out of the ordinary happens. In our hypothetical scenario, we're going to replay the day with the addition of a friend (who doesn't interact with you directly) who willed for you to have a more positive encounter, and the experience comes out different. So when I say that the effects are unpredictable I mean that there's a general goal being achieved, and that there are no specific events that can be anticipated as being the direct outcome of, in this case, wishful thinking. Is there a thing like this that cannot be accurately tested if it existed? Is the demand for precision in an experiment a limiting factor when testing something that may have vague or general effects?

Originally Posted by Invader

Allow me to specify then. Let's imagine a scenario in which you live out a day of your life doing whatever it is you do, and nothing out of the ordinary happens. In our hypothetical scenario, we're going to replay the day with the addition of a friend (who doesn't interact with you directly) who willed for you to have a more positive encounter, and the experience comes out different. So when I say that the effects are unpredictable I mean that there's a general goal being achieved, and that there are no specific events that can be anticipated as being the direct outcome of, in this case, wishful thinking. Is there a thing like this that cannot be accurately tested if it existed?

Well, since we're allowing ourselves the ability to "replay" time in this hypothetical scenario, there's no reason why we wouldn't be able submit the above claim to empirical test. We simply need to assess whether Day 2 (the day replayed + outside wishful thinking) was generally more positive for the person than Day 1 (the "control" day). There are a number of ways to do this: We can ask the person him or herself at the end of each day a series of questions concerning how satisfied they are with the day's events and then compare the two sets of answers; we could have a group of outside judges who are blind to hypothesis read over each day's events and decide which one generally contained more "positive" events; etc. The only question here is how we want to operationalize "positive encounters," because this will help us decide which method is most appropriate to test the claim.

Originally Posted by Invader

Is the demand for precision in an experiment a limiting factor when testing something that may have vague or general effects?

Precision in an experiment is nice, but it's not really the principal demand. It's more important that confounding variables are held constant and alternate explanations accounted for. These factors usually have nothing to do with precise measurement.

The problem with vague claims is not that they aren't testable -- they are -- the problem is that without giving specific predictions, whoever made the claim can simply change their mind about exactly what they were claiming after the fact. So vague claims often lead to noncumulative science, but they aren't inherently untestable or unscientific.

Originally Posted by DuB

Well, since we're allowing ourselves the ability to "replay" time in this hypothetical scenario

Misunderstanding. I didn't mean that we'd have the literal ability to replay the day as a means of testing, only that there'd be a difference if the day could be replayed with the extra person in order to illustrate that the wishful thinking would have some overall but non-specific effect.

Originally Posted by DuB

The problem with vague claims is not that they aren't testable -- they are -- the problem is that without giving specific predictions, whoever made the claim can simply change their mind about exactly what they were claiming after the fact. So vague claims often lead to noncumulative science, but they aren't inherently untestable or unscientific.

Would it be possible then to have a computer that can model a situation according to our current understanding of physics and then have an observer attempt to effect the outcome, or is something fundamentally wrong with that?

I can think of an example of what I'm trying to suggest. If we didn't know about electromagnetism, for instance, and decided to build a machine that worked on other principles that could make a theoretical model of a situation based on that understanding of physics, the information it would produce would be in line with that model of physics. Say we have two conducting wires next to each other, one with alternating current flowing through it, and the other disconnected from any power source. The simulation would suggest that the second wire has no current moving through it. The real world would demonstrate the contrary, that the second wire has current induced in it from the first.

I'm still not sure that what I'm saying is clear, or at least that modern computers can simulate complex events and still be true to our understanding of the physical laws without being effected by any other unknown phenomenon that we aren't aware of yet. Does that make sense?

Originally Posted by Invader

Misunderstanding. I didn't mean that we'd have the literal ability to replay the day as a means of testing, only that there'd be a difference if the day could be replayed with the extra person in order to illustrate that the wishful thinking would have some overall but non-specific effect.

I understood that you intended it as a purely hypothetical suggestion.

Originally Posted by Invader

Would it be possible then to have a computer that can model a situation according to our current understanding of physics and then have an observer attempt to effect the outcome, or is something fundamentally wrong with that?

I can think of an example of what I'm trying to suggest. If we didn't know about electromagnetism, for instance, and decided to build a machine that worked on other principles that could make a theoretical model of a situation based on that understanding of physics, the information it would produce would be in line with that model of physics. Say we have two conducting wires next to each other, one with alternating current flowing through it, and the other disconnected from any power source. The simulation would suggest that the second wire has no current moving through it. The real world would demonstrate the contrary, that the second wire has current induced in it from the first.

I'm still not sure that what I'm saying is clear, or at least that modern computers can simulate complex events and still be true to our understanding of the physical laws without being effected by any other unknown phenomenon that we aren't aware of yet. Does that make sense?

I think it makes some sense. There are a couple different points here.

To address the issue about what modern computers can and can't simulate: computer models often exhibit interesting and unexpected emergent behavior when only basic assumptions are programmed in, and these phenomena can be tested against the empirical facts. So your example about programming in the assumptions given by a certain theory of electromagnetism and then observing whether the emergent electrical behavior matches well with empirical data makes sense.

Whether or not a human observer could passively affect the outcome of the simulation is a somewhat different issue. It really comes down to the type of claims being made by the observer. To answer your other question, there is not any fundamental reason that I can see why they couldn't affect the simulation. The simulation is implemented as a part of the natural world just like everything else and thus should be open to this kind of manipulation. It just seems to me like it would be an unusual context in which to demonstrate these kinds of effects, and I could imagine some of the people who purport to be able to do this sort of thing to protest, perhaps reasonably, that it's an entirely different situation when dealing with computer simulations. And if they're not making this claim in the first place, we wouldn't really have a good reason for testing it

You are talking about two different concepts. What a thing is, and what it can do. What a thing can do limits the methods by which one can test it--which only means use that thing for a specific response. However, the statement is correct; Science cannot do anything--that statement is an anthropomorphism.

And some of the things you state, especially about individual experiences, that they can never be social experiences has always been true--that is just a simple point in understanding. We can not standardize, as food what we have already ate, but we can standardize what may be available to be eaten. Or again, we choose or constructs standards things from which abstractions may be made, we cannot standardize if or if not a particular person can make an abstraction.

Thus proof requires individual participation in a naming convention which means that proof is not universal, but particular to individuals. One certainly cannot prove anything to no one, or every one, not even to some ones--as proof is the compliance of grammar with perceptual data--no one can prove anything for anyone else. Even Aristotle, which was not the sharpest tool in the shed, understood that if one cannot perceive a thing, or lost perceptual ability, then that portion of knowledge was not to be had by them.

This is just basic stuff. But it does indicate that the common intellect is not interested in knowing, only in repeating by rote.

Everything is testable. We may not be there in our ability to test it, but everything is. Well, It is never possible for everything to be testable at the same time, but we will continue to advance and to test. There is no such thing as 'beyond the realm of science', Science IS everything, whether or not we know it.


In conclusion:

Any sufficiently advanced technology is indistinguishable from magic.
--Arthur C. Clarke

Originally Posted by Indeed

Everything is testable. We may not be there in our ability to test it, but everything is. Well, It is never possible for everything to be testable at the same time, but we will continue to advance and to test. There is no such thing as 'beyond the realm of science', Science IS everything, whether or not we know it.

This is indeed the popular view. But I think there is something to say about the process which science follows (namely, the scientific method) which confines the applicability of science to things which are observable, measurable, and empirical. Thus science, by its own rules, only applies to things which are perceivable and is not necessarily a theory of general knowledge. Christopher Langan addresses this in one of his papers, as follows
"[The scientific method] is a theory only of sensible things that have two further attributes: they are non-universal and can therefore be distinguished from the rest of sensory reality, and they can be seen by multiple observers who are able to “replicate” each other’s observations under like conditions. Needless to say, there is no reason to assume that these attributes are necessary even in the sensory realm. The first describes nothing general enough to coincide with reality as a whole – for example, the homogeneous medium of which reality consists, or an abstract mathematical principle that is everywhere true - and the second describes nothing that is either subjective, like human consciousness, or objective but rare and unpredictable…e.g. ghosts, UFOs and yetis, of which jokes are made but which may, given the number of individual witnesses reporting them, correspond to real phenomena."

If something isn't testable by science, it cannot influence our world in any way. Testable is for me the same as detectable though. I guess some things can be very hard to test, like the observation of UFOs. As PhilosopherStoned says, we observe B but we don't have an A. This is usually where most of the fuck ups happen, as people insert some imagined paranormal explanation as the A in the equation. Stop doing that.

Everything true is testable.

Take shared dreaming. Many people call it testable, yet they have experienced it. Therefore, they must have tested it and got a positive result, no matter what the test was. Please don't misunderstand me; I don't believe in SD. I have done numerous tests on SD and they have shown negative results. PM me for more, since telling too many people will invalidate the results.

Another example is lucid dreaming. Nearly everybody except Stephen LaBerge thought is was untestable, until LaBerge came and said it was testable and got a PhD for that.

I believe that everything is testable, the problem lies in asking the right questions and looking in the right places. Take electromagnetic energy for example. We cannot physically see it but we can measure it and understand how it interacts with matter and other phenomena in the universe. The same thing could be said about gravity. So I suppose I am saying that yes one day science will figure it out, like I said its a matter of asking the right questions.

God can't be tested directly, which means that you can't actually make God a guinea pig. However, it is possible to test it indirectly. Anything is testable on way or another.

Originally Posted by ludr

God can't be tested directly, which means that you can't actually make God a guinea pig. However, it is possible to test it indirectly. Anything is testable on way or another.

Scientifically there's no difference between direct and indirect. Either something is testable, or it isn't.

God. Highly educated people know there is no such thing as god, but there is no way to test for it.

Originally Posted by ninja9578

God. Highly educated people know there is no such thing as god, but there is no way to test for it.

It is possible to test if something is present, but if something doesn't exist someone will always claim you'll find it under the next rock you turn.