Printable View
I have often wondered whether or not psychology is a pseudo-scientific branch of human knowledge. On the one hand psychoanalysis attempts to empirically observe a patients behavior and words but on the other hand I get the impression that many psychological explanations for human behavior are too speculative given the questionable evidence(the possibility that the patient is lying) at hand.
Pseudo-science? No, not at all. It's definitely a science. Good psychological experiments are set up in such a way that the patient doesn't know enough about the situation to lie. Also, psychology is not only focused on behavior that a patient would lie about at all. It's simply the study of the mind, its mental states and processes. Sleep studies, for example, are psychological studies. A patient can't lie about their brain waves.
Unless you mean pop-psychology, which is mostly bullshit. :p
There's a lot more math in psychology than most people realize. Studies are subject to as rigorous a peer review as any other science. Now, psychiatric medicine as it's put into practice is another matter. Particularly at 'street level,' very little psychology is practiced. The sole aim of interacting with patients is to determine what drug(s) to throw at the problem, and the drug of choice depends as much on fads and pharmaceutical marketing as any genuine familiarity with current research. Really, it's a problem with medicine in general in this country: doctors become familiar with current research primarily through pharmaceutical marketing campaigns, which generally consist of $100-$125 per-plate dinners at fancy restaurants accompanied by a Power Point presentation, all on the drug manufacturer's tab, as was the original research in most cases. So, the question of what to prescribe comes down to who bought you the best steak while making a more-or-less adequate case for their product.
Yes it is a pseudo-science on a whole. It's getting better though. Most have moved on from Freud's psychoanalysis, except for a few specific things.
Tao is correct about all that drug stuff. America needs to catch up to Australia on that one. We've banned all "gifts" from big pharma to doctors and misleading advertising is followed up by a fine (which is measly usually (to them anyway lol)).
There's been a thread about this not too long ago. I posted a lot about the upcoming DSM V.
At the moment they basically just keep adding more and more afflictions. When it is blatantly obvious that a great majority of them are just various presentations of anxiety, OCD or schizophrenia.
But as I said, some research is being done with fMRI. To see what is causing the problems. Apparently nicotinic receptors are a prime candidate at the moment for a lot of mental problems.
But yes, atm, with the way it's being practiced, it's pretty much as close to pseudo science as you can get, while still maintaining a faint air of scientific entitlement.
I would think that most clinical psychologists (for example) are just as good scientists as any other. All the psychology I've been taught and witnessed at uni was all just as scientific and evidence based as my biology classes.
You would think that wouldn't you. The truth is that it isn't.Quote:
Originally Posted by Pensive Patrick
If you listen very hard you can hear a DuB typing furiously away at this very moment.
This is true I didn't take into account experiments with the patient unaware. I guess I was more addressing psychoanalysis "the talking cure". I would say that psychology is very concerned with behavior, in many psychology schools mental states are examined through behavior. I think the study of the mind through empirical means is more in the realm of cognitive science (they would also consider themselves scientists not psychologists).Quote:
Originally Posted by Savy
@Taosaur- I'm am very skeptical of a "doctor" who prescribes me pills to treat everything, this is another reason why I think alot of people are wary of psychology.
I guess the reason why I brought this thread up is because I feel alot of psychological claims describing behavior are too speculative and based off of questionable evidence. I agree with Tommo I think it is getting more scientific in its methods in the latter part of the 20th century.
Yea, I'm pretty postive psychology is a science. I don't feel like studies such as the Milgram experiment to be too speculative, do you? The test was redone several times, always with the same results. Also, what about Pavlov's dogs? That's also psychology, and it doesn't even have to do with humans. What about studies in the behavior of babies? Certainly conclusions are drawn from observation, and we can never know absolutely if we are correct, but the odds are that we are, especially when we can create direct cause and effect responses. Often, patients aren't even asked questions that they can lie to. They don't have an opportunity to lie, and it's not really their answers we're interested in, anyways. We're interested in the behavior.
Psychology goes so much further than crappy therapists who throw drugs at you.
The reason I've said "I would think that" is because I don't have any evidence to back my assumption up.Quote:
Originally Posted by tommo
Do you have evidence that clinical psychologists aren't as good scientists as say molecular neurobiologists?
This thread is ultimately about our opinions so I expect we'll just have to agree to disagree...
I had a feeling you'd misinterpret that. By "you", I just meant, people. Like "You would assume psychologists base their diagnoses on scientifically proven grounds", addressing no one in particular.Quote:
Originally Posted by Pensive Patrick
I have plenty of evidence. Just look at the DSM V. There is a website for it to get feedback. At least there was last year. Dunno if it's still up, just Google it.
Therapist =/= Researcher, just like Doctor =/= Biologist and Engineer =/= Physicist. Applied science is rarely as tidy and, well, 'scientific' as theory, precisely because it's applied: it has to work in the real world. A psychological therapist's position is no different in that regard than a physiological therapist's (i.e. doctor).Quote:
Originally Posted by tommo
Perhaps it isn't tidy but applied stuff has to be just as scientific, really... in the sense that it has to have empirical basis. Engineers don't speculate about tensile strengths; they find them out, accurately and repeatedly, and that's basically exactly what science is.
Yes, it is a science, the workings of which are largely unknown and imprecise given it's complexity, emotional involvement, and the fact that it's trying to grasp a birds eye view and describe a process that it is. lol
Empirical data inform the engineer's, doctor's and psychologist's decisions, but many of those decisions are not and cannot be made empirically. In real-world situations, there are often competing explanations, approaches and methods, as well as extenuating circumstances to be weighed. My point was that just because application cannot be empirical in every respect does not discredit what science does inform it.
Psychology is a science. Although a lot of its findings are absolute.
Edit: aren't*
Yes.
All my psych classes at uni are scientific. (half the people drop the major after learning that)
Well I just noticed this thread, but I see I've already been mentioned :P
I think it's a fair question given the common perception of what psychologists do. For a variety of reasons, people seem to have an incredibly skewed view of what exactly psychology is and what most psychologists do. This is evident even in this thread. I'll outline two examples below and offer some reasons why I think this is. I'll also offer some of my personal thoughts on the matter.
First, the OP brings up psychoanalysis and wonders, given the status of psychoanalysis, whether psychology has much useful to say about human behavior. Of course, it's a bit like wondering if the theory of vitalism discredits biology, or if phlogiston theory discredits chemistry. These are archaic and, in retrospect at least, frankly ridiculous theories. I've never met a psychoanalyst--I've heard that there still are some, somewhere, but as far as I'm concerned they are mythical creatures. One would be right to question the utility of these theories, but fewer people think to ask if that's really what the fields they ostensibly represent actually do. In case I haven't made it clear yet, psychoanalysis has no relation to anything modern psychologists do.
Second, most people in this thread have asked about the status of clinical psychology in general. Again, those are fair questions, but what's striking to me is that, to most people, that's just what psychology is. However, in reality, slightly more than half of us have no interest whatsoever in clinically abnormal populations. Psychology is the study of how human behavior and cognition work, and accordingly, most of us study normal people. I won't ever have a "patient" and I'm quite sure that any potential patient who knew a thing about me wouldn't want me as their clinician. (And if they doubt that, I'm sure that some of you could warn them properly.) I don't intend to denigrate clinical psychology--what they do is undoubtedly important--I only mean to point out that, while they essentially comprise the public image of what psychologists do, they in fact represent a subset of the field. More accurately, psychologists just seek to understand the why and how of human action.
So, putting aside those issues, we are left with the original question: is psychology really a science? It seems to me that, on any reasonable definition of what "science" is, the answer must be yes. As psychologists we attempt to systematize empirical observation of human behavior and cognition and then to synthesize those observations into general theories of human behavior and cognition. What more would you ask of the enterprise of science? As far as I can discern, most people seem to identify "science" with fields and studies that are what I would call "sciencey"--that is, if you wear white lab coats and fiddle around with colorful beakers of strange, caustic liquids and write out intimidating mathematical equations that are long enough to fill a chalkboard, well then by golly, you must be doing real science(tm)! It's a pretty naive view of what science really is.
Perhaps a more interesting question then is: how good is psychology doing as a science? Here I would probably agree with the majority of posters that the best answer is: not as well as we would like for it to be. Psychology, as it turns out, is really hard to do. Human behavior and cognition are--surprise!--really, really, really complicated. Predicting the amount of money that someone will choose to place on a risky gamble (I study decision making so this broadly represents the type of questions that I tend to ask) tends not be be quite as easy as predicting how how long a textbook will take to hit the ground if I drop it from a second-story window. But what can you do? You try to account for the myriad of influences that go into such a decision as best you can. If you have suggestions for how we can do that better, we're all ears.
@Dub- thanks for responding, it is evident that I have an outdated notion of what psychologist actually do. However I am skeptical of the notion that human behavior can be addressed with a broad stroke. As you have said human behavior is an really complicated.
The example with the textbook was spot on, we can use induction to infer how long it will take to hit the ground but it gets rather tricky when we try to predict how humans will act in a certain situation. We can use controlled experiments to observe human behavior(of the participants) and then apply the results to the whole population(when the experiment only observed the behavior of 5-10 people). Are social/economic/geographic variables taken in consideration in conducting these experiments?
I framed the question "Is psychology a science?" because I hoped that it would strike an emotional chord with people who support psychology and people who don't as well as spark heated debate but what I really want to know is, is there any validity in the psychological explanations of human behavior?
Well, a study involving 5-10 people would be a little extreme, but it's true that even the largest experiments involve observing data for a relatively small subset of the population and then generalizing those data to the population. We use statistical theory to address this, but it's a fact that such statistics implicitly rely on mathematical assumptions which may or may not actually be realistic. The degree to which wide generalization can be accomplished through experimentation is an open and somewhat controversial issue.
Regarding demographic variables, different areas of psychology (and even different psychologists within each area) approach them in different ways. Some areas, such as so-called "cultural psychology," spend almost all of their time actively searching for differences in basic effects as a function of demographic variables. I think that such data are important, but I tend to disagree with the approach taken by many of these researchers. They appear to be very interested in demonstrating how some of our basic theories seem to break down in unusual cultural situations, but they appear to have far less interest in actually revising those theories to address the issues of when, where, and why these basic effects differ by culture. In my view, science ought to be about progressive theory development. Cultural differences are a fact that we must deal with, but we must do so by developing theories of those differences, not by simply hunting for cultural differences and then, upon finding them, declaring the original theories to be defective. In my estimation, the latter tact is more common than the former, and this is unfortunate.
The most common view, held by most psychologists even if it's never articulated, is that for the kinds of basic mental processes most of us study, there's probably not much good reason to suppose that these basic processes differ widely as a function of demographic variables. Of course, the degree to which that is actually true is an empirical question.
Regarding validity. Psychological explanations for human behavior are all around us. When you say, "My boss refused my raise because he's a prick" or "The reason she can't remember my name is probably that she doesn't really like me," those are psychological explanations. So the question of validity is probably best framed as: are the explanations given by psychological science more valid than the everyday explanations given by non-psychologists? And again, the answer seems to be yes. Consider the fact that ten non-scientists can look at the same set of behaviors and give you ten mutually contradictory, but somehow "obvious," explanations for why the behaviors occurred. Does absence make the heart grow fonder, or is out of sight out of mind? Do birds of a feather flock together, or do opposites attract? Are you never too old to learn, or can you not teach an old dog new tricks? Do clothes make the man, or should you never judge a book by its cover? Everyday attempts at psychological explanation can "explain" any possible behavioral outcome, thereby actually explaining nothing at all. We need scientific psychology to cut through all the bullshit and address the questions of human behavior systematically and rigorously.
It's not at all clear to me that we'll ever have psychological theories that are as precise and elegant as in the so-called "hard" sciences (which, paradoxically, are the sciences which are actually easiest to do). As I wrote earlier, human behavior is a huge complicated mess. But throwing up our hands and declaring its explanation not worth the effort seems to me to be exactly the wrong reaction. I prefer that we dive in and try to sort it all out as best we are able.
Tha's psychiatry. Psychologists cannot prescribe drugs.Quote:
Originally Posted by Taosaur
Psychology is one of the social sciences.
I've heard rumors that in some places psychologists are able to prescribe certain drugs. If that's actually true, I think it is a huge mistake. Take it from me: We are not qualified to prescribe drugs!
@Dub- You are a psychologist?
Euphemisms like "Does absence make the heart grow fonder, or is out of sight out of mind?" are clearly misguided and redundant. This is why whenever someone says something like "I can read into people" I always laugh and think they are just a little bit full of shit. In this regard I can see the need to systematize a range of explanations for human behavior rather than leaving it to the average persons misguided confidence at their ability to "read people".
I think there are aspects of human beings that defy labels or explanations. I do not think humans are a thing or event but a process of becoming and constantly changing making attempts to understand us difficult to say the least. For instance my father has known me for almost 20 years and yet there are still aspects of my personality that he can never know and not because there are things that I don't tell him (I tell him everything) but just for the simple fact that he cannot experience what I experience and I feel he only know the "surface me" and not who I truly am. I feel that who most people "truly are" is indescribable.
I guess to put another way, if I was born in a lab and observed by a panel of psychologists, they would know intimately "who I was" by observing my behavior and speech, but there will still be some aspects of me that they will never know and cannot know, if that makes sense.
Also I didn't mean to come off as saying that we should throw our hands in the air and quit psychology. I most definitely see the significance of what psychology is attempting to do and think that we as humans can benefit from this knowledge considerably, we just have to make sure this knowledge has a valid basis and isnt mostly speculation.
And also can you recommend any psychologists? Ive read Jung, Lacan and Skinner but that's about it, I'm not very knowledgeable on the subject.
...hence my specifying that I was talking about "psychiatric medicine" and pointing out that "very little psychology is practiced" in the actual delivery of mental health services.Quote:
Originally Posted by Supernova
I get the impression talking to non-wealthy people who have ended up on meds that even if a psychiatrist's or MD's name is on the scrip, it's often case workers with limited medical background who are, for all intents and purposes, making the diagnosis and deciding the course of treatment. A lot of people who end up in the system should be so lucky as to actually lay eyes on a psychiatrist, much less a clinical psychologist.Quote:
Originally Posted by DuB
................ ohQuote:
Originally Posted by Taosaur
I think only psychiatrists [pdoc]should be able to prescribe meds. For nearly a decade or so, General Practitioners prescribed me various things from sleeping pills to anti-depressants. One such Dr. labeled me as having bi-polar. Only one experience with the meds prescribed ended well (I handle Prozac perfectly fine).
My psychiatrist also serves as my psychologist and I wouldn't trade him for anything. I don't understand when people say they have to go to 2 or 3 different mental health doctors. It's hard enough opening up to and trusting ONE person- let alone 2 or more!
I think a lot of progress has yet to be made. A lot of it does seem like speculation- but it's a logical sort of speculation. When I went to see my pdoc he told me to forget about labels. He asked me what my problems were and went from there. It still took a couple of years to find the right meds.
I would say it's definitely a science. But it's an ever evolving one :)
But do you think that's a good idea? My views on the matter are a little more conflicted than I initially let on.Quote:
Originally Posted by Taosaur
Sort of. I'm working on my PhD in experimental psychology and my career goal is to do behavioral research at a university. But strictly speaking, the APA says that you're not "really" a psychologist until you have your PhD. So I guess I'm not "really" there yet.Quote:
Originally Posted by stormcrow
Well frankly, it's not surprising that you hold such a skeptical opinion given the list of psychologists you mentioned having read. If that was my sample of what psychologists do, I'd be pretty damn skeptical too. Those guys are certainly an interesting read, but they're not to be taken too seriously. Skinner's view was at least backed up by some systematic experimental data, which is generally more than can be said for the other two, but his dogmatically narrow philosophy of psychology was pretty roundly overturned around 40 years ago on both empirical and conceptual bases.Quote:
Originally Posted by stormcrow
Reading psychologists is not really the same as reading philosophers. For starters, I wouldn't recommend sifting through the literature by author (i.e., getting one person's view on everything--few psychologists even have strong views on everything, and you probably ought to be suspicious of the ones who do!), but rather by subject area (i.e., getting everybody's view on one specific topic). So what areas are you interested in? This book is one of my personal favorites (this book also is a fascinating counterpoint), but recommendations would really depend on what you're interested in. It's a vast field.
Thanks Dub I really don't know that much about the subject(besides the ones I mentioned) but Im really just interested in psychology in general but of course I am very interested in epistemology. Thanks for the book recommendations I actually saw one of them at half price books awhile ago and was wondering what does heuristics mean lol? Ill most def check them out, thanks.
Psychology as a field is certainly a science, but not all psychologists are scientists. I think that's the distinction that's important to make. There is a lot of bad science out there, and psychology has seen it's fair share.
(on the note of favorite fields of psychology, I myself like personality psychology and cognitive psychology. Of course neuroscience in general is fun.)
It falls quite well into the definition of science... :D.
Though (unfortunately) insofar my knowledge of psychology is limited to a high school course, I would say it is definitely a science. Experiments are run, observations are made, and conclusions are drawn. People often think that psychology is nothing more than an anxious person laying on a couch and talking about how much he wants to sleep with his mother, but there is a lot of research going on (especially recently) in universities and the like that try to explain why people behave the way they do (which, by the way, is essentially the definition of psychology).
Even if it for some reason does not qualify as a science, though, I would certainly not call it a pseudo-science because, for one thing, research is done, examined, and conclusions that have at least some real-world applications are drawn (an example would be finding out what medicine works best for schizophrenics).
At least, that's how it all seems to me. But remember, I have limited knowledge on the subject. I'd say DuB is a much better choice of someone to listen to.
Psychology as a science is an old debate. It is not a science in the currently popular definition of science, as it cannot be proven objectively and it does not explain things on the basic level (physiology), instead it explains theories that happen to work. It is as if there were a 15 story building where we explain the 12th as if it were the foundation, even though there are many levels under it.
Jung was probably the most science-like of the bunch. He got it down to two attitude types (objective/subjective to the outer/inner world) and four functions (obj/sub perceiving, obj/sub judging). van der Hoop goes further explaining how each of the functions works, that is, the steps that each do.
Jung also has a theory on theories. Explaining that we each have a theory based on our own psychological make up (collected works, end of book 6).
So, it really depends on what you mean by science. Psychology does use a scientific approach and does break things down to their smallest parts, it is based on theories and the theories are tested, but all those tests are subjective.
I think it's often treated in as scientific a way as it can, but due to the nature/complexity of what they're studying, it is much more difficult to be objective about the results. Even in a perfect double-blind study, where you THINK you're only controlling the one variable you're testing for some kind of cause-effect, in truth, there are a LOT of variables you aren't aware of, and that today's technology can't pin down accurately. The brain is a giant black box, big enough that all test results should be taken with a grain of salt. A random sampling of people (all from the same city/neighbourhood, usually) is NOT the same as, say, a random sampling of molecules in a chemistry experiment.
In fact, I'd bet most "random samples" for psych experiments (especially those conducted by master's and phd students) are not all that random at all. They often just consist of psych-101 students who need to take part of experiments as part of class. If the test results depend at all on there being a good spread, then the test is useless.
Also, I've recently had a good discussion with a friend of mine, who is a mathematician, and who has spent some time working with psychologists and got to know some psych depts at a few universities quite well. These departments/faculties have a lot of ego (and money) riding on being "self-contained" (i.e. no need to outsource basic courses to other faculties, e.g. mathematics). And unfortunately, this results in most psychologists having a completely inadequate math background, which is important. A lot of them haven't properly studied and learned statistical analysis, and so their data analysis is also something you should take with a grain of salt. Unless a qualified statistician looks through the raw data and how statistical methods were applied to derive the general conclusions, I'd take those results with a grain of salt as well. Of course, it's my expectation that anything important is double-checked by someone who actually knows statistics, but I have a feeling that might be asking a lot.
So, in conclusion, I think it IS a science, at least conceptually, but due to the nature of the problem, and some other factors, the results tend to be more fuzzy and unreliable, in spite of the scientific method being employed (or at least attempted). Also, a lot of test results depend on the test subjects (people) relating their personal interpretations of their experience (that is what it was like when I part-took in some psych tests at school). That means that instead of directly and objectively MEASURING the results, each data point is being filtered through the belief system, prejudices and subjective experience of a different individual.
It is true--perhaps even obvious--that experimental psychologists lack the same facility with statistical data analysis that a statistician would have. But I don't think this in itself is a convincing reason to be skeptical of the data analyses of psychologists in the whole (although probably it is a fair reason in some circumstances, as I discuss below).Quote:
Originally Posted by Replicon
Ernest Rutherford, a chemist, said: "If your experiment needs statistics, you ought to have done a better experiment." He presumably said this with tongue slightly in cheek, but on my reading there is at least a sensible underlying point. (To give an indication of Rutherford's penchant for overblown declaration, this is the same man who infamously claimed that "all science is either physics or stamp collecting.") Clean, simple experiments tend to yield easily interpretable data. The more complicated the data, the more we must rely on statistics to interpret them; and the more we must rely on statistics, the more it seems that we are, in some sense, increasing the inferential distance between the raw data and our conclusions as researchers. This distance makes most of us suspicious, and probably rightly so.
Most psychologists, therefore, strive for experimental designs that are elegant and simple, the required data analyses straightforward. And for the most part (allowing plenty of exceptions) they pretty well achieve it. The typical psychologist rarely needs to step outside the bounds of the classical general linear model in analyzing his or her experimental data. In fact, the most widely reported data analysis procedure in the field is simply a t-test of some kind or other, the simplest cases of GLMs. One need only be familiar with diagnosing and addressing violations of the assumptions of the GLM (some of which the model is quite robust to anyway) to be at least a perfectly competent data analyst, even if not a statistician. This familiarity is of course common knowledge among psychological researchers.
I allowed above that in some cases one might justifiably be skeptical of the conclusions that a psychologist draws from his or her data on these simple educational grounds. In general, this will be increasingly true as the data analysis becomes increasingly convoluted. In these cases, one might question whether the psychological researcher is qualified to handle such complex data competently. But I should point out that it is also in these very cases where it is most likely that at least one verifiably competent, statistically trained individual will be called upon to serve as one of the peer reviewers for such a paper. It is then the job of this individual to see to it that the data analysis reported by the psychologist is adequate and appropriate before the study is to be published in the psychological literature.
In summary: the data handled by most psychologists are well within the modest educational means of the researchers to analyze appropriately, and for the data that are not, it is one of the many functions of peer review to ensure that they are still handled correctly.
As one might reasonably expect, the issue of when verbal reports on mental processes are or are not reliable and/or valid is one with which psychologists have been intimately familiar for some time (a classic discussion of the topic can be found in Nisbett & Wilson, 1977, which incidentally is one of my favorite papers). Without knowing the details of the experiments in which you participated, I can just say that I am pretty confident the researchers in question were well aware of these important issues.Quote:
Originally Posted by Replicon
One thing to think about is that we are often interested in participants' descriptions and explanations for their own thoughts and actions even when we think that the participants don't have a shred of privileged authority on which to give valid, objective reports of such things. For a hypothetical example, we might ask a participant to list some of the reasons why they married the person that they did. In reality, we might just be interested in counting how many reasons the participant spontaneously indicates, without believing that any of the reasons are of any explanatory value. However, we are usually content to let the participant go on believing that we are actually interested in the those reasons and are willing to take them at the participant's word. :teeth:
DuB, thanks for the insightful post.
Maybe I was being a bit pessimistic when I said "it's asking a lot." :) Really, so long as they make the raw data available, any interesting result is likely to get cross-checked.
I totally get that they're aware of these factors, and do their best to come up with elegant experiments, but even in your hypothetical examples, there are unknowns creeping in, that you can't just mitigate easily. If you're interested in the number of reasons they give, you're likely trying to correlate the number of reasons ("rationalization factor?" hehe) to something specific. However, a variance in numbers of reasons given might be more highly affected by another, unanticipated factor. It's probably true that with a big enough sample size, and good enough randomization, a correlation would be meaningful. In other words, if you're isolating two groups of people that are random, except for one attribute you want to measure, there is a size where the attribute split will overwhelm other random factors that could affect the results. But since there's no good way to know, statistically, how big of a sample you need to have any kind of expectations about the result (since, among other reasons, you don't actually know just how strong a correlation to expect), any experiment conducted by a master's student on a relatively small number of people (in the tens or hundreds), I would think, has a reliability issue to be dealt with.Quote:
One thing to think about is that we are often interested in participants' descriptions and explanations for their own thoughts and actions even when we think that the participants don't have a shred of privileged authority on which to give valid, objective reports of such things. For a hypothetical example, we might ask a participant to list some of the reasons why they married the person that they did. In reality, we might just be interested in counting how many reasons the participant spontaneously indicates, without believing that any of the reasons are of any explanatory value. However, we are usually content to let the participant go on believing that we are actually interested in the those reasons and are willing to take them at the participant's word.
What kind of statistical analysis do they go through to ensure, to the best of their abilities, that the results will be reliable? If you think there's a correlation, but the experiment shows just randomness (no correlation), how do you know whether to conclude that the sample set needs to be much, much bigger, or that you can mostly abandon trying to find a correlation, because there isn't one?
I guess sometimes, they can do some calibration... like if people are rating pain on a scale from 1 to 10, they understand that everyone has a different tolerance, and therefore a different scale, so they might have them rate common painful things to get a feel for skew...
After reading this passage a lot of times, I think I see the point you're getting at, but I'm still not completely sure, so correct me if I misinterpret you.Quote:
Originally Posted by Replicon
First a side note on "good enough randomization." Achieving what is essentially completely random assignment to experimental conditions is trivial. In the simplest case of two experimental groups with no constraints on sample size, it can be done perfectly adequately just by tossing a fair coin before each participant arrives. For more than two groups I like to use RANDOM.ORG - True Random Number Service; and more complicated randomization is usually easy to accomplish with a simple R script. So I have to confess that I don't see what your worry is on this issue, if indeed you have one.
Now let me try to paraphrase my understanding of your more central point about small samples being biased and/or unreliable. Assume we have a small sample of 10 people which we randomly assign to either the treatment or control conditions. Assume also that, whatever our experimental manipulation and outcome variable may be, it happens to be the case that the manipulation has a strong effect on the outcome variable for men, but little or no effect on the outcome variable for women. Let's say that, by chance, we end up with a total of 6 men and 4 women in our sample. Finally let's suppose that random assignment leaves us with 4 men/1 woman in the treatment group and 2 men/3 women in the control group. So obviously in this case we should expect to see a stronger effect on the outcome variable for the treatment group vs. the control group simply because of the unbalanced gender split. And if for some reason we failed to measure gender and take that factor into account in our analysis, that would make it look like the treatment is simply more effective than the control overall, when in fact the truth is more subtle and qualified than that. And your point is that random quirks of this kind should be far diminished in much larger samples, due to the law of large numbers. Is this interpretation basically right?
That is all true; but of course, it is perhaps the most basic function of statistical analysis to deal with precisely these issues. To start off in brief, technical terms: there is a term for sample size in the confidence interval formula for a parameter estimate which functions such that an increase in sample size is associated with a monotonic decrease in the confidence interval around that estimate. So parameter estimates (such as an estimated difference between two experimental groups) that are based on very small sample sizes (as in our example above) will have correspondingly wide confidence intervals around them (that is, there will be a lot of explicit "uncertainty" built around the estimate). This means that with small sample sizes, it typically takes a very strong "true" treatment effect and/or very rigorous methodological controls for the effect to clearly emerge empirically due to the built-in uncertainty in the parameter estimates. And in larger sample sizes, where the problematic issues we discussed above tend to be less so, there is correspondingly little uncertainty built around the parameter estimates and so "true" treatment effects can be revealed more easily.
All of this is to say that, while the issues you raised are real and will always be a possibility in an analysis, it is not the case that small samples are particularly vulnerable to this problem.
Well, you can never "know" for sure, but for most simple tests there are fairly straightforward techniques for obtaining estimates of this kind which the researcher can then use to inform their decision about whether to continue. These techniques are known as power analysis. For simple tests, they essentially consist of working algebraically backward through the standard statistical testing procedure by first assuming some nonzero true effect size (for example, you could assume that the estimated effect size you obtained from a small pilot study is equal to the true effect size) and then solving for what the values of certain variables (such as sample size) would need to be in order to have a particular probability of correctly revealing the true effect--this probability is known as the statistical power of the test. Alternatively, you can solve for statistical power given a fixed a priori sample size, or for what the true effect size would have to be given a fixed power level and sample size, etc.Quote:
Originally Posted by Replicon
For more complicated tests, these estimates are often obtained through Monte Carlo simulation, where instead of assuming a nonzero effect size X and deriving power analytically, you generate many, many simulated data sets where the true effect size is programmed to be X on average, analyze all of the data sets using the statistical procedure of interest, and then estimate power simply as the proportion of simulated experiments in which you correctly uncovered a treatment effect.
Mmm, not really sure what you're getting at with this one.Quote:
Originally Posted by Replicon
Took me a while to get back to DV after a trip :)
I appreciate the detailed explanation about the methods. It's interesting reading. So the sample set size can be nicely mitigated in a lot of cases.
The other thing I enjoyed with psych testing was the misdirection. They're often measuring one thing, but telling you the point of the experiment is something completely unrelated, so it doesn't artificially prime the results.
Now, for some of the other points:
I wasn't talking about randomizing within a sample set. That is, of course, trivial. I was really getting at biases within the sample set compared to "the set of everyone in the city/country/world" :)Quote:
First a side note on "good enough randomization." Achieving what is essentially completely random assignment to experimental conditions is trivial. In the simplest case of two experimental groups with no constraints on sample size, it can be done perfectly adequately just by tossing a fair coin before each participant arrives.
Sometimes, tests are conducted with specific bias in mind (e.g. college students), but if you want to make a generalization about people in general, I would think that limiting the sample (based on location, like "a college") could have a significant effect. Even if you advertise all over a big city, your total sample set consists of "people with a propensity to respond to psych experiment ads for the money" which may or may not be a meaningful bias (sometimes, you just don't know).
Unless the census does the random selection (kind of like jury duty), and the experiment is conducted in many cities and areas, it's not really "a set of random people with no bias" :) Again, I bet that for most experiments, that doesn't really matter, but I always get these nagging feelings in the back of my mind when it's not addressed directly.
(this was in response to my comment about calibration)Quote:
Mmm, not really sure what you're getting at with this one.
What I was getting at were the integrity of the data points, since they are often filtered through people's subjective experience.
I'll give you a quick example. I took part in one experiment that was targeting a correlation between "how good you think you are" and "how shaky your confidence about that gets after an experience of being bad at that thing you think you're good at." (they didn't tell you that though)
The way it worked was something like this (slightly hazy memory here btw):
1) You fill out a form that asks questions about how good you think you are at math. It was usually "a scale from 1 to 10, 1 being no good at all, 10 being genius" or "strongly agree, strongly disagree, etc." type multiple choices.
2) Then, you take a math test. They deliberately made it really hard. Like, harder than what I've seen on math competitions. They wanted to create the feeling of "failing a test" under time crunch, and the associated stress. They succeeded - the thing LOOKED like it should have been easy, but it was really fucking hard.
3) You fill out another form similar to (1)
And the idea was, they compare your confidence about your math abilities before and after the failed test. (they only told us that part after - throughout, we thought it was a completely different thing meant to measure something else - don't remember the details though)
Anyway, what I was getting at was, if you ask two people to rate the intensity of a subjective matter on a scale from 1 to 10, two people who experience the same thing will still put a different number. If they're generally insecure, then no matter how good they actually are at math, they'll rate themselves lower.
We've already kind of covered this stuff, when you mentioned that they might ask subjective questions, and they'll just count the number of reasons they give, etc. except even THOSE data points, though better, will have been filtered through the subjective experience.
My global, overall point, is really that while there are some great techniques for tightening up test results, and you've outlined a few of them, testing is an expensive process, and I'm sure not all official tests undergo the right kind of rigor. Of course, the same is true for ALL experiments, but the "black box" you're dealing with in a psych experiment is often more complex and has more unknowns than the black box in other fields.
Science to me is anything that can be proven with results. It is the tool of the philosopher to use Science because everyone has a subjective outlook upon the world and its necessary to have reliable studies which show what tests true over time and on massive scales. The Scientific Method should be utilized in as many far reaching categories of discovery as possible, including esoteric concepts. You cannot distinguish an entire field as Science or Not Science, judge the studies themselves and decide for yourself if its saying something credible. If you only accept what is scientifically proven you're denying yourself your own rich, subjective experience.
We have it quite same as UK
Psychiatrist is a doctor who has specialized in psychology, he treats patients and prescribes medicine if needed. Unlike psychiatrist, psychologist do not do medical diagnoses on patient. (Which is under IDC-10). Psychologist can, however, do psychodiagnostics ( describe the nature of problems, chart the strengths and weaknesses of the customer, and find means to make patients situation better etc.). In Finland psychologist cannot prescribe medicine, because that is the responsibility of psychiatrist as a doctor.
In addition of that there is psychotherapist, which is more specialized and requires base degree from healthcare ( doctor, psychologist etc) along with other conditions fulfilled. All these three ( psychologist, psychiatrist and psychotherapist are under surveillance and control of Valvira which is National Supervisory Authority for Welfare and Health. It supervises practices of medical nature. )
Many people like to see that psychology is not a science. This controversity comes from the fact that psychology studies one of the most hardest subject to come by and that is human mind and human as complete being. Many people seem to have completely twisted views about psychology or limit it to Freudian stereotype they have heard. Psychology is a wide and vast field of different studies. If you think of neurology, which is part of psychology, it is studied and researched completely in scientific method. But psychology has also lot of subjective information gathering, in addition of objective methods, because no mind is alike another. They are working hard to map and find common and universal laws that apply to all humans and that is mighty hard.
Beside of that there are many fields that normal people doesn't always know psychologists work on.
# career choice psychologist
# parenting and child psychologist
# school psychologist
# psychologist in mental health services
# health center psychologist
# work and organization psychologist
In addition psychology can be applied to almost all careers and situations where human behavior is part of it. Road and city planning, IQ and aptitude tests, sports etc.
I can talk of this more if anyone is interested. :)