Banner
    Diederik Stapel: Another World Class Psychology Fraud
    By Hank Campbell | November 1st 2011 12:01 PM | 53 comments | Print | E-mail | Track Comments
    About Hank

    I'm the founder of Science 2.0®.

    A wise man once said Darwin had the greatest idea anyone ever had. Others may prefer Newton or Archimedes...

    View Hank's Profile
    Were you buying it when Diederik Stapel of Tilburg University in the Netherlands claimed meat was behind all the aggression we humans have?  Vegetarians were, sure, along with plenty of other anti-science hippies when a supposed study matches their world view and gives them a jolt of dopamine, but most of us just shook our heads.

    Stapel also did a study claiming scientists discriminated more if their labs were messy.  Really, psychologists can lament they are not taken more seriously but they did little to police their own - until recently.  Marc Hauser was forced to resign, Satoshi Kanazawa finally got the ridicule he deserved, and now an investigation shows Stapel committed data fraud in dozens of publications and even 2/3rds of the theses he supervised were 'tainted'.

    He has apologized and said he "failed as a scientist" but we have to give him a break on that one; he was never a scientist so he could not have succeeded anyway. 

    "People are in shock," Gerben van Kleef, a social psychologist at the University of Amsterdam, told Gretchen Vogel at Science.  Really?  If I take an informal poll of Science 2.0 contributors, no biologist or geologist or physicist is going to be shocked.  How are social psychologists surprised by something everyone else knew?

    Generally, surveys of students are rubbish anyway but Stapel took it to a whole special level; he didn't even bother to do any.  He just made up the results and when he did any surveys at all he still made up the results.  It's almost like he was laughing at people in his own field, or maybe he wanted to be the Tiger Woods of Social Psychology fraud and see how much he could get away with.

    Why didn't they catch it sooner, even after people raised concern levels?  Unlike physics or biology, social psychology is too scientifically fuzzy to say someone is wrong or demand data; if his results weren't replicated, other researchers assumed they were doing something wrong.

    Gosh, I hope his study claiming that we use better manners if a wine glass is on the dinner table isn't on the questionable list.  I changed my whole life based on that one.  Who am I kidding?  All of his studies are probably fraudulent.  The University of Amsterdam is even going back to his Ph.D. work.

    It may seem like this is a real crushing blow for social psychology, like the Catholic church trying to be more liberal and rehabilitating pederasts and getting busted for it, but it is just the opposite - the fact that social psychology is now demanding accountability because junior researchers are blowing the whistle on senior people, no matter how famous they are, is a very good sign.

    Comments

    His premise was perfectly logical, except that I would expect that only the meat of carnivores will turn their eaters aggressive. Mutton, on the other hand, will make people sheepish; chicken will make hockey players afraid of going into corners, and eating tomatoes will make people more Italian...


    I better stop before people take me seriously!
    I almost did take you seriously... :)

    I was having so much fun with it...
    I'll start by agreeing that the introspection caused by this is a good thing. Independent of whether his findings were plausible or replicable, people were shocked because Stapel is well known and had many personal connections with others. About the rest of your post, however....

    The wine glass finding may or may not be true, but the more general finding that objects in the environment can prime behaviour below the actor's conscious awareness is well established and replicated in dozens, maybe even hundreds, of studies going back three decades. And the wine glass finding might be trivial, if true, but the general finding on priming raises a lot of questions about the nature of consciousness and what it means to have executive control of behaviour.

    You've presented no evidence that fraud is more common or likely in social psychology than any other discipline. You've got three anecdotes, where one (Kanazawa) is a sociologist who works at a school of economics and the other (Hauser) who could be called biologist or neuroscentist just as easily. Only Stapel is clearly a social psychologist. But in psychology they teach us that the plural of "anecdote" is not "data".

    You've also presented no evidence that social psychology is any "fuzzier" or less rigorous than biology or physics. (How do you measure "fuzzy" anyway?) Human behaviour is hard to study because is multiply determined. We can't throw people in a tank like a fish, or breed hundreds of generations or them like a fruit flies. People don't sit still waiting to be dug up, like rocks.

    To say that social psychologists don't demand data when studies cannot be replicated is false and ignorance on your part. It definitely happens.

    "Generally, surveys of students are rubbish anyway." Evidence please? In some cases yes, such as when undergrads and "grown ups" are quite different (e.g., political views). Of course, external validity should always be kept in mind. But but when studying basic human processes why should it matter if the participant is a student or someone else? You write as if psychologists have never considered these issues, which they have.

    I agree with everything justinfr said. In addition "survey" makes it sound as if most of (social) psychology studies are actually correlational, this is NOT true. Most studies in social psychology are experiments - they systematically manipulate one or more variables.
    Also, I happen to be a vegetarian AND a social psychologist and, no, I did not at any point blindly believe that vegetarians are less aggressive than meat eaters, and I most certainly did never think it likely that meat-eating CAUSES aggression (which of course is a different thing entirely - even if meat-eating and aggression would correlate a causal link would still have to be established). In fact, your classification of all vegetarians (and social psychologists) as anti "real" science is prejudiced and therefore in itself anti science.
    That said - there ARE problems in social psychology and in psychology in general. For one, the incredibly high publication pressure leads many scientists to value quantity over quality. In addition - and again at least partially because of the immense time pressure - reviewers are not always as thorough as they should be. Also, we currently lack "big" theories that are well-rooted the available empirical evidence to guide future research. And of course Diederik Stapel's behavior is inexcusable. However, you make it sound as if scientific fraud and a lack of quality control only happen in psychology. This is most certainly NOT the case.

    Hank
    However, you make it sound as if scientific fraud and a lack of quality control only happen in psychology. This is most certainly NOT the case.
    No, nor do I intend to imply it.  But for a field that claims to be science, psychology has a much harder time both keeping out woo and flagging it when it happens.   Now, you may be part of the younger generation who expected science and instead found older researchers using psychology as some kind of dodge for popular tripe hypotheses - if so, good for you.  The public will regard psychology as a science over time provided it is not simply generating publicity for goofy people who believe we evolved to like a certain car grill.
    "How are social psychologists surprised by something everyone else knew? ... Why didn't they catch it sooner, even after people raised concern levels? Unlike physics or biology, social psychology is too scientifically fuzzy to say someone is wrong or demand data; if his results weren't replicated, other researchers assumed they were doing something wrong."

    If this were true, there'd be no causa Stapel. I guess this is a case of scientific arrogance as illustrated in this cartoon.

    P.S., Hauser's affected papers were in Cognition, Proceedings of the Royal Society, and Science. No social psychology journals. They were also all studies on primates. Should we conclude that "Generally, surveys of primates are rubbish anyway" ?

    Hank
    I'm no Jean Piaget but I thought we taught reading comprehension in grade school:
     Really, psychologists can lament they are not taken more seriously but they did little to police their own - until recently. Marc Hauser was forced to resign, Satoshi Kanazawa finally got the ridicule he deserved, and now an investigation shows Stapel committed data fraud in dozens of publications and even 2/3rds of the theses he supervised were 'tainted'.
    You have now left two comments noting those guys are not social psychologists despite me never claiming they were.   Hauser was a professor of psychology so claiming he might be a fuzzy neuroscientist is silly.  Kanazawa claims to be an evolutionary psychologist and no one in that field denied his credentials when he was claiming liberals have prettier daughters than conservatives. Outside the London School his other position is in a psychology department.   

    Your feelings may be hurt than not everyone regards 'the science of feelings' as an actual science, but the solution is not to circle the wagons around these clowns, it is to make the field more scientific.  As I noted, the junior researchers did that area a service by expecting there would be science and calling him out for making stuff up.
    Okay, I thought you were using Hauser, Kanazawa, and Stapel as anecdotes to malign social psychology specifically but you meant to use them against psychology as a whole. My bad.

    Let's be clear that I don't defend those three in any way. Stapel is a fraud, Hauser is definitely shady, and Kanazawa is a crank. I also laud the junior researchers who called out Stapel and Hauser. I think we agree 100% here. But regardless, those three are still just anecdotes. I only take exception to the disparaging tone of your post about psychology as a discipline.

    "Make the field more scientific". Mainstream psychologists use the scientific method, proposing hypotheses, running experiments, and developing theory. If you have a case that it's not "actual science" then let's hear it. Do you have a better suggestion for how we should study people?

    Hank
    You're making the definition of science too broad, to try and get it to cover psychology, and then insist being called a science is the only way for social psychology to be legitimate.  I disagree on both counts; studying emotions is just fine, that does not mean it is a science. Psychology, no matter how you seek to spin it, is analysis not prediction.  I agree that profiling has been useful, for example, but it is not a science.  It is a craft that uses understanding of natural laws, like an artist uses color theory. That does not mean an artist is a scientist.

    Engineering is not science either. It's not an insult to tell an engineer that, they know they apply principles of physics but are not scientists.  If you talk to any engineer, they will say that makes them better than scientists.

    A physicist can tell you where any ball dropped off a building will be each second as it moves toward the ground; what can psychology tell us about a person who jumps off that building?  Nothing. Psychologists will just find data and try to match it to a topology. It has value. It is not science.
    what about psychophysics? our experiments are reproducible; our (best) theories are quantitative and predictive; and most (if not all) of us consider ourselves psychologists.

    there's plenty of other experimental psychology out there, not psychophysics per se, that is of good quality and involves models and theories which are productive, falsifiable, and quantitative (e.g. study of memory).

    we also look down on the social psych people, by the way.

    Hank
    I don't know how valid psychophysics is overall but it sure has similar flaws; measuring perception is one thing. Inferring meaning or bias to the length of a line and manufacturing habituation error as a quantifiable thing regarding a line is quite another.

    Brainpunking is not science.  I love watching Apollo Robbins work - he is a mental engineer, much like any person using applied psychology is using scientific principles.  But that does not make them scientists.  It makes them engineers.

    I don't look down on social psych people, I just think a lot of them took an easy road they thought meant science credibility and if neuroscience is in the Dark Ages, psychology is a lot farther back than that.

    I ask you the same question; if psychophysics is science, what is the dominant theory all students accept and have had validated, like evolution is in biology?

    So science comprises which fields then? Just physics? When I look at the top of this website, why is there a "social sciences" menu (that includes psychology)? Would you consider any study of human behaviour scientific?

    Of course psychology can't say anything above a falling object, that's not its scope. But what can physics predict about the behaviour of a person in a specific situation? Nothing. This isn't to say physics isn't important or scientific, just that it has a different scope of inquiry.

    Gerhard Adam
    Of course psychology can't say anything above a falling object, that's not its scope. But what can physics predict about the behaviour of a person in a specific situation?
    Are you claiming that psychology can predict "the behavior of a person in a specific situation", in the same way that physics can predict the behavior of a falling object?  If so, what are the laws of psychology that allow such a prediction to be made?


    Mundus vult decipi
    Hank
    You've basically invalidated psychology as a science yourself - you claim it can do something physics cannot do.   But it can't.

    I am not saying psychology is not interesting or worthwhile or valuable.  In our toolbar we also have mathematics (not science), philosophy (not science), history (not science), education (not science) but - and this is from 5 years of running Science 2.0 - the only people who complain about not being science are the social field.   Why do only they complain?  It's a mystery of psychology.

    If psychology were a science, though, it could tell us why.
    Hank, you claimed that mathematics is not a science (as you do on other places of this very interesting site).
    But for a "neo-platonist" mathematics is part of the reality (mathematics is discovered, not invented). Therefor mathematics is science (for a neo-platonist).

    Hank
    Subjective definitions and relativism are the problem, not the solution.  You could also claim speaking Italian is a science using your method.  When something is everything, it is nothing.
    "You have now left two comments noting those guys are not social psychologists despite me never claiming they were. Hauser was a professor of psychology so claiming he might be a fuzzy neuroscientist is silly. Kanazawa claims to be an evolutionary psychologist and no one in that field denied his credentials when he was claiming liberals have prettier daughters than conservatives. Outside the London School his other position is in a psychology department."

    Call them psychologists as opposed to sociologists or neuroscientists if you want, the point is that at best you provided two anecdotes of fraud in the broad domain of psychology/cognitive science. Is there any reason to think that falsifying data is a bigger problem in psychology than in other disciplines?

    Every scientific discipline has results that are surprising and counter-intuitive, and every discipline generally trusts in the peer-review process, replication, and academic honesty. For every counter-intuitive, "head-shake" inducing finding that Stapel reported, there are other findings by other researchers that are just as counter-intuitive, but are supported with real data and rigorous research. As in physics or biology, when evidence is provided, the fact that a result is counter-intuitive doesn't make it any less real.

    And of course it goes without saying that the junior researchers should be applauded for sticking their necks out.

    Hank
     Is there any reason to think that falsifying data is a bigger problem in psychology than in other disciplines?
    No, but it's kind of a straw man.  I also wouldn't find falsifying data in an astrology journal or a television show about ghost hunting.  What they accept as data and then massage it to a theory is the problem.

    As noted, people didn't call Stapel on it because they assumed they were doing something wrong when they tried to reproduce his work.  There is so little rigor and methodology it was more common to accept that they couldn't reproduce his method than that there was no method at all.

    So if you want to say psychology is a science, like biology, what is the theory behind psychology?  I don't mean something without exceptions or nuance, I am not trying to paint an unrealistic picture,  but what is the theory of psychology that every student learns and accepts and has been validated?
    i would say that the general theory of psychology - at least of experimental psychology - is that there is a functional isomorphism between the brain and the mind, that the mind is accessible to observers through behavioral study, and that the brain is accessible through physiological study.

    in perception science and psychophysics, this shows up in the relative independence of different perceptual qualities; i.e. behavioral studies can reveal that luminance, contrast, and color are encoded relatively independently (changing one does not change how another is seen), different aspects of spatial structure are encoded separately (scale, orientation, direction of motion, etc.); while physiological studies can show that the brain's responses to these image properties are similarly segregated.

    Hank
     A link between the physical and the mental that can be analyzed through studying behavior is not a theory, it is a mission statement.  
    Bonny Bonobo alias Brat
    Very interesting article Hank. I followed one of your links which said the following :-
    Diederik Stapel was suspended from his position at Tilburg University in the Netherlands in September after three junior researchers reported that they suspected scientific misconduct in his work. Soon after being confronted with the accusations, Stapel reportedly told university officials that some of his papers contained falsified data. The university launched an investigation, as did the University of Groningen and the University of Amsterdam, where Stapel had worked previously.
     The Tilburg commission today released an interim report (in Dutch), which includes preliminary results from all three investigations. The investigators found "several dozens of publications" in which fictitious data has been used. Fourteen of the 21 Ph.D. theses Stapel supervised are also tainted, the committee concluded. 
    At least two earlier groups of whistleblowers had raised questions about Stapel's work, the commission found. No one followed up on their concerns, however. Stapel's fabrications weren't particularly sophisticated, the committee says, and on careful inspection many of the data sets have improbable effect sizes and other statistical irregularities. His colleagues, when they failed to replicate the results, tended to blame themselves, the report says. Among Stapel's colleagues, the description of data as too good to be true "was a heartfelt compliment to his skill and creativity," the report says.
     The report recommends that the universities of Groningen and Tilburg look into whether criminal charges are appropriate based on the misuse of research funds and possible harm to Stapel's students resulting from the fraud. The University of Amsterdam, where Stapel did his Ph.D., has apparently not been able to determine whether his thesis was fraudulent or not, in part because some of the original data records were destroyed.
    This has some pretty huge ramifications doesn't it? Maybe there should be random audit checks done throughout the world, if this much 'scientific' fraud has happened in one place we need to know the extent that data is being falsified in other Universities, and not just in the social sciences, surely all such 'scientific' data and interpretations should be eligible for random audit checks and verification?
    My article about researchers identifying a potential blue green algae cause & L-Serine treatment for Lou Gehrig's ALS, MND, Parkinsons & Alzheimers is at http://www.science20.com/forums/medicine
    As an passionate advocate of science and its methodology you should first clean up your own mess before pointing towards others. In particular, you stubbornly argue that three mentioned cases are all psychologists. So what? What does that mean, scientifically speaking? There you go, it does not mean anything, it is same old cherry-picking practice people use when they want to claim law or regularity based on few individual cases. Also, derogatory ton on subjects of psychological studies illustrates how unscientific in essence your reasoning is; because, for science, there is no trivial problem - science is interested in the nerve system of a Brasilian Frog, gas structure of a distant planet, market bubbles in middle century textile trade, or anything else on this Earth and beyond. We can not be equally interested in all of its subjects, but we can at least respect these interests and scientific endeavor per se because there is no good or bad research subject, there is only good or bad research.

    But, most interestingly for me, your post and consequent replies illustrate on its own exactly the point that you are vigorously deny: how important and interesting study of psychology really is. Because from all your discussion about fraud in (social) psychology and its scientific credentials one could not learn much - it is rather unsubstantiated, opinionated, anecdotal account of some events. But it is interesting to see that an obviously intelligent, educated and well versed person would follow his emotions, i.e. his apparent disdain for certain scientific disciplines, and tried to build the case of it being nothing but a total waste of time. And you see, because of this property of human mind to disregard its reason and succumb to the emotions and various motives study of 'the science of feelings' is so immensely fascinating and important at the same time.

    And finally, in regards to the lack of predictability in psychology you should just read your posts: I am quite sure that you will see substantial regularity in the substance as well as in the level of arguments, notions, motives, attitudes, values, emotions, word choices, grammatical structures, etc. In other words, it is pretty predictable stuff, notwithstanding pretty wrong indeed;)

    Hank
    What does that mean, scientifically speaking?
    Three years ago, this would never have happened, which means psychology is emerging from a 25 year morass of "I will believe your woo if you believe mine" and is trying to take back the field from cranks. 

    As I said, I see the recent scandals as a positive thing.

    The rest of your comment is making an emotional argument about why emotions should be studied; I never said they shouldn't.  I also think art and music should be studied, but that won't make those science.

    Gerhard Adam
    I was looking for something that resembled a "law" of psychophysics and came across this entry:

    Neural Coding and the Basic Law of Psychophysics
    http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1994651/

    In reviewing this, it sounds like gibberish.  I'm trying to figure out what is being said, and can't help but walk away with the impression that the net result of all these studies, is to conclude that the sensations are linearly responsive to stimuli.  In other words, the harder you rub against something rough, the rougher it feels.

    "On the basis of direct observation of power law relations between impulse rates in single neurons and physical intensity, Mountcastle and his colleagues proposed that subjective intensity is linearly related to the neural signal on which it depends..."

    "We analyze the steps between a stimulus and the subjective response to it, and we argue that the appropriate test of a neural coding hypothesis is consistency. If there is no consistent relationship between a putative neural coding measure and subjects’ reports, then the hypothesis can be rejected; if there is a consistent relationship, it emerges directly in the consistency test whatever it might be (linearity, a logarithmic law, a power law, . . .). We review a series of studies that have used the consistency test to examine the neural coding mechanisms underlying tactile texture perception. The studies eliminate all but a single hypothesis, but, more relevant for the main point of this review, the relationship that emerges is linearity."
    How is this anything but obvious (albeit couched in exceedingly obscure language)?
    Mundus vult decipi
    rholley
    Some useful background reading:

    Report: Dutch 'Lord of the Data' Forged Dozens of Studies

    Robert H. Olley / Quondam Physics Department / University of Reading / England
    Hank
    It's silly, right? What is easier than surveying students? Stapel couldn't even be bothered to do that.
    MikeCrow
    Unless you don't like their answers!

    Then you have to alter the data or throw it away, you can get caught doing that, so just make it up.
    Never is a long time.
    Thank you for your article. It is good to see fraud being recognized and caught.

    I am curious about some of your claims in the comments section. As part of a requirement for science, you seem to suggest that there must be broadly agreed upon theories (for example, evolution with biology). Within social psychology, there are a number of broadly agreed upon theories(as an example, cognitive dissonance theory). If your point is that there should be a single overarching theory that all of a particular discipline stems from, no branch of science meets this requirement. There are efforts in physics to reconcile the different forces and findings to form a grand unified theory and eventually a theory of everything - these do not yet exist. The theory of evolution in biology does not cover biogenesis. A little clarification about what your point is and how it is a requirement of science would be much appreciated!

    From what I've learned, the key component in what makes something a science is if it uses the scientific method. By this definition, social psychology is a science - there are falsifiable hypotheses which are proposed and rejected based on empirical data. People are randomly assigned to different conditions, with changes in their responses (sometimes self-reported, sometimes behavioural, sometimes biological or neurological) measured. There is a great deal more than just handing out surveys.

    You've also mentioned a focus on analysis rather than prediction as being an issue. Social psychological theories are used to make predictions. There are many factors in what determines people's behaviour, so these predictions work better for predicting how people under different circumstances or with different characteristics will behave on average, rather than with the high level of specificity that you can get with something like chemistry or physics, where there are relatively few factors. Could you explain more about what you mean regarding analysis and not prediction, and how this relates to psychology and science?

    Hank
    Within social psychology, there are a number of broadly agreed upon theories(as an example, cognitive dissonance theory).
    Holding conflicting ideas simultaneously is uncomfortable? That puts Leon Festinger up there as the psychology equivalent of Newton? Okay, but to the rest of the world that is not a theory.

    Psychology is easier to just make stuff up because it lacks methodology.  You personally may feel like it is rigorous
    From what I've learned, the key component in what makes something a science is if it uses the scientific method.
    but your definition of being a science as you list it is so arbitrary that making hot chocolate is a science; I predict if I add sugar to chocolate, it will get sweeter.  Experiment. Verify.  There, it is now just as valid as anything Einstein was able to accomplish?

    What you lack in your example is actual predictability.
    By this definition, social psychology is a science - there are falsifiable hypotheses which are proposed and rejected based on empirical data. People are randomly assigned to different conditions, with changes in their responses (sometimes self-reported, sometimes behavioural, sometimes biological or neurological) measured. 
    It tells us nothing at all about how people will act.  It only tells us how a sample of people did act. That's no more science than economics is.
    Thank you for your response.

    Cognitive dissonance theory goes beyond the idea that "holding conflicting ideas is uncomfortable" - it is about how that tension is resolved, resulting in changes in people's attitudes, their actions... your comparison is similar to saying "Newton told us that stuff falls when we drop it? That's really not much of a big deal".

    Psychology is easier for people to make stuff up because there are more factors that affect people's behaviour. Scientists may "make up" hypotheses, but then these hypotheses are tested. What lack of methodology are you referring to exactly?

    Regarding your comment about hot chocolate, that is a great example of empiricism. If we consistently replicated this finding, we could indeed conclude that adding sugar to hot chocolate will make it sweeter. What your example is lacking that both psychology and physics have is theory. We try to develop our understanding of what is going on, and use theories to then make further predictions. This is what happens in science, including psychology - it sounds like you may not be aware of this process occurring in psychology, but it is there.

    Finally, psychological theories and findings are used to make predictions, not just retroactively explain what has happened. If you're looking for cases where research and predictions from social psychology have been used in practical ways, you might want to look at the work of Elizabeth Loftus , regarding eye-witness testimony, or Carol Dweck's work on Entity and Incremental theories of intelligence, and how that work has been used to increase students' math grades, or the work that's been done on romantic relationships that can predict the dissolution of those relationships when people are still in their "honeymoon phase". There are huge bodies of work that tackle prejudice and racism, and how to reduce them, determining the factors that enhance and inhibit motivation and willpower, the moral decisions that people make... to name a few. To say that social psychology tells us nothing at all about how people will act doesn't seem to match up with what the field has done.

    If psychology wasn't a science able to predict people's behavior, sales people wouldn't use it.

    What I'm saying is that
    (a) It is true that data are easier made up in noisy methodologies, related to systems that you can't isolate, because the noisier the data (for instance, any rating scales produce is vastly noisier than reaction times) the more you can be unknowingly fooled by your own skewed vision. This is related to process purity and the absence of it, for instance if you use a rating scale, there is much noise and less explanatory power (though there might be predictive power to be obtained from the data), because for one you have no clue as to whether the responses are the result of unconscious processes or conscious ones, and the more you dabble in the latter, the more variance you'll have. Now, science also has expectations. I recall seeing this booklet by Millikan for the oil drop experiment, where at one point he got data he didn't like and scrapped them. The difference is that he didn't like them because hard theory suggested it should be otherwise. And subsequent replications also showed he was right. In psychology, we also expect things based on theory, but there are so many potentially unknown variables that we can try and control, but never fully. In and of itself it's not a problem, but it may make it more prone to seeing what you want to see... A good article on that (w respect to medical science) is here http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and...

    (b) However, not all research in social sciences is simply connecting stuff with stuff. It's a slow moving glacier, but there are things that are discovered and that work. PR uses this stuff. It is known how to make people buy bogus products they don't need. It is known how to create this need. Admittedly, we are always talking averages here and nomotetic research, and people, in contrast to particles, have something like a personal history, a continuous logging of information over numerous years that influences their behavior. If one is sceptic, one can say that the only way you would be able to really predict the behavior of one person is if you could trace back every single influence from conception to the present moment. So, Second Foundation is some way off. All we can say is that that Person P has, given a context that resembles that of the experiment, has a chance C of producing behavior B. On a group level, this becomes more predictive of course. Any process in which the individual characteristics play less influence, will be easier to predict. But even on an individual level, there are a number of things. The effect of liking people more who who imitate you without you realizing. If you know this, you can do this. There are techniques. They do not work 100%, but they work. Same with PR and publicity. The fact that not 100% of the variance is explained, is because of influences that your model does not incorporate. Does that make it no science? I don't know. Physics has a model of how particles X should behave. Experiment. One particle x behaves otherwise. Theory not complete. But you wouldn't just say the theory is total bull if it predicts 99.9% of the particles? Hell, there's physics that only works on earth and they worked their way up from there.

    What I'm saying is that it is not so simple to dismiss psychology as a science simply because it is psychology. Probably psychology is too broad to be judged as one. Probably Second Foundation is far away. But if prediction is your criterion, there are quite a number of things that can be not just explained but predicted. And even if you cannot isolate the system, the predictions will become more accurate the more we know. Currently the field is simultaneously pushed forward and held back by functional neuroimaging, which, most of the time, is concerned with brain areas correlating to X, and as a consequence has so many functions mapped to, say, the prefrontal cortex, with very little prediction that can be made based on prefrontal cortex activity alone...

    But we'll get there. I'm confident that, give it the 2000 years that physics had to crawl out of the idea that the world was hexagonal, psychology will be able to shove your child into a scanner and say, this one should have preferably an environment with these characteristics, and then they will turn out so and so. Ok, maybe it takes 4000 years. So if prediction is all it takes to be science, that's silly because that's just a matter of time.

    Science as you describe it, is that part of investigations which is concerned with explaining variance for the full 100%. The more we know about the world, the more this percentage increases. Physics wasn't science 1500 years ago. And I don't think the full field of psychology is, by that account, a science now. But it does show that dividing it in science and non-science is not very productive at all, just as desperately claiming to be a science when you are not is futile. Yet no psychologist would claim that they explain 100% of variance, they just respond to a heap of elitist crap that comes with the science/non-science division.

    Hank
    If psychology wasn't a science able to predict people's behavior, sales people wouldn't use it.
    Again, this is a fundamental understanding of what science is.  Applying knowledge is not science, or your plumber is a scientist. The rest of your comment proceeds farther from this common misconception.

    A large part of the problem is that, unlike plumbers and engineers, psychologists refuse to understand what science is and then rationalizing why their gray, movable definitions are okay ("But it does show that dividing it in science and non-science is not very productive at all,") - that means there is no point in having a definition at all.  The guy who cooks at the breakfast diner I frequent can now claim to be a psychologist using the same rationales people commenting are using.
    Ok, fair enough. I'm new to this -very interesting and funny- blog, so I'm probably asking what you already gave numerous times, for which my apologies. But can you give me then, in one clear sentence, a definition of science, which includes the obvious criteria in which it qualitatively and definitively differs from social "sciences". So that I can stop the fuzziness and move on.

    Mind, I'm not trying to prove that psychology is a science (though I think it isn't *yet*). The way I see it, one reason psychology isn't a science is that it works top-down, starting from observable behavior and trying to induce from that putative mechanisms that might cause said behavior, rather than starting from, say, actual known quantifiable brain mechanisms and working up to behavior. I was just picking on the fact that you mention several times "predictions". Which is not something psychology fails to make, nor something that sets it apart from science.

    Hank, your counter-point for everyone who is attempting to defend psychology or social psychology is that their definition of "science" is too broad. Of course, just what science does and does not entail has always been contentious, both among scientists themselves and within the domain of the philosophy of science. Pinning down a clear definition is more difficult than you seem to claim. Would you care to provide us with your definition of what science is and does?

    You also seem to be comparing psychology to engineering, as an "application of science". But you neglect that the majority of psychological research is theoretical in nature and is not applied in its focus. Certainly it has real-world implications, but so does theoretical physics. Otherwise, the theories couldn't be tested. Criminal profiling, marketing, and counselling are examples of applied psychology. Basic research into human cognition, motivation, emotion, and behavior are not.

    Other commenters have already pointed out that psychology (including social psych.) includes controlled experiments, testing of theories, generation of hypotheses, prediction of behaviour based on theory, statistical models, falsification, and peer review. What other aspect to science is there that is not included there? The only other point you've brought up is that psychology may not have a "dominant theory" -- but this is just an outdated Kuhnian vision of science that neglects the mulitiplicity of ways in which science is actually carried out (yes, physics included).

    So just to repeat myself, I think the discussion could be moved forward if you were to provide a more coherent definition of just what you think science is.

    Hank
    You also seem to be comparing psychology to engineering, as an "application of science". But you neglect that the majority of psychological research is theoretical in nature and is not applied in its focus. 
    The core issue it matches none of the criteria for a science - it is not based in theory, there is no fundamental theory in psychology - and, like engineering, is primarily applied, like with criminal profiling.   

    Obviously over time it will become more scientific but that doesn't mean all aspects of it will; physics is a science but the aether is no longer a valid hypothesis nor is the steady state model of the universe.  And so psychology may one day be possible as a science.  I've yet to see a psychologist who wants to be accountable the way science is - the rationalization is that would be too limiting - but psychologists want to claim science legitimacy regardless.  No idea why, but it happens.  Some biologists resent not being a hard science like chemistry too.  It happens.

    So existential-phenomenological psychology is a science just like physics or chemistry?  Outside psychology, you will have an impossible time explaining that.
    Ok, this broadly answers my question, too. I would still like to see a clean definition though.

    But I think we can agree that psychology might move there, but at the actual point it isn't there yet. It will, though. Give it some time. After all, when physics was but 100 years old, it wasn't the fancy stuff yet that gave us electrical shavers, just a bunch of thinkers with a load of conjecture.

    What I do think is that for psychology, trying to "be scientific" before it actually is might be detrimental to the starting point for good science, i.e. observation. As a consequence of over-application of "controlled scientific methods", instead of true valid data on human behavior, we have a subset of artificial behaviors that occur predominantly in college freshmen :-)

    But I'm positive that Second Foundation is around the corner in a couple of millennia.

    Definition of Hank Campbell: Ignorant.

    Hank
    Thanks for your contribution to the discourse.  Only 9 out of 10 dentists recommend sugarless gum, so one clearly thinks cavities are good, and you seem to be okay with Stapel committing blatant fraud and making social psychology look bad.  I do not; ironically, I care more about the reputation of social psychology than you do.

    I'll continue to highlight fraud so those people can continue to get booted out and you can continue to support fraud by attacking anyone who seeks to whistleblow it.
    "Again, this is a fundamental understanding of what science is. Applying knowledge is not science, or your plumber is a scientist."
    I can't understand your way of thinking. Could you explain exactly what the science is according to you? I will try to explain you my way of thinking. The science is based on "applying knowledge". First of all our senses gather information about the reality around us and we apply this knowledge. Without applying knowledge there is no scientific experiment. Don't you think that science attempts to describe and understand the nature of the universe? And no matter if you are a plumber, hairdresser, Julia Child or Stapel - you can consider youself a scientist if you perform observation, identification, description, experimental investigation and theoretical explanation of phenomena? Your results could be spectacular or not but the process is indeed the science. That is my point of view now. But maybe I am wrong.

    "The core issue it matches none of the criteria for a science - it is not based in theory, there is no fundamental theory in psychology - and, like engineering, is primarily applied, like with criminal profiling. "

    It is based in theory derived from observations, experimental, statistical tests, etc.

    No, it doesn't have a fundamental theory, but I think there have been good examples of how no science has a fundamental theory that explains and reconciles everything.

    Previous posters have already addressed the issue of it being applied. It is not primarily applied.

    I don't think anyone has a problem with anyone criticizing Stapel or calling out fraud. The issue is derogating an entire field because of it.

    We still have not seen a clear definition of science from you, and your reaction to examples of basic psychological theories have been to trivialize them, just as easily as one could trivialize a theory in physics (see the comment above: "Newton told us that stuff falls when we drop it?").

    Hank
    No, it doesn't have a fundamental theory, but I think there have been good examples of how no science has a fundamental theory that explains and reconciles everything.
    Yet psychology has no fundamental theory of anything that anyone agrees on so that is a big difference than something like biology, where everyone agrees on evolution and argues about mechanisms.
    We still have not seen a clear definition of science from you, and your reaction to examples of basic psychological theories have been to trivialize them
    This isn't grade school.  The issue truly is that no one in psychology (in these comments anyway) seems to know what science is, so they make up their own definition that happens to match what they have chosen to study.  Economists can do the same thing, as can politicians and artists.  Still not science

    I wrote an article about a fraud and there are a bunch of people whining "We're science, were science!" yet no can be bothered to even use a search engine to find and accept a definition unless it matches their confirmation bias.  So, you are right, I haven't bothered to respond to requests for 8 subjective definitions of science which none of you agree with anyway, or you wouldn't be asking.
    Hi Hank,

    Okay, according to a definition I found using a popular web search engine, science is a systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions about the universe.

    So everything that doesn't test explanations and predictions about the universe cannot be called science according to this definition. Hence, doing research on human behavior is not science, only research. Right?

    But on the other hand, since you claim that because most of the psychological research is done with students - who are in fact the most eligible test subject for psychological research - the method is flawed since students are a very specific subgroup of humanity and thus results found with students should not necessarily apply for the general population (a point on which I totally agree with you!), I would like to point out, that most research in the "true" scientific fields, that is chemistry, physics, geology and biology, is done on the planet Earth and not on any other planet in the universe and accordingly one cannot necessarily conclude that the results found on the planet Earth apply to the universe in general.

    Thus, research in these "true" scientific fields done only on Earth should just be called research and not science, since the findings are limited to the planet Earth and do not test explanations and predictions about the universe as the definition of the term science demands. Obviously most scientists in these fields just assume that the results found on the planet Earth apply to the universe in general without ever having double-checked that assumption.

    So if you want to be strict about the usage of the term "science" in reference to psychology, I think you should also admit that most of the research done in physics, chemistry, biology and geology does not fulfill requirements of the definition of science and thus can only be called research and not science. Right?

    Cheers,
    George

    Hank
    Hi George,

    Essentially science seeks to understand the world according to natural laws.  The fundamental question that psychology or sociology or anthropology or economics must ask is, do they help anyone without expertise understand the world better?  Right now, they don't.   In slapping Stapel, I was not picking on social psychology as much as I was picking on psychologists - and I said at the end I am heartened that the newest generation of psychologists have had enough of two decades of woo in this field and that woo culture and tolerance for silliness is a product of my generation.  If you are part of that younger generation of psychologists, demand accountability from everyone, regardless of how popular they are.  As the field matures - and the mind is the last great frontier, for sure, so it is not going to happen soon that it becomes a hard science - it will get better.  Before the 1850s, all medical doctors were regarded as quacks but Harvard changed that by demanding accountability from anyone who wanted to enter their medical school.

    None of Stapel's peers raised a flag about him, younger people did - people he knew, who expected to enter the field and converge on a science understanding of the mind and found instead there are a lot of charlatans.  That's a good thing.

    People are instead getting hung up on finding ways to make what they do science - a shocking compulsion (or physics envy) given the nature of what psychologists claim to be.

    Hank,

    It's funny that you are arguing for the upholding of science, when you yourself make empirical claims that are founded on the thinnest air. I have two objections to what you have said:

    1. Experimental psychologists are, in fact, scientists. As long as you are rigorously employing the scientific method, it is considered science--no matter what you are studying. If a psychologist designed an experiment with an independent and dependent variable, proper control groups, proper statistical analysis, and methods that are replicable, then it remains to be seen that he or she is a scientist. It does not matter whether the data have been replicated-- it is still science nevertheless.

    2. Having an underlying theory for everything is unnecessary, and it sure does not downplay the reputation of any scientific field. Newton himself admitted that he could not explain gravity, and that it was not necessary to do so. Those who need theories for everything (you seem to be one of those people) might as well throw away their faith in science and take up Christianity. I would never suggest doing this to any rational human being, but 1) I question your rationality and 2) it seems like this move would please your desire to explain everything.

    Hank
     It does not matter whether the data have been replicated-- it is still science nevertheless.
    and
    Having an underlying theory for everything is unnecessary
    on part one, I agree null results have value, assuming their is a chance for a real result. The problem is that everyone on the planet is a scientist when the core criterion is " an experiment with an independent and dependent variable".  Regarding control groups, well, stop using college student surveys and pretending you can make real controls and studies will get taken seriously.

    On part two, I agree an underlying theory of everything is not necessary - everything is a big idea, which is why your statement is a straw man.  So is there any valid theory in psychology that is accepted and taught universally?  I don't mean a ToE, I mean a ToA.

    You seem to be resentful that psychology isn't science even if some experiments are scientific; as I noted in earlier comments psychologists are the only field that have simultaneously allowed woo to be published in journals or online (practically all of Psychology Today, for example) yet then insist it is just as valid as chemistry.  Engineers are not scientists simply because they use science methods, nor are psychologists. It isn't a bad thing not to be a science, my whole career was in engineering and not science.  It isn't like I said you were a postmodernist or something useless.
    The fact that you invoke surveys as the main tool for psychology shows that you have no clue as to what you are arguing against. The most interesting results in social psychology are found in social cognition, where surveys are hardly ever used. Rather, most social psychologists in this sub-field rely on experiments. I will refer you to hundreds of papers if you are not convinced.

    Also, to state that everyone makes use of the experimental method of utilizing depend and independent variables is an overestimation of human reasoning. Since when do lay people partake in such methods of discovery?

    Your second point is incoherent, so please restate it in different terms.

    Engineers do not use science methods. They do not seek to discover anything about nature by way of the scientific method. So you are wrong in that aspect. However, engineers rely on principles discovered by scientists. In this respect, how are psychologists like engineers?

    Larry Arnold
    Some years ago I applied to do an undergraduate Psychology degree at a certain University.

    I was turned down, and on contacting the admissions tutor to ascertain why, was told that Psychology is a scientific discipline and  I did not have a single science A level to my name, let alone a recent one which would be the required basis for understanding psychology as taught by said University.

    "Ah but I have a recent psychology A level", I responded

    "But that is not a science " was the reply.

    This is a true story.

    Hank
    I love it!
    Hank,

    As an IO Psychology Masters student and an avid proponent of the scientific method, I am at least a little offended by your implication that psychologists are some kind of second class group of academics unworthy and undeserving of the title "scientist" and that we would be better grouped with those losers in the humanities (tongue in cheek; I am aware of the irony of that last comment).

    I agree, unsurprisingly, with Eric Garr in that whether or not a discipline can be considered a science is determined by the extent to which we follow the scientific method.

    The use of surveys does not make our discipline non-scientific. Making causal conclusions on the basis of cross-sectional data? Sure, that's pseudoscience. But the use of cross-sectional data in it of itself does not constitute non-science. E.g. a strong relationship between perceptions of organizational permissiveness and absenteeism is an empirical finding that can be replicated, disconfirmed etc. and of practical value for organizations.

    Undergraduates as a predominant source of data? We are well aware that this is a limitation and thus we do additional field studies so as to obtain converging evidence that lab experiments (which may be excellent in internal validity) also have generalizability to the real world as well.

    The fact that "woo" (a term I despise, despite being often used by James Randi whom I most respect) is occasionally published in Social Psychology does not invalidate the entire field. Is it embarrassing? Absolutely. But it seems to me you're using what is otherwise an isolated event to justify your own elitist (mis)perception of an entire field, not caring whatsoever for methodological differences between the various subfields of psyc (e.g. cognitive vs. social research is drastically different).

    I am eager to hear your response.

    - Joe

    Hank
    You're exactly the kind of person who tripped up Stapel; you went into this field expecting science and many like you around him did also and were not impressed by his 'credentials' and called him out when they sensed his science was missing.  I don't dismiss the entire field, I ask that it circle the wagons less.  For being psychologists, an alarming number in the field are easily duped into a 'you are with all of us or against all of us' that would make George Bush ask them to be more nuanced in their thinking.
     thus we do additional field studies so as to obtain converging evidence that lab experiments (which may be excellent in internal validity) also have generalizability to the real world as well.
    Maybe in your lab.  It is easy to find hundreds of studies published in the last few years that are nothing but surveys and conclusions, just data mapped to the topology the researcher wanted.  If this were an isolated event, there would be no reason to write about it. I've gone after physics frauds too but the key difference is no physicist rushed to his defense the way psychologists have Stapel, even though everyone knows he is wrong.
    an isolated event to justify your own elitist (mis)perception of an entire field, not caring whatsoever for methodological differences between the various subfields of psyc (e.g. cognitive vs. social research is drastically different).
    So which of these sub-fields, in your estimation (since you lack my "elitist misperception") are not science?
    Hank
    Two more Diederik Stapel papers have been retracted.