Banner
    Null Hypothesis And The Logic Of Skepticism
    By Massimo Pigliucci | September 4th 2009 09:58 AM | 10 comments | Print | E-mail | Track Comments
    About Massimo

    Massimo Pigliucci is Professor of Philosophy at the City University of New York.

    His research focuses on the structure of evolutionary

    ...

    View Massimo's Profile
    Being a skeptic is a rather lonely art. People often confuse you for a cynic, and I’m not using either term in the classical philosophical sense, of course.

    In ancient Greece, the cynics were people who wished to live in harmony with nature, rejecting material goods (the root of the word means “dog-like,” and there are various interpretations as to its origin). The Western equivalent of Buddhist monks, if you will.

    The skeptics, on the other hand, were philosophers who claimed that since nothing can be known for certain the only rational thing to do is to suspend judgment on everything. That’s not what I’m talking about.

    A skeptic in the modern sense of the term, let’s say from Hume forward, is someone who thinks that belief in X ought to be proportional to the amount of evidence supporting X. Or, in Carl Sagan’s famous popularization of the same principle, extraordinary claims require extraordinary evidence. In that sense, then, what I will call positive skeptics do not automatically reject new claims, they weigh them according to the evidence.


    And of course we aren’t cynics in the modern sense of the term either, i.e. we don’t follow Groucho Marx when he famously said “Whatever it is, I’m against it!” (Of course, he was joking, though that seems to be the motto of the current Republican party.)

    Now, you would think that few people would object to the pretty straightforward idea (which can actually be formalized using a Bayesian statistical framework) that one’s beliefs should be adjusted to the available evidence. You would also think it hard to disapprove of the corollary that — since the evidence keeps changing and our assessment of it is perennially imperfect — than one ought not to subscribe to absolute beliefs of any sort (except in logic and mathematics: 2+2=4 regardless of any “evidence”). Boy, would you be wrong!

    For one thing, the positive skeptic finds herself more often (in fact muchmore often) than not in a position to (provisionally) reject a given claim rather than (provisionally) accepting it. Why, you might ask? Shouldn’t the expected likelihood of the truth of a claim a priori be something like 50-50, in which case the skeptic should accept and reject beliefs in about equal manner? No, as it happens, things aren’t quite that nicely symmetrical.

    One way to understand this is to think about a simple concept that everyone learns in statistics 101 (everyone who takes statistics 101, that is): the difference between type I and type II error. A type I error is the one you make if you reject a null hypothesis when it is in fact true. In medicine this is called a false positive: for instance, you are tested for HIV and your doctor, based on the results of the test, rejects the default (null) hypothesis that you are healthy; if you are in fact healthy, the good doctor has committed a type I error. It happens (and you will spend many sleepless nights as a consequence).

    A type II error is the converse: it takes place when one accepts a null hypothesis which is in fact not true. In our example above, the doctor concludes that you are healthy, but in reality you do have the disease. You can imagine the dire consequences of making a type II error, also known as a false negative, in that sort of situation. (The smart asses among us usually add that there is also a type III error: not remembering which one is type I and which type II...)

    What’s that got to do with skepticism? Whenever confronted with a new claim, it’s reasonable to think that the null hypothesis is that the claim is not true. That is, the default position is one of skepticism. Now the tricky part is that type I and type II errors are inversely proportional: if you lower your threshold for one, you automatically increase your threshold for the other (there is only one way out of this trade-off, and that’s to do the hard work of collecting more data).


    So if you decide to be conservative (statistically, not politically), you will raise the bar for evidence, thereby lowering the chances of rejecting the null hypothesis and accepting the new belief when it is not in fact true. Unfortunately, you are also simultaneously increasing your chances of accepting the null and rejecting the new belief when in fact the latter is true.

    Human beings are thus bound to navigate the treacherous waters between Scylla and Charybdis, between being too skeptical and too gullible. And yet, the two monsters are not of equal strength: if we accept the assumption that there is only one reality out there, then the number of false hypotheses must be inordinately higher than the number of correct ones. In other words, there must be many more ways of being wrong than right. Take the discovery that DNA is a double helix (the true answer, as far as we know). It could have been a single helix (like RNA), or a triple one (as Linus Pauling suggested before Watson and Crick got it right). Or it could have been a much more complicated molecule, with 20 helices, or 50. Or it may have not been a helicoidal structure at all. And so on.

    So when trying to steer the course between skepticism and gullibility, it makes sense to stay much closer to the Scylla of skepticism than to bring our ship of beliefs within reach of the much larger and more menacing Charybdis of gullibility. The net result of this prudent policy, however, is that even positive skeptics are bound to reject a lot of beliefs, with the side effect that their popularity plunges. As I said, it’s a lonely art, but you can take comfort in the psychological satisfaction of being right much more often than not. This will not get you many girls and drinking buddies, though.

    (Caveat: I have actually argued in a technical paper that we should abandon the whole idea of null hypotheses and embrace more sophisticated approaches to the comparisons of competing explanations. But that’s another story, and it doesn’t change the basic reasoning of this post.)

    Comments

    One thing you don't address explicitly here are the relative risks inherent in type 1 and type 2 errors and how that should change our approach to belief acquisition (or at least our actions). So sometimes, it is important not to miss a "true positive" and sometimes it is important not to accept a "false positive." The technical details don't change, but our approach to the evidence should (although here the difference between *acting as if* and "believing that” might be important. How should our actions be related to our beliefs? To read more: http://personalmoneystore.com/Cash-Advance/
    kerrjac
    Great post.

    I've found that a general attitude of skepticism is somewhat common in science. However, it's not so much a question of skepticism vs no skepticism, but of the targets to which skepticism is applied. All too often, skepticism is applied to more superficial factors.

    Extending your medical test example, consider the situation (not uncommon) where a patient complains of physical symptoms, tests come back negative, and the doctor assumes that the patient is healthy. The doctor ends up questioning the reliability of the patient's complaints, pitting them against the soundness of the medical tests and standards. It's not an issue of type I error; from the perspective of science, it's a deeper issue, concerning underlying assumptions (eg, our ability to use tests to validate every physical symptom).

    Debating the give-&-take between type I&type II error is completely valid. But one should avoid getting stuck in an either-or-mentality. Both errors are necessary to avoid, but (even if the hypothesis is statistically supported) neither excludes the possibility that the research question maybe junk.

    It's common to see a sort of technical skepticism which takes aim at things like type I&type II error. Of greater import is a deeper brand of skepticism which examines the research question, the assumptions being made, its measure relative to logic and common sense and related constructs, and the quality of the data. The latter approach, I completely agree with you, is all too rare.
    Hank
    All too often, skepticism is applied to more superficial factors.
    Yes, in the 'official' American skeptic movement, it's only cool to be skeptical about Christianity and Bigfoot - and nothing at all in science, including some things that must be regarded as fashionable.
    I found your childish attempt at political commentary ( “Whatever it is, I’m against it!” (Of course, he was joking, though that seems to be the motto of the current Republican party.)) a real shame. After reading the remainder of your article, however, I realized it was a the type of perfunctory and fashionable commentary so often spewed by simple-minded intellectual wantabes. Thus, I could only laugh and make a note to avoid anything else you should write for this website.

    What's in a name? Call it "positivism" instead of skepticism. I'm a positivist, wouldn't you like to be a positivist too?

    Skepticism is just naive belief into opposite. Symptoms of pathological skepticism
    http://www.eskimo.com/~billb/pathsk2.txt
    For example, many skeptics are believing, Aether theory was refuted by Michelson-Morley experiment. What they didn't realize was, particle environment spreads energy in two dual ways (in longitudinal and transversal waves) and single experiment cannot refute them both. The same naive skepticism follows acceptation of cold fusion, antigravity or room temperature superconductivity findings. We all are losing incredible amount of money by this ludicrously naive stance.

    Why ruin a good piece about logic with a gratuitous poke at Republicans? Since it is evident that many groups could be inlcluded in the point made by Groucho, it seems more correct to note that the attitude cited here is endemic to human nature and not merely Republicans. The sneering disregard that many liberals have for conservatives does not display logic or skepticism so much as it displays contempt for those who have different ideas. Or, to put it another way, "Whatever they say, I"m against it."

    Unfortunately, the childish attempt to broadly discredit the republican party in one line, gave notion that you are judgemental and narrow minded. Interestingly, it overshawdowed what could have been an intellectually stimulating piece.

    Hank
    Wha ...?  A militant leftwing atheist in academia took a shot at Republicans???  

    You are new to science, I guess.  :)

    In a perfect world, politics would never come into play but in the real world, it often does.
    I also question your acceptation ;)

    Will you provide a link to the technical paper you reference at the end of your post? I've been spending a lot of time lately mulling over the epistemological underpinnings of the null hypothesis, and am interesting in seeing critiques of the technique.