Banner
    Relative Risks And Why Anecdotes Can Be Lousy For Making Decisions
    By Kim Wombles | August 21st 2010 05:27 PM | 15 comments | Print | E-mail | Track Comments
    Fear blinds us, immobilizes us, and makes fools of us. Scary stories abound on the internet, through emails, and in conversations, and dangers lurk in the dusty corners waiting to pounce on us and tear our loved ones from our grasps. We know this. We feel it viscerally. And sometimes we shake in our boots. 

    We've got enough real dangers, and we do, without adding in made-up ones. We do a terrible job at assessing risk. Don't believe me? Which is safer? Driving or flying? If you said driving, you're so terribly wrong and have let both the illusion of control and the availability heuristic make you run with your gut. 

    If we admit this truth, that we are bad at assigning risks, then we should be looking for what science, statistics, and probability have to show on relative risks before we go off with our guts thinking we've chosen the least risky decision.

    And I write this as a mom who lives this, every day. Your gut is not what you want to go with when you're making medical decisions. It isn't. You want some good hard data to back your decisions. My son has factor V leiden, a blood clotting disorder which caused a stroke when he was nine. At that time, once the doctors determined he had the disorder, we had to decide on preventative treatment. Do you give a nine year old coumadin for the rest of his life? Our doctor decided with one stroke to go on that the risks of coumadin were greater than the risks of another blood clot. Our son went on the less risky aspirin. If you have kids, you know that aspirin in children can be dangerous. It was a game we played, pulling aspirin when he had colds. Falls, bumps, bruises are scary. Will it lead to overclotting?  Bumps to the head? Beyond terrifying still to this day.

    When he had his wisdom teeth out, he was sent to a hematologist for an alternative to aspirin in the weeks leading up to the surgery. The doctor was horrified that he'd been on aspirin for a decade instead of the standard coumadin and immediately wrote a prescription for that for after the surgery and one for lovenox injections leading up to the surgery. 

    And I started digging through the literature, pulling up articles showing that where there had only been one incident (especially with a decade with no additional ones), that lifetime coumadin, indeed coumadin for one year, conveyed far greater risk (3 out of 100 patients have bleeding problems on coumadin in a year, with 1 death for every 200 patients on coumadin from excess bleeding). My son's real risk of a thrombolytic event was around 5 in 1000.  You can see right there that the risks were vastly higher on coumadin than off, and after consulting with an academic hematologist, my son's doctor deferred to his judgment and back on aspirin my son went.

    When your family member's life is at stake, fear can be deadly, especially if it paralyzes you. It's incredibly hard to make sound, rational decisions when fear has coiled itself in your gut and twisted you in knots, so in order to be collected in crisis, you have to practice those reasoning skills in calm times. You have to look at the evidence dispassionately, and you have to defer to experts who know better. Sometimes that is indeed your immediate doctor, but even they can be paralyzed by fear and can make rash decisions. 

    When you've got a chronic condition, you've got to become the expert on it because chances are your doctor or your child's doctor won't be, won't have the latest literature, and won't have thought out hypothetical situations before hand and will simply react, as my son's hematologist did.

    It's okay, when it's not an immediate life or death situation to step back and say wait a minute, is that really standard of care? Is that what the guidelines call for? If you're deviating, what is your reasoning for it? It's hard to do, but it's your child's care.

    The trick, though, is to find reputable sources, and just because it's on pubmed doesn't mean it's sound science or the standard of care. You have to ask questions calmly and you have to be willing to listen to the answers without letting emotion get in the way. If you're letting anecdotes guide your medical choices, you're not looking at the situation rationally and demanding proof. 

    We need to embrace skepticism, and all the critical thinking tools at our disposal, in order to make good decisions, sound decisions, rational decisions for ourselves and our family members. If we can't do that when deciding on how to treat our children's colds and earaches and what preventative care we should use, then how will be able to do that when our child is seriously ill, potentially dying, and our decisions must be deliberate and rational? If you can't think critically on an easy day where it doesn't matter, how will you manage it when your life or your child's life does?

    In writing about the vaccine fears, Daniel Gardner,  in The Science of Fear, wrote:

     “The danger here is that we will collectively cross the line separating skepticism from cynicism. Where a reasonable respect for expertise is lost, people are left to search for scientific understanding on Google and in Internet chat rooms, and the sneer of the cynic may mutate into unreasoning, paralyzing fear. The end state can be seen in the anti-vaccination movements growing in the United States, Britain, and elsewhere. Fueled by distrust of all authority, anti-vaccination activists rail against the dangers of vaccinating children (some imaginary, some real-but-rare) while ignoring the immense benefits of vaccination – benefits that could be lost if these movements continue to grow.”

    I don't presume that I knew medicine better than the hematologist who reacted knee-jerk without considering the relative risks. I don't. Obviously. What I did know better that day, though, and in the subsequent time period of calmly convincing her that the numbers didn't support her treatment plan, was that ultimately, the buck stopped with me. I had to live with the decisions I made regarding my son's medical care and I had to do so with the most accurate, the best of, information. I am his mother and his legal guardian. I make the decisions he cannot make for himself and I have an obligation and a duty to do it well, to get it right. Fear gets in the way of that. 

    What I have to fall back on in times of crisis is critical thinking skills, a whole bunch of tools to help me make the best decisions I can with the information I have. 

    You should not decide to vaccinate or not to vaccinate based on anecdotes. You should not be swayed by anecdotes relating autism to vaccine injury, nor should you be swayed by stories of vaccine-preventable deaths. Anecdote doesn't give you the numbers. It gives you fear.

    Sometimes you have to roll the hard six. Make sure when you do, you have the numbers behind you. What are the relative risks? Which is less risky? For some individuals, vaccines are too risky. These people rely on herd immunity. For most of us, the relative risks are less with vaccines than without. 

    Say what you will about Penn and Teller, but their throwing balls at pins was an effective demonstration of relative risks. You have to decide do you have the balls, all of them, and can you effectively juggle them. Because if you're wrong, if you've gone with anecdote over science, you may have live with some spectacularly crappy results instead of shooting craps.

    Comments

    rholley
    What a story!  Quite a lot to take in at one reading.
    Two small questions:

    (1) Do you have a web reference to Penn and Teller? (2) What does "shooting craps" mean?  Also "roll the hard six"?  Perhaps games of chance are different on this side of the Atlantic.
    Robert H. Olley Quondam Physics Department University of Reading England
    Hank
    Craps is pretty popular here, first in lower income neighborhoods (because it only requires two dice) but now in casinos - basically, if you roll a 7 or 11 on the first roll, you win, if you roll 2, 3 or 12 you lose - any other number is the Point, and if you roll the point before you roll a 7, you win.
     
    A hard six is two dice showing 3 and 3 but I had to look up why it is so hard.  Turns out that has a probability of 3% whereas a six by any other combination has a 14% chance.  In casinos, a hard six pays 7-1 but any other six pays only 7-6.   Hard indeed!

    No idea on Penn&Teller, though.   :)
    rholley
    Reminds me of those essays my students did on Fermat, Pascal, and the development of Probability Theory.
    Robert H. Olley Quondam Physics Department University of Reading England
    kwombles
    Penn and Teller recently did an episode on Vaccines. It can be found at http://autism-news-beat.com/archives/1225. Vaccines as a cause of autism is hotly debated in the autism community, despite the overwhelming evidence to the contrary. :-)
    “Nothing in the world is more dangerous than a sincere ignorance and conscientious stupidity.” --MLK, Jr.
    robarcher
    Hi, you might be interested in Jonah Lehrer's work on intuition, might give you more confidence in your decision making...
    kwombles
    Hi Rob, either you're being funny or you didn't read the post.
    “Nothing in the world is more dangerous than a sincere ignorance and conscientious stupidity.” --MLK, Jr.
    Hank
    If historical patterns hold true, you can read Andrea Kuszewski's work on the same topic a few weeks before Lehrer.
    robarcher
    sorry forgive me if that came out wrong - I only meant that Lehrer's argument might reassure you that even when you're in a position where you *can't* know all the facts pertinent to a decision, our intuition is better than we sometimes think it is.  I totally agree with the main thrust of your argument though, I was merely trying to add to it.

    Thanks Hank for that reference too...
    kwombles
    Thanks for explaining; I spend a lot of time on my other blogs working to helping individuals who have fallen for illusory correlations along with the availability and affect heuristics that going with the gut is a dangerous and often foolhardy choice. Intuition, where we're discussing what have become automatic processes, is not a thing to be dismissed out of hand, but it must be intuition based on experience and knowledge, along with the awareness of the pitfalls of various cognitive biases. In other words, even then, the rational mind should check the intuition.
    “Nothing in the world is more dangerous than a sincere ignorance and conscientious stupidity.” --MLK, Jr.
    I'm also positive for FVL along with having APS. After one DVT/PE episode, I'm weighing whether to stay on coumadin indefinitely or not. I've seen the 1 in 200 statistic for fatal bleeding complications but it seems to include the whole range of people taking coumadin including the elderly. I can't find statistics that show the risk for young and otherwise healthy patients. I've consulted 3 drs who believe my risk of reclotting is much higher than my risk of bleeding. You quote your son's risk at 5/1000 or 1/200 for another clotting episode. Is that strictly for FVL? Where do those numbers come from? From the numbers you mentioned, your son's risk of clotting without coumadin or of a fatal bleed on coumadin appear to be equal. I'm not trying to make you second guess your decision, I'm just looking for information to use in making my decision.

    kwombles
    The risk of any clot is 5 in 1000 per year, not the fatality rate. The fatality rate for coumadin per year is 1 in 200, placing the greater chance of fatality clearly on the side of the coumadin. This site puts it at 1 in 1000 (http://www.med.illinois.edu/hematology/ptfacv2.htm) but this study puts the annual incidence at .06%: "Activated Protein C Resistance and Factor V Leiden: A Review" by Adam Rosendorff, MD; David M. Dorfman, MD, PhD. The decision on whether to medicate or not is one that should be between the patient and the doctor. There are risks either way.
    “Nothing in the world is more dangerous than a sincere ignorance and conscientious stupidity.” --MLK, Jr.
    I thought everyone "knew" that flying was safer than driving. I was also under the impression that the truth of that depended heavily on how you chose to compare them, looking at deaths per mile, per hour, etc. Apart from that, I can't help but wonder if anecdotes are always such a horrible basis for judgment. After all, statistics are easily manipulated, especially by people with an agenda. Anecdotes are usually passed on to you by friends, people you trust, who are more likely to be relating their own experiences with a view to being genuinely helpful.

    Gerhard Adam
    Which is safer? Driving or flying? If you said driving, you're so terribly wrong and have let both the illusion of control and the availability heuristic make you run with your gut.
    I thought you made an excellent point and you're absolutely correct to argue about the necessity of checking the data yourself and reaching your own conclusions.  Having said that, this particular statement of yours is interesting primarily because it is anecdotal.  There's absolutely no question that the results can be manipulated into whatever your desired outcome is. 


    kwombles
    While I didn't substantiate the comment in the post, it is far from anecdotal and was based on the rate of fatalities per year for airplane crashes versus the fatality rate for automobiles per year. Freakonomics covered this on pages 150-151. It is also covered at this site: http://www.newton.dep.anl.gov/askasci/gen99/gen99845.htm. This also shows flying to be safer than driving, although it stipulates nonstop flying, interestingly enough: http://deepblue.lib.umich.edu/bitstream/2027.42/868/2/79493.0001.001.pdf Good science isn't about manipulating to get the data you want.
    “Nothing in the world is more dangerous than a sincere ignorance and conscientious stupidity.” --MLK, Jr.
    Gerhard Adam
    While I didn't substantiate the comment in the post, it is far from anecdotal and was based on the rate of fatalities per year for airplane crashes versus the fatality rate for automobiles per year.
    Actually it is quite anecdotal, since usually such statistics only use commercial airlines and not general aviation data.  Secondly, they are usually based on passenger miles which will always skew the statistics in favor of the airlines (rather than on the number of trips taken per person).  In fact, a strong argument can be made that passenger miles is a red-herring, since there is no basis for believing that the risks increase with distance.  Similarly one could argue that commercial airline pilots are more experienced and professional, but then one would have to compare other ground based forms of travel that are equally professional (i.e. buses).

    Just to show that I'm not biasing the data here, the point of accidents needs to be to employ the statistics per event (i.e. the number of times one gets into a vehicle and the probability of something happening to them).  Therefore automobile travel incurs substantially more trips than air travel does, however while more fatalities occur, they are actually fewer when one considers comparing the total number of trips per year for each.   

    I'm not suggesting that air travel is unsafe, only that it isn't the orders of magnitude safer than is commonly portrayed.  Once again, you may disagree with my approach to the data, but it is intended to show that the results are quite anecdotal and depend completely on how one elects to interpret the data. 

    Add a comment

    The content of this field is kept private and will not be shown publicly.
    • Allowed HTML tags: <sup> <sub> <a> <em> <strong> <center> <cite><TH><ul> <ol> <li> <dl> <dt> <dd> <img> <br> <p> <blockquote> <strike> <object> <param> <embed> <del> <pre> <b> <i> <table> <tbody> <div> <tr> <td> <h1> <h2> <h3> <h4> <h5> <h6> <hr> <iframe><u><font>
    • Web page addresses and e-mail addresses turn into links automatically.
    CAPTCHA
    If you register, you will never be bothered to prove you are human again. And you get a real editor toolbar to use instead of this HTML thing that wards off spam bots.