Banner
    Chocolate Makes You Smarter, So You Stop Reading BBC News
    By Tommaso Dorigo | November 19th 2012 04:20 AM | 13 comments | Print | E-mail | Track Comments
    About Tommaso

    I am an experimental particle physicist working with the CMS experiment at CERN. In my spare time I play chess, abuse the piano, and aim my dobson...

    View Tommaso's Profile
    UPDATE: BBC radio contacted me to let me know they corrected their mistake. I am very glad to hear that! So you can continue reading BBC after all!

    Probability inversion is one of the nastiest mistakes one can do handling the results of a statistical analysis, invalidating to the roots the interpretation of the data to the point that the whole work effectively becomes useless. Unfortunately, it is a very common entertainment for journalists reporting scientific results, and oftentimes scientists themselves fall in the trap.

    In a nutshell, the issue is taking a statement on the probability of observing some data, given some hypothesis, and turning it into a statement of the probability of the hypothesis given the observed data. If I observe that a chunk of LHC data does not contain any Higgs boson and calculate a probability of 0.001 of observing such an effect if the Higgs boson does exist, that does not mean that there is a 99.9% probability that the Higgs boson does not exist, given the observed data!

    The inversion mistake is nasty because it is quite easy to make it: we are accustomed to reason in terms of probabilities of the causes. We observe some data and we should stick to making statements on the probability of the data, but we much prefer discussing hypotheses, and we err. (Note that I will resist the temptation to enter a discussion of Bayesianism here, since the focus of this article is elsewhere).

    For today's example of probability inversion, let us take the BBC News site, which features a fun piece titled "Does Chocolate make you clever?". In it, it is argued that there is a strong correlation between the number of Nobel laureates of one country and the chocolate consumption in that country:

    "Messerli took the number of Nobel Prize winners in a country as an indicator of general national intelligence and compared that with the nation's chocolate consumption. The results - published in the New England Journal of Medicine - were striking.

    When you correlate the two - the chocolate consumption with the number of Nobel prize laureates per capita - there is an incredibly close relationship," he says. "This correlation has a 'P value' of 0.0001. This means there is a less than one-in-10,000 probability that this correlation is simply down to chance."

    Note that this is written in a rather cryptic way. it is totally unclear what it means to say that a statistic -the correlation coefficient- has a p-value of 0.0001: does it mean that the particular value found happens only once in ten thousand trials ? Or that a larger correlation is found only once in ten thousand cases when there is no correlation ? Or a smaller one ? See, these are the pitfalls of careless reporting of mathematical expressions. Anyway, we are entitled to assume for the sake of arguing that the correlation coefficient is found to be large, and one as large or larger has been estimated to occur only once in ten thousand trials if there is no real correlation between the two quantities being studied (see, we talk of probability of the data!).

    Now, what is meant by "This means there is a less than one-in-10,000 probability that this correlation is simply down to chance" ? This is the culprit. Does this mean that the observed data is very odd, if one assumes there is no chocolate effect on the brains of the future Nobel prize recipients ? Or does it rather mean that the hypothesis that chocolate does nothing special to your brain is extremely improbable ? I bet a dime that 99.99% of BBC News readers will get the latter. Probability inversion trap !

    (I leave alone the issue of what may possibly have to do being "smart" with getting a Nobel price, because I am in a kind mood today).

    At this point, my advice to dr. Messerli, or to the journalist if it is her fault in reporting the sentence (alas, this is often the case), is to go eat a bar of chocolate. It will do no good to their mental abilities, but they will probably enjoy it.

    Comments

    "Note that this is written in a rather cryptic way." I hope the irony does not escape you.

    dorigo
    Hi Andrea,
    no I never had the pleasure to meet Bill, as far as I can recall.

    Cheers,
    T.
    One of my favourite examples of journalists committing this mistake:

    The German tabloid Bild ran an article with the heading "The core of the Earth consists of PURE GOLD" (http://www.bild.de/news/aktuell/news/erde-kern-gold-527532.bild.html)

    What actually happened was that some Australian scientists had found out that 99% of all the gold on earth is found not in the crust or mantle but in the core... hardly surprising given the atomic weight / density and the total volumes of material involved...

    Did you ever read this essay (The Earth Is Round (p < .05)): http://www.citeulike.org/user/mdreid/article/2643653 This is a great exposition of the pitfalls inherent in p-values.

    dorigo
    Thanks! Enlightening read. Just posted a quote.
    Cheers,
    T.
    somebody should drum into the journalists that correlation does not imply causation.

    This is all classic Dumbed-Down media pandering to their client readership (dumbed-down). It's similar to the the NY Times piece after the cancellation of SSC, correlating it with the "funny names" of elementary particles!! (See "Discovering Women" episode on Melissa Franklin/Harvard, where she expresses discust about this) It's just an Info-tainment piece, "based" on some data (keywords: chocalate Nobel Prize)

    "Why the NY Times doesn't get the right spin [ "story" ] on our data"
    -- Dr Melissa Franklin/Harvard, experimental Particle physicist

    "Facts tell [ Scientific analysis ], STORIES SELL [ enterainment based journalism ]"
    -- Motorsports saying ("Marketing")

    Below is a compact description of the "problem":

    reader from Cincinnati, OH November 1, 1999:

    "Everyone who gave this book one star should realize that this book is
    entertainment. Hancock is not a scientist or an academic of any kind - he's a
    journalist! ... Of course Hancock tailors the facts to fit his theories - he
    is not constrained by truth, science, or even ethics. He is a journalist.
    ...This book, and all those like it that preach pseudo-science, appeal to the
    majority of people in this world who are scientifically challenged. Most
    Americans don't have enough scientific knowledge to understand the technology
    they face everyday, much less untangle the fact and fantasy in this book. It
    is ENTERTAINMENT, but it's Dangerous - Science interpreted by a Journalist!"
    -- review of "Fingerprints of the Gods" (author is CRACKPOT journalist Graham Hancock)

    [ crackpot conspiracy theory = extraterrestial civilization in Antarctica were our ancestors, the clown has a book & 3-part Discovery Channel (!!??) episode..more media-dumbing. Making $$ off the crap. ]

    It should be noted that BBC Director General (George Entwistle) just resigned under cloud of scandal, related to Jimmy Savile (former BBC media icon) child-molestation scandal. Pulled the BBC documentary on Savile (implicating BBC as harboring a child-molester), plus an article falsely implicating Lord McAlpine as part of child-molestation ring. Entwistle was previously linked to slanted reporting of AGW (Global Warming alarmism). It all adds up to BAD journalism by BBC.

    "Once you lose Credibility, YOU CAN NEVER GET IT BACK"
    -- Journalism motto

    Good journalism is like Scientific Method: present both sides (objectively), & let the reader decide.

    Details below:

    Carl Brannen (Facebook)
    In Jan 2006 the BBC held a meeting of “the best scientific experts” to decide BBC policy on climate change reporting. People filed Freedom of Information (FOI) requests to find out who these experts were but the BBC refused to release the information. Then someone found the list using the "wayback" internet machine. It turns out that of the 28 attendees, only three were scientists (alarmist scientists of course). The rest of the attendees were straight-up activists and journalists. And the BBC attendees are so left wing that they're the ones who were forced out of the BBC for falsely accusing a Tory (conservative) politician of child molestation. See the rest here: http://omnologos.com/why-the-list-of-participants-to-the-bbc-cmep-jan-20...

    Latimer Alder (22 hours ago)
    "I disagree that these four were ‘low-level representatives’.
    Entwistle was ‘Head of TV Current Affairs’,
    Boaden ‘Director of News’,
    Rippon ‘Editor World at One,/PM’ – the main radio news and current affairs programme’ and Horrocks ‘Head of Television News’.
    Also in attendance was Jana Bennet ‘Director of Television’.
    How on earth any organisation can properly function with such a mish mash of a management structure is completely beyond me, but within those limitations, these guys were heavy hitters running the the most influential broadcasting outlets on Climate Change. Between them they covered all of the BBC’s news and current affairs. And they all spent a whole day being propagandised to by the activist wing of the alarmist movement."

    Entwistle resignation from BBC
    http://www.bbc.co.uk/news/uk-20284124

    "During his 54 days in charge, Mr Entwistle has also had to deal with controversy over the BBC shelving a Newsnight investigation into former BBC presenter and DJ Jimmy Savile, who police say could have abused as many as 300 people over a 40-year-period.

    As a result, an inquiry is examining whether there were BBC management failings surrounding the Newsnight's Savile programme not being broadcast, and another inquiry has begun into the culture and practices at the BBC in the era of alleged sexual abuse by Savile. Another review is to examine sexual harassment policies at the BBC.

    Mr Entwistle's resignation came after he was criticised for his performance during an interview on the BBC's Radio 4 Today programme on Saturday, in which he admitted he had not read a newspaper article revealing the case of mistaken identity involving Lord McAlpine, and that he had not seen the Newsnight broadcast when it aired on 2 November as he "was out"

    rholley

    And here is another example of ‘journalistic downgrade’.  Four says ago, I read this rather Scrooge-like headline in the Telegraph:


    How charity makes life worse for Africans

    Giving to charities that help African villages actually increases poverty, a study has claimed.
     . . . . . . .
    Researchers, working in conjunction with Addis Ababa University, concluded that those aged 15 to 30 with access to taps were three times more likely to migrate to a larger city or town in search of work and food than those without ready access to water.

    I was rather alarmed by this statistic, and so I contacted the senior author, who very kindly pointed me to this press release:

    The hidden consequences of helping rural communities in Africa

    Improving water supplies in rural African villages may have negative knock-on effects and contribute to increased poverty, new research published today [14 November] has found.

    Not so dire.  And here is the original paper:

    ‘Rural to urban migration is an unforeseen impact of development intervention in Ethiopia’


    by Mhairi Gibson and Eshetu Gurmu in PLoS ONE.

    A worthwhile and serious read.  However, I can think of other reasons why it might well make sense to stop reading BBC news.

    Robert H. Olley / Quondam Physics Department / University of Reading / England
    Hm, the BBC article is quite clear to me -- the graph shows the linear correlation r and the p-value.
    Near the end it also says (i.e. the author Messerli is quoted to say)
    """
    It might surprise you that we are trying to make a serious point. This is a classic case where correlation, however strong, does not mean causation.
    """
    From the graph one can see that there is a "strong (linear) correlation" r=0.791 and a low p-value
    --- but he doesn't say "causation" .

    More on
    http://cardiobrief.org/2012/10/10/chocolate-and-nobel-prizes-go-together/
    --
    luigi

    dorigo
    Dear Luigi,

    BBC changed their piece after I published the text above, and they notified me. Hence the update at the top.

    Cheers,
    T.
    ..and I've seen the UPDATE exactly *after* that I' ve hit the button, of course....another example of selective and obstinate blindness...