Banner
    Fact Checking Or Copy Checking? Journalism In Science
    By Hank Campbell | September 30th 2011 04:30 AM | 15 comments | Print | E-mail | Track Comments
    About Hank

    I'm the founder of Science 2.0® and co-author of "Science Left Behind".

    A wise man once said Darwin had the greatest idea anyone...

    View Hank's Profile
    Imagine if a journalist sent their story to former Republican Vice-President Dick Cheney to 'check the facts' - what would the outcry sound like from other journalists and the public?

    Yet it turns out a surprising number of writers in science will send parts of their articles to the  researchers whose work they are covering under the guise of fact checking, and that seems to be okay with many others.  Is science journalism that hard? Is that a good policy? Ananyo Bhattacharya, chief online editor of Nature, gets it right when he notes in The Guardian that kind of behavior doesn't serve journalism, science or readers.

    I have noted many times that science journalism departments have disappeared not simply because media companies are greedy but rather that when science journalists became advocates and cheerleaders, they stopped being trusted guides for the public and the public stopped reading mainstream media science.  Science journalists were always under the gun by scientists for getting too much wrong, while the public more recently distrusted journalists due to perceived bias, so one founding principle of Science 2.0 was to make more scientists into journalists(1) and at least get the facts right.

    It turned out to be a lot more complicated than that, at least when it comes to providing context. Science journalism is hard and certainly some journalists may be "fanboys" of science and so aren't going to write anything that will arouse the ire of someone they have heard of(2), but some fact checking for anyone is a good idea - however, sending an article, or even parts, to the source can get weird.  I understand why it happens, especially when a complex story has a short time frame, but it still shouldn't happen.

     Scientists who get written about here can gripe if they want but the solution is to either write their own Science 2.0 piece or let it go and be happy with a press release in the future.  Bhattacharya says some scientists might 'withdraw' their participation if they don't like an article or get to check the content in advance but so what?  The Science 2.0 brand is strong, as is Reuters or anyone else of any size.  Is a journalist going to refuse to talk to USA Today if they don't get pre-approval of a piece? I think not. A lab writes one article per year and if a science writer is so star-struck they think they need one researcher to make an article valid, they are in the wrong field.

    Science 2.0 fave Prof. David Kroll broached the topic after watching an interview with Trine Tsouderos where he heard that she ran some swatches of copy by sources in the interests of 'getting it right' - and his informal poll of other science writers confirmed they do it too. Egads, did the reading public just feel their hearts sink a little? 



    I think they are wrong, 100 percent of them, but he got similar input from mainstream science writers so you see the problem.  Again, science is hard, I understand that, and it's hard because writers aren't always dealing in facts or quotes but also in context, and a primary source can be tough going for a journalist who is expected to somehow be literate in a dozen fields that researchers got their Ph.D.s in - and to write articles in a day (or, in the case of some consumer science sites, up to six pieces each day!).  I have fact checked my context plenty of times - but with researchers in those fields who are here at Science 2.0, not with the researcher whose work I am writing about.  'Tell me if I am getting this right' to a fellow writer who happens to have an education depth I do not is much different than making the primary source an assistant editor.


    In true science media fashion, Kroll's article has 109 comments while the actual video itself has only been seen 300 times by the entire world, which means most of the commenters never watched it, they took the context on faith.  It's easy to criticize the public for not doing their own research, clearly most in science don't either.

    How could a journalist cover ClimateGate by sending his copy to Michael Mann or people at East Anglia?  Four times per year we get articles hinting about life on other planets and a missing link but it would be pointless to let them spin their work for the public.

    In 5 years of Science 2.0, I have found scientists incredibly gracious in their time, both in answering questions and in writing articles, they want the public to know what they are doing - but we have never given in to a request or a demand to see a pre-print, because in a comment on Kroll's piece, John Rennie lays out what might happen; if our coverage of the work is favorable, they might talk about their 'help', which the public will regard as being a public relations effort rather than insight, and if our coverage is critical, they can cite the items a writer was unsure about as proof we didn't know what we are talking about.

    The only solution is to let them provide context and insight when you are covering their research, but with no strings attached and no editorial pen.  If they don't like having no control, move on, regardless of who it is.

    Deborah Blum mentioned Edward Teller once demanded the right to see the interview in advance, and she gave in, but that was likely a case of a young journalist being star struck and wanting a big name.   Despite only correctly a few technical details, she said he hated the piece anyway and never wanted to talk to her again.

    She was never going to win in that scenario and science journalism remains primarily a thankless job today - but it can be respected again if the public sees journalists as outsiders with a gift for making complex topics understandable, and not wanna-be scientists, the way actors and the public regard movie critics, or policy advocates. 

    Fact check, yes.   Copy check, no.  Get back to being trusted guides - including, it must be said again, for people who are not your political persuasion.  Ask the awkward questions, even of scientists and studies you like.

    NOTE:

    (1) Over time, my respect for science journalism has grown and I would instead contend editors were the bulk of the problem in the last decade.  Yes, some journalists are in science to advocate their progressive world views as opposed to caring about science, but for the most part the science journalists remaining want to get the story right.

    (2) If humility is not evident, it will be bestowed. From inception, the science writers here knew to euthanize me if I ever came close to writing something like Adam Bly of the now-defunct SEED Magazine wrote about his love of science and its contributors:
    My role as editor of this magazine offers me few pleasures greater than sharing a meal with a fascinating scientist. It starts with the ideas on the table, certainly, but for me it's also the distinct cadence, the fluttering of the hands, the brush of the forehead, the coy grin that lets you know you're being let in on one of nature's secrets. Inspired by a friend here in New York who regularly hosts great thinkers in his home, I've even installed (with considerable effort) an oversize blackboard in my dining room for those occasions when you just need to see it in chalk.

    Comments

    vongehr
    If it went down Adam Bly fashion, it was of course ridiculous. On the other hand, why not send it first to Dick Cheney to check the facts? Likely there is indeed something that he can clear up best in an article about him? Just sending it does not imply the author must go along with any of the changes Dick may suggest. This fact checking should be encouraged and additionally include at least two competitors.
    Hank
    I agree that is scientific fairness, but a journalist would never agree to that.  Any change means the subject can claim they helped and by asking at all a journalist establishes he may be wrong and needed to ask.

    An independent party is different; I have asked plenty of scientists here to check my copy to make sure I am interpreting correctly, that is fact checking or, in my case, context checking.  
    vongehr
    Just went to the article you linked - agree with most of the comments (cuz they agree with me, he he). "the subject can claim" anyway, and why would the journalist need to "establish he may be wrong"? He is human, right? Those are usually able to be wrong.

    Writers are very 'adept' in having the correct facts and nevertheless build the story around it in such a way that it becomes totally misleading and distorts the facts. So the distinction between fact versus copy checking isn't really there, especially not with dumb editors doing the final touch. The more original and cutting edge, the more important it is that the final product is seen once more.
    "Kroll's article has 109 comments while the actual video itself has only been seen 300 times by the entire world, which means most of the commenters never watched it, they took the context on faith. "

    What an astounding assumption! How do you KNOW that the 109 commenters didn't all watch it (perhaps several times) before commenting? Perhaps it means that one third of the people who watched it were moved to comment?! You may be right, but did you contact all of them to check your facts?

    You clearly have a very firm opinion on whether the subject of the piece should read it before it's published. Personally, I agree with Sascha, I can't see any reason why a person should NOT be allowed to read through the article before it's published, whether they're a scientist or a politician. The author then has the right to include their changes, or not. As a reader, I'd far rather read an article that gets it right the first time, than one which has a comment buried somewhere beneath it because the author (and the first google search entry) got the subject's birth-date wrong. For starters, it makes the author look like less of an idiot.

    "I have fact checked my context plenty of times - but with researchers in those fields who are here at Science 2.0, not with the researcher whose work I am writing about."
    To congratulate yourself for checking your facts with other science journalists, rather than the people who've actually done the work in the field? Find a definition for hubris ...

    Hank
    ption! How do you KNOW that the 109 commenters didn't all watch it (perhaps several times) before commenting?
    How do you KNOW any of them did? If it only had 300 views and it is safe to assume their core audience watches more rather than an outside link, not many commenters watched it. You may not agree with a 95% confidence interval, which is fine, but why would that make you so indignant?
    To congratulate yourself for checking your facts with other science journalists
    If there are three science journalists here, I am surprised.  I said researchers who are Ph.D.s. You can be forgiven for not reading the article, but you missed the most important part and then declared I have hubris based on your incorrect conclusion.
    Re the 109 commenters, I (stupidly) assumed that as it was YOU who made the assertion, it'd be YOU who had to have the evidence to back it up - basic concept of scientific proof and all that. It's also one of the basic journalistic principles (If you print it, and you can't prove it, you're likely to get sued - or fired - or both). But you go ahead and make any assertion you like with nothing to back you up, and blog about it. Eventually someone will take you to court.

    Re the fact that you have only three science journalists, to quote earlier in the article "so one founding principle of Science 2.0 was to make more scientists into journalists". Clearly you've either failed in your principle, or there are more science journalists than you realise. Perhaps you don't classify them as such because they don't happen to agree with you? I still believe that the person who is most likely to pick up an error in your interpretation of an article about a particular person is - the particular person. After all, they're generally pretty familiar with the topic.

    Hank
    Clearly you've either failed in your principle, or there are more science journalists than you realise. 
    The third alternative is that you lack simple reading comprehension. I tried to be nice because I understand you are new to Science 2.0 but your ill-fitting attempts at being either funny or prescient fail, leaving you smug.   At no time did I ever write I made scientists switch careers, they are still scientists, they simply write science themselves instead of filtering it through journalists.

    It is impossible to let the subject of a story edit the story. You are neither a scientist nor a journalist so I understand that may be a disconnect for you, but if it wasn't a disconnect for every journalist except a few in science, it wouldn't have been worth writing about.
    Smug? Pot, let me introduce kettle.

    Sometimes people don't believe what you believe. As a scientist, that should mean that you use logic and reason to support your argument, and that you're willing to admit it when they have a point, or when you've got it wrong. You might want to work on that...

    Speaking of reading comprehension, at no point have I suggested that the subject should EDIT the story, just review it and suggest changes if there are things that are wrong. To make it absolutely clear to you, here's part of what I actually said..
    "I can't see any reason why a person should NOT be allowed to read through the article before it's published, whether they're a scientist or a politician. The author then has the right to include their changes, or not "
    But glad to see you're impugning the reading comprehension of others, Mr Pot.

    I've got better things to expend my energy on than a flame war, so I'm turning notifications off now. You can go back to just listening to all your adoring fans....

    It's a bit more complicated than what you're dicussing here. Google up the Heidelberg Appeal or Oregon Institute of Science and Medicine and see if jounalists are the only ones who can be forgiven for double checking the facts.

    Then again, this approach assumes quality and expert - & infallible - journalists.

    Checking the copy might have saved one journalist from publishing a very embarrassing assertion that a new study linked heel length with attractiveness. The quotes were correct, but the writing got it completely wrong. Even the article abstract asserted that heel length was an anatomical measure, not part of a shoe. Had that writer let the researcher check the article, she might have avoided looking ignorant and sloppy. (Considering that heel was clearly defined in the publicly accessible abstract, I even wondered if the writer was scientifically literate).

    I work on book-length works, and my publishers would consider it irresponsible to not have copy peer reviewed. I believe that peer review is also the basic principle behind journals. The refusal to have copy checked does make journalists stand out as arrogant and not holding themselves to very high standards.

    Hank
    I work on book-length works, and my publishers would consider it irresponsible to not have copy peer reviewed. I believe that peer review is also the basic principle behind journals. The refusal to have copy checked does make journalists stand out as arrogant and not holding themselves to very high standards.
    Right, and I have a luxury journalists may not have - a whole lot of experts that are members here and only a click of the chat button or an email away.  So they can check my concerns for context, like I said.   I never contended any non-expert should lock and load a poorly understood article without making sure they understood it.

    Some of this problem will become less relevant and more historical.  There was a time when every media outlet had science reporting and adding more people lowered the quality threshold.   If the field continues to narrow and we are left with Greg Critser's and Carl Zimmer's, we're obviously getting top-shelf stuff, just a lot less of it to go around.

    On your final point, no respectable peer-reviewed journal lets the author of a study do their own peer review.
    Hank,

    Go back and read the string of comments under David Kroll's posts again. I don't think anyone, including me, is talking about seeking approval or permission for quotes, etc. It's not about allowing sources to edit copy. It's not about writing stories by committee.

    My feeling is if I don't feel comfortable showing sources chunks of copy before it goes in, then I shouldn't be publishing it. If my sources have problems with it, I should be able to defend what I wrote and if I cannot., then I need to do more reporting.

    I don't think anyone has ever accused my stories of being wishy-washy or written by committee. And they all included this rigorous fact-checking process. Certainly not every source was happy with everything I wrote - and yet look! I still wrote what I wrote because it was solid and true and accurate and fair.

    All of this handwringing seems pretty silly to me.

    Hank
    Hi Trine,

    I wrote
    an interview with Trine Tsouderos where he heard that she ran some swatches of copy by sources in the interests of 'getting it right' - and his informal poll of other science writers confirmed they do it too.
    'some swatches of copy' to get it right is not alleging you write by committee, it's what you said, and I don't think anyone here read more into it than that.  Various journalists have various styles when it comes to that (obviously - else there would be no reason to discuss it) but do you think it's common in other forms of journalism?

    What handwringing?  
    Definition of handwringing: "An excessive expression of distress"
    or
    "Egads, did the reading public just feel their hearts sink a little? "

    Hank
    Okay, we are going to disagree on handwringing - you are the writer, not me, so I suppose I can concede that.  It isn't handwringing in my mind because I don't think sadness at seeing people run their copy by sources is excessive.  If all forms of journalism run parts of their text by their sources, then I think all journalism is a concern - I just never heard of it being done before. 

    I've only written 1,200 pieces or so, and obviously much looser than what you do in mainstream journalism, but on various complex topics and I have never had the source of a study tell me I got it wrong.  They didn't always like it, and argued nuance (like arguing over whether a heart sinking a little is handwringing) but no one said I was wrong. Sometimes it was because I had someone help me make sure I was discussing it properly.  I never said not to ask an expert, I just don't like asking the source itself.  It's not a big deal to have one person on the Internet disagree with your method.  Celebrate diversity.