Banner
    Publish Or Perish - Is That Journal Peer-Reviewed?
    By Hank Campbell | September 10th 2011 04:02 AM | 6 comments | Print | E-mail | Track Comments
    About Hank

    I'm the founder of Science 2.0®.

    A wise man once said Darwin had the greatest idea anyone ever had. Others may prefer Newton or Archimedes...

    View Hank's Profile
    Once upon a time, journals were horribly expensive to produce and to read.  Your research might only be read by 200 people but those 200 people knew the work was vetted by reviewers.   It had a quality standard.

    Open access publishing is a blessing and a curse in some ways.   Some very popular journals are not peer-reviewed, they are instead looked over by an editor who may or may not be qualified to determined its technical validity - but since they are taking money from scientists to publish the article, rather than relying on subscriptions, the concern is that the researcher is now the customer rather than the science.

    Open access publishing has been fantastic for researchers and the public because it gave everyone access to data - but it has growing pains.   Who is to blame?  Publishers who regard publishing a lot as a necessary evil in order to publish good stuff?  Maybe.   Corporate media is corporate media and anyone generating millions of dollars is thinking in terms of perpetuating its existence.

    David Colquhoun, writing in the Guardian, instead blames funding agencies and university public relations people.  It's a bizarre claim; funding agencies want to know that society is getting value out of research and the way to do that is to track how many citations and papers that result.  Nothing in a funding agency mandate says researchers should just pay a thousand bucks to someone publishing 7,000 articles a year.  Likewise, university media groups have a thankless job - they have to do their best to make dry research interesting to journalists and the public and researchers often have little interest in that.

    Yet Colquhoun says "hard-pressed authors go along with them". What does that even mean?  Does he know any scientists?   Literally every scientist I know would jack up any marketing person who tried to oversell their research.  Research gets oversold all of the time - we get a missing link story every two months and astronomy could practically co-brand sentences 'may have implications for finding life on other planets' but researchers, for the most part, get that their research is incremental.  No one is curing cancer.  It's still interesting and that doesn't have to be reflected in a story on Science 2.0 or USA Today.

    He does note an alarming statistic;  the US National Library of Medicine indexes 39 journals that deal with alternative medicine. They are all 'peer-reviewed'', he says.
    The peer review for a journal on homeopathy is, presumably, done largely by other believers in magic. If that were not the case, these journals would soon vanish.
    This leads to a credibility problem in science and it can placed in the context of our desire to make research be more open; how will people know when peer review is meaningful and when it is not? We have done half a decade of work to make science more open, and it has worked, but how do the public know what open access is credible?

    There's good news too.  The twin of open access science is the rise of citizen science and citizen journalism.  In politics, non-experts are a keen source of information about political candidates when the mainstream media picks someone they want to win and glosses over the facts.   So it goes with science; science bloggers/columnists have replaced science journalists as assets when it comes to providing context for research, new and old.

    Science bloggers have their own biases too so how will the public know which science bloggers are agenda-free?  Well, they won't, people will continue to read sources that affirm what they want to believe, as they always have, but at least they can read science studies now, more than ever.

    Comments

    Stellare
    "We have done half a decade of work to make science more open, and it has worked, but how do the public know what open access is credible?"

    Finding reliable information is one of the greatest challenges for decision-makers of all kinds.

    As someone from the scientific community I know the limitations of peer-review - and I also know the limitations of open access (without peer-review). The best we can do is perhaps to have a mix?
    Bente Lilja Bye is the author of Lilja - A bouquet of stories about the Earth
    Hank
    A mix is ideal - I called Australopithecus Sediba a big win for open science because not only did the researchers make it available, but because it is available it will be ground zero for a lot more research and so therefore it will get used a lot more and be a huge discovery as far as citations.  In contrast, something like Darwinius and its museum that won't let anyone outside in its protected sites ends up producing suspect claims.

    Scientists are no different than anyone.  The bulk of them will be followers, so they aren't embracing any open access journal until it has the impact factor of Nature, and some will be hucksters so they want to pay to be in an open access journal just to have it out there and it's fine if some editor glances at it and runs their credit card - and the confusion is among the science community the same way it is the public.  Polling most scientists about some well-known open access journals will find them all lumped as peer-reviewed, even though they are not.

    I have been kicking around a Science 2.0 journal for years.  The block is the current system and being 'ahead of its time' - I think I can do a truly open publication, not pay-to-read like Nature or pay-to-publish like PLoS but truly free and peer-reviewed.  And I could even pay reviewers, which is bucking the system.  But it may be too much change for a field that took 2,000 years to get to Science 2.0.  :)
    'Yet Colquhoun says "hard-pressed authors go along with them". What does that even mean? Does he know any scientists?'

    My understanding is that David Colquhoun is a pharmacologist at University College London (http://en.wikipedia.org/wiki/David_Colquhoun).

    Hank
    I was asking somewhat rhetorically - claiming that scientists are being pushed around by marketing people made me wonder if he was just making stuff up for effect. The bulk of scientists don't even like speaking to the media, they are certainly not getting pressured by 24-year-old people writing anonymous press releases.

    I am sure it may happen, and he may be taking anecdotes and extrapolating them out to make it sound common, but it isn't common.  He may know far more scientists than me but generally scientists are embarrassed by fluffy, exaggerated press releases about their work - at the next conference they attend, their friends will make goat noises at them and run away and there is only so much 'the marketing department wrote that' that will fly.
    I think perhaps you should check before asking whether I'm a scientist. There is this thing called Google -it's quite easy really. You should try it.

    More seriously, I wonder where you got your information from. In every case that I've followed up of a hyped up story in the media, the origin of the hype has been the university press release (which has, one hopes, been approved by the authors). The only sin committed by the journalist was to reproduce the press release without reading the paper critically.

    I'm touched by your faith in scientists, but I fear that it isn't always justified. The publish-or-perish culture promoted by universities and funding agencies has put enormous pressure on scientists to hype their findings, and sadly not all of them resist the pressure.

    Hank
    Hi professor,

    Your position at UCL was in the article and 'does he even know any scientists?' was rhetorical. No one else reading this actually assumed a professor in pharmacology doesn't know any scientists.

    I agree some stories in the media are hyped but that does not mean the scientists are being pushed around by their press departments, which is where you and I differed.  The truly outrageous types of stories I mentioned (missing links, life on other planets) are the scientists themselves so that can't be laid at the feet of a funding agency either; nebulous 'pressure' claims do too much to make behavior exculpatory, the same as a gang member who stabs someone going to court and claiming his circle made him do it. If the modern generations of scientists have become that mentally weak we have a much greater problem than too many journals.  

    Everything else in what you wrote, including concern about the alarming number of alternative medicine peer-reviewed journals, I agree with.  But those are the price of making science more open.

    There is no way to have taxpayer-funding with no accountability or open science without some bad science.  I tend to give the public and scientists more credit (perhaps too much, as you noted) but the Science 2.0 experience has validated that.

    Best regards,

    Hank