Open Access And Good Citations: The PLoS Factor
    By Hank Campbell | June 22nd 2010 04:20 PM | 32 comments | Print | E-mail | Track Comments
    About Hank

    I'm the founder of Science 2.0®.

    A wise man once said Darwin had the greatest idea anyone ever had. Others may prefer Newton or Archimedes...

    View Hank's Profile
    The big war in science during this decade has not been Republicans against human embryonic stem cell research or Democrats against agriculture, it has instead been open access publishing versus subscription peer-reviewed journals.

    Open access publishing of science results, freely available to all, would clearly kill subscription-based peer-reviewed journals.  Right now, those peer reviewed journals are terrifically profitable for multiple companies despite the fact that everyone is saying print is dead.   These companies add value to researchers, they say, by having a higher impact than other companies that do less marketing, etc.

    But something strange has happened.   Open Access journal PLoS One received its first Thomson Reuters Journal Citation Reports impact factor and it is a respectable 4.351.   Science and Nature, with 29.747 and 34.480 respectively, are not shaking in their corporate boots just yet but it's a sign that things are changing. 

    Why talk about PLoS One?  Because that is the one traditional media hate the most.  They are basically okay with PLoS Biology (12.196 impact factor, down even from 2006) because it is a niche imprint and loses money so is no threat to them.   PLoS One, on the other hand, is the thing that keeps the doors open at PLoS.    They accept 7 out of 10 submissions and, if you have read some of the stuff that gets printed, you have to wonder what the 30% rejections must look like.   But because they charge $1,350 for publication, and are in the top 3 in volume worldwide, they are basically printing money for the company.

    Nature is not a fan, calling it cynically an effort to "challenge academia's obsession with journal status and impact factors." (Giles, J (2007). "Open-access journal will publish first, judge later" Nature 445 (7123): 9. doi:10.1038/445009a. PMID 17203032.)  And indeed, original editor Chris Surridge once wrote "With a journal like PLoS ONE Impact Factor will be irrelevant" but they are pretty darn proud of impact factor now, much like they originally said money-focused corporate media was a curse but they proudly frame the success of PLoS One in terms of how much money it makes because that is a sign of success.

    It was assumed by competitors that a mass volume product like PLoS One would generate some revenue but would fail when it came to being meaningfully cited and thus have no value for researchers.  The latest results debunk that idea.   So it's all good, right?

    Yes and no.   A lot of it is automated, so costs are not high, but with success comes attention and attention means more manual work because you can't let kooks slip through the cracks.  As all of us know, kooks are drawn to anything 'open' that can legitimize them.     That means more people looking at things and, if profitability is to remain, that means convincing people to work for free.    Scientists like outsiders and PLoS has been the alternative for people who distrust multi-million dollar corporations, "non-profit" or not, but now that PLoS is a multi-million dollar publisher, and profitable, the scientists who have been enthusiastic advocates because PLoS was the underdog may begin to wonder if they should continue to go above and beyond what they do for anyone else.

    But if they can navigate the perceptual issues (hint: hire someone from Apple marketing - they make closed-source tools using child labor in foreign countries, charge too much, and have a bigger market capitalization than Microsoft yet have the reputation of being the cool, edgy outsider for hipsters) their model is going to take over the world, because traditional publishing says you can have volume or you can have a good impact factor.

    For now, PLoS One is doing both.

    More on the open access aspects of Science 2.0:

    Sharing Research Leads To Good Citations

    Open Access Doesn't Lead To More Citations, Says Study

    Why Do Democrats Want To Get Rid Of The NIH Public Access Policy?

    The Downside To Open Access Science

    Open Access War On The American Chemical Society


    "They accept 7 out of 10 submissions and, if you have read some of the stuff that gets printed, you have to wonder what the 30% rejections must look like."

    The Nature, Science, Cell group is hardly any different. In fact, regardless of journal prestige, a lowly undergrad can often pick out loads of errors in any given paper. The stuff in "lower" journals is often better quality-wise, being less ambition and less inflated. It is up to the reader to be paying attention and thinking critically, not for the publisher to dictate was is worth reading.

    Peer review has this fundamental flaw in the first place where the closer the reviewers are to a field, the more likely they are to have some sort of personal interest in the review, and the more disinterested they are, the less likely they are to know the first thing about the field. Thus, while it sort of works to weed out the exceptional junk, it does not guarantee quality. And editors are even more pointless in that regard. I say let the information roam free, both the good and the bad stuff alike - perhaps the readers will be a little more aware and be less prone to simply copying what the generations of scientists have said before them.

    If the "high-tier" journals are so high quality, then why is it that they seldom ever get something as basic as the tree of life right? I mean, we've known for at least 15 YEARS that fungi, plants and animals DO NOT form a 'crown eukaryote' group with a single origin of multicellularity. Yet why is it that I still see this assumption prevalent in Cell, Nature and Science? They're doing little better than PLoS ONE, but with a lot more pomp...


    "But because they charge $1,350 for publication, and are in the top 3 in volume worldwide, they are basically printing money for the company."

    This implies that $1,350 is somehow unreasonable. But is it?

    It implies just the opposite.   If I have a study and the only barrier to publication is $1350, that is meaningless compared to $200,000 in funding or more.   Otherwise the wait to publish could be months while other publications negotiate in series, since there is basically no peer review to bog anything down at PLoS One.  For that low cost and low editorial barrier alone, they can be successful, but it doesn't mean they will be, since researchers won't spend any money at all if the publication is overrun with cranks.  PLoS One does both so obviously something is being done right.

    2 other people ask for clarifications on quality that is 'not very good' - it is a case of the human condition that people with a vested self-interest in a project will read an article like this, saying  PLoS could take over peer review, and find objections to smaller parts, since I am betting no one there has read all 4,400 articles published last year - that's 12 every day - and objective people would find the idea that there would not be rubbish in there is silly.   But I am not a researcher so I am allowed plain language researchers are not.  

    Cataloging bad studies would unfairly single out some (while missing others that might be worse, since we are talking about 4400) and take us down a pointless rabbit hole, when I have said many times open access is a key aspect of Science 2.0, so I am not going to discourage it.
    "...since there is basically no peer review to bog anything down at PLoS One." [citation needed]

    Where did that statement come from? Would you care to cite your source? Review at PLoS ONE is rigorous, objective, and fair. Do you have an example to the contrary?

    Look, I get that you work for PLoS One so you want to defend it but your insistence on painting it in a way that 4400 articles are published and they are all 'rigorously' peer-reviewed is not going to stand up to any objective use of logic much less practice.    Going to a site with an audience of 1 million readers and defying them to find rubbish in PLoS One is not going to end well for PLoS and I am too big a fan of open access publishing to encourage that.  You should not encourage it either.    Nature will be happy but not many others.
    Each submission will be assessed by a member of the PLoS ONE Editorial Board before publication. This pre-publication peer review will concentrate on technical rather than subjective concerns and may involve discussion with other members of the Editorial Board and/or the solicitation of formal reports from independent referees.
    The only 'rigorous' peer review under criteria for publication is 
    The PLoS ONE board of Academic Editors, and any invited external peer reviewers, will evaluate submissions against these criteria.
    'Any invited' can be none.  Submitters can also specifically block out reviewers, regardless of whether they are qualified or not.

    And further evidence of disinterest in peer review, they specifically engage in a cultural agenda by stipulating
    PLoS Medicine, PLoS Biology and PLoS ONE do not consider for publication papers where any of the research costs or authors' salaries have been funded, in whole or in part, by a tobacco company.
    This implies that anyone funded by a tobacco company is unethical and dishonest, so the content is not being judged on its merits in an objective fashion by peer review, but rather excluded in advance.
    It's not clear to me why the tobacco issue is being brought up, but the PLoS policy is consistent with that of several other "traditional" journals (such as those published by the American Thoracic Society.) Specifically, the position regarding tobacco industry-sponsored work of another of the PLoS journals (PLoS Medicine) was recently explained quite cogently here:

    The criteria for publication in PLoS ONE are clear:

    To be accepted for publication in PLoS ONE, research articles must satisfy the following criteria:

    1. The study presents the results of primary scientific research.
    2. Results reported have not been published elsewhere.
    3. Experiments, statistics, and other analyses are performed to a high technical standard and are described in sufficient detail.
    4. Conclusions are presented in an appropriate fashion and are supported by the data.
    5. The article is presented in an intelligible fashion and is written in standard English.
    6. The research meets all applicable standards for the ethics of experimentation and research integrity.
    7. The article adheres to appropriate reporting guidelines (e.g. CONSORT, MIAME, STROBE, EQUATOR) and community standards for data availability.

    The question is not whether there is work in PLoS ONE that you would call "rubbish." I'm sure that there is. Likewise, the example that you cite, Nature, has published its share of regrettable science. As one example, you could ask them how that "memory of water" article ( has held up over time.

    "'Any invited' can be none."
    Yes, just like at nearly every other journal I've encountered, editorial decisions are possible at PLoS ONE.

    "Submitters can also specifically block out reviewers, regardless of whether they are qualified or not."
    Yes, just like at nearly every other journal I've encountered, an author can request that certain individuals not review their work.

    Hank, you're wrong. The work published in PLoS ONE is peer reviewed. Rigorously. Does that mean that mistakes never occur? No. I'm sure that things that shouldn't slip through occasionally do -- just like at all of those other journals.

    I think I'm done on this thread.

    It's not clear to me why the tobacco issue is being brought up
    No matter how qualified the scientist or how good the research, any relationship to tobacco disqualifies it.   Yet you believe your relationship to PLoS makes no difference in your defense of PLoS One.

    I don't think it does either, of course, but I feel that funding or pro bono source does not disqualify anyone and judge people and their work on merit rather than a cultural agenda.  

    On the rest of this, basically you are arguing because this is not a puff piece about PLoS One, a publication you clearly like and respect.   And that was my point in the article - the success of PLoS One is in defiance of what traditional publishers say is possible because it couples enthusiasm (which billion dollar companies cannot muster) with an impact factor that is worthwhile.   I am the originator of Science 2.0 and open access is a key part of it so I would sooner cut off my arm than disparage open access - and Mike Eisen knows me well enough to cut it off for me if that were the case.   But PLoS One cannot get a free pass just because I like the co-founder, the concept and PLoS Biology.

    Citing examples where Nature has also produced rubbish is not a defense of PLoS One, instead it is saying that PLoS is the same as the institutions it seeks to replace.   And I know that is not the point you should want to make.  The rest of your argument doesn't refute anything I say about a lack of peer review.  If you can contend Nature or Science gives articles a free pass without any peer review, that is going to blow up the industry.  And I look forward to that.
    "If I have a study and the only barrier to publication is $1350, that is meaningless compared to $200,000 in funding or more."

    I wonder how many researchers have that kind of funding in these tight economic times?

    I may be wrong on this part (recently) but it was once a time that certain grants had a publishing requirement built in, so basically the money was put aside to produce it.    Some researchers pay to 'own' it, for example, by paying Nature or whoever (a lot more).   

    PLoS uses a generous creative commons so it is free to read and I assume it is not abused. 
    Science is such a incestuous cesspool it's ridiculous. I've seen crap papers get into Nature/PNAS and other "quality" journals because the authors come from a prestigious pedigree. Secondly, working in my current lab has shown me it's not really about the quality of work but how you use key words. For example, putting Vgamma2Vdelta2 T-cells, malaria, AIDS, macaque model into the title/abstract can easily push a paper from a 'low-tier' journal into one of those prestigious journals. Finally, the bigger and better your claims and speculations are, the better your chances at getting published in the high-tier journals.

    My mentor has basically told me "it's not how good your work is, but rather how much better you are at tricking the reviewers that your work is groundbreaking and important." Publishing in science is a giant game. If you play the game right, you can get anywhere you want. For someone like me, who abhors politics, ass-kissing etc etc, it's enough to drive me away from academics. The last thing I want to do is to be a Post-Doc, let alone a faculty researcher.

    Anyways, that's my 2 cents. I'm a 4th year PhD student at a fairly prestigious university studying malaria, AIDS and T-cell development,

    The last thing I want to do is to be a Post-Doc, let alone a faculty researcher.
    Since you can make more money in the private sector and not have the publish or perish concerns, it may be surprising to us that anyone is in academia.   But we have to be glad they are.

    I started this site and academics were incredibly gracious about writing, despite any semblance of a business model in the beginning.    Without academics, there is no way a site like this could have been successful without raising millions of dollars in advance.

    The notion that only academia does basic research is a circular reasoning one made by people who have never been in the private sector, but for the cost academia is the best value around.  The US has <6% of the world population and produces 32% of the science, so the system works.

    yea, I agree. Academia works, it's just that [b]I[/b] don't like how it works.

    "They accept 7 out of 10 submissions and, if you have read some of the stuff that gets printed, you have to wonder what the 30% rejections must look like."

    The high acceptance rate of PLoS ONE reflects its focus on technical accuracy rather than on perceived impact. ONE's publication criteria are clear, objective, and available to all. If an author submits a paper that meets these criteria, it will be accepted without regard to an editor's (or reviewer's) idea of whether it meets some subjective standard. I have found the review process at ONE to be rigorous and fair and am proud to be part of it -- as an editor, reviewer, and author. Perhaps Hank could be specific about what it is that has been published in ONE that he thinks didn't meet these criteria?

    We publish well-done science as judged by rigorous peer review and strongly encourage discussion and evaluation after publication. What is so scary about that?

    Disclaimer: I am the Section Editor for Infectious Diseases at PLoS ONE. These opinions are mine and do not represent any official stance of PLoS.

    You know what the problem is - the propaganda team PLoS One has hired for taking care of bloggers like you. They are shameless nuts and they are the one who diluted the brand value of PLoS. The missing link in the story is the quality of editors PLoS One has hired, which was intentionally kept as low barrier. So How they recruit editors for PLoS One? You blog something positive about PLoS One and with some average credentials in your hand you will become a section editor (like the one you see in you comment section). So I doubt the quality of editorial board PLoS One has recruited for its cause aka Open Access. Also they keep hitting anyone who dare to write anything negative about the PLoS One, You are lucky enough that you got a blog with audience where you can express your outrage against these pathetic OS parasites. There is no doubt that PLoS One is money making machine and thats why there is so much noise. What's with All This Peer-Review Stuff Anyway? Never heard before from PLoS One editors.

    I don't think it is as organized as wanting to eliminate anyone - obviously I am a fan of open access and there is no one out there big enough to eliminate us - but small movements like open access under fire from organized multi-million and -billion dollar companies (and a powerful Democrat that gets campaign contributions from them) need zealots, and sometimes zealots misfire.   

    I did see some negative comments about me (and this article) elsewhere but that's the risk of writing on the Internet.  I tried to respond but it's pointless and I don't really understand the objection, other than they don't like that I remain skeptical 4400 articles can have 'rigorous' peer review.  My article is overwhelmingly positive about the impact of PLoS and PLoS One but they were already worked up over another article (I have linked to it now in Related Articles below for context) so that may be part of it.

    As long as PLoS is open access, we are their friend, regardless of whether or not a paid marketing person there throws some insults at us.
    Hello Hank,

    I'm a freelancer for PLoS ONE. This is my own view: I am not speaking for PLoS.

    To reiterate what Adam Ratner was saying about peer review, PLoS ONE is definitely peer reviewed. If there is "basically no peer review to bog anything down at PLoS One", I suppose I was hallucinating yesterday when I made peer reviewer suggestions for some submissions! Also see this list of PLoS ONE's reviewers up until December 2008:

    I was among those who were skeptical of PLoS ONE when it launched, but they responded to the early criticism. I believe that most of the cases where Academic Editors don't consult peer reviewers are where they are rejecting articles; sometimes the flaws are so obvious that they don't need anyone else to rubber stamp that decision. Certainly some articles will be less than perfect, but this is the case with all journals and it is something PLoS ONE works hard to avoid. Submissions that appear to be by cranks, have ethical problems, or are otherwise clearly problematic are usually weeded out early by the PLoS staff, the section editors, or the academic editors.

    And yes, anything funded by tobacco companies is refused. I don't know why you're upset about that in particular, it's not a "cultural agenda". Tobacco only harms health and tobacco companies often distort their research. See for example this work by Lisa Bero: The full justification is in a PLoS Medicine editorial, in which they point out that the American Thoracic Society has the same policy on their journals:

    "Going to a site with an audience of 1 million readers and defying them to find rubbish in PLoS One is not going to end well for PLoS". I don't see why this would end badly for PLoS, particularly as PLoS ONE was set up with the explicit intention of receiving views from readers. Please do find articles you have criticisms of and make comments on the articles or blog about it. If you stay silent, how will the editors and authors ever know that there was a problem?

    A common implication is that PLoS ONE is a money-making article mill, but authors can request a no-questions-asked waiver of the publication charge and the academic editors aren't paid by PLoS, so they have no monetary incentive to accept articles: this would be a strange set-up for a journal only interested in profit (and refusing to publish work by tobacco companies would be equally peculiar).

    Regarding the 70% acceptance rate, this isn't particularly high for a journal that has no selectivity on interest level. Many submissions to PLoS ONE will often have gone through review at other journals, including other PLoS journals, and wise authors will revise their manuscript based on the criticisms they've received before (especially as the same reviewers might see it again). Isn't it depressing enough that even when only assessing basic scientific soundness and coherence, 30% of submissions are still rejected? And peer review is not merely an accept/reject filter: significant changes will have been made to most of those accepted manuscripts.

    You state that "they are pretty darn proud of impact factor now". I don't know where you got this from; the only interest in the Impact Factor is how it affects submissions (no prizes for guessing how). PLoS does not promote the IFs on its sites.


    Have to correct the record here WRT "no peer review at PLoS One". We just published a paper there that describes a screening methodology, and it was most definitely peer reviewed. We spent a couple of months addressing reviewer concerns. Really, it's just false that there is no peer review - only that "impact" of the research is not really to be considered. There's a good case that this is best left to the marketplace of ideas, hence PLoS One's remarkably high impact factor.

    The argument was whether or not they were all peer reviewed, as used to be the claim.  Now, press releases from PLoS One only claim the more accurate 'open access journal' and they leave out the peer-review part.  So sure, one or dozens may have been peer reviewed, that was never in dispute - but 7,000 of them are not. which is what zealots claim.

    We love open access but loving something does not allow for letting marketing fogginess make erroneous claims that hurt the entire field.
    Please check the home page.
    PLoS ONE is an interactive open-access journal for the communication of all peer-reviewed scientific and medical research.
    All of the biology articles I read were peer-reviewed. Needless to say, there are number of publications from various Nobel laureates.

    Your claim is that PLoS One peer reviews 7,000 articles per year?  I like PLoS, I support open access, but that is a blatant falsehood and if they are claiming it, the science community should demand an investigation.  No one would let such a ridiculous marketing claim slide if it were a peer reviewed journal.
    I having nothing with PLOS ONE. However, I would encourage you to read this post

    It's a magnificent logistical achievement.  Just PLoS One were able to manage 200 quality peer reviews every day in 2011, including Christmas.  With under a dozen people.  It is the most productive company in the world and should be taught in business classes for the next century.

    Like I said, I have no issue with open access and am a big fan of PLoS journals.  But Richard Poynder knows more about open access than anyone and consistently laments that PLoS One over-charges, under-reviews and over-hypes. But if PLoS says they don't, well, I guess that is good enough.

    I wonder which team of quality reviewers gave this article the big thumbs up.
    Thanks for the article! That is actually quite an interesting theory. Do you just like attacking anything discussing theology even if it brings up what might have created the theology in the first place? So that I can save some time in my review of it... where do you find the technical errors?

    Personally I think they should let tobacco companies publish. Just because someone writes up something based on solid technical grounds doesn't mean it makes good common sense. It does seem a bit ridiculous to not let someone defend themselves even if no one believes them. I guess the problem is that so few people choose to use the common sense given them and follow the crowd like lemmings.

    As far as PLoS One, I plan to publish there so that I can limit the amount of political nonsense associated with dealing with other journals. I spent six months getting professors to agree about what is the prefrontal cortex because Neuron and Cell continue to publish articles using descriptions which stopped being in use 50 years ago and are now quite incorrect based on cytoarchitecture.

    Congratulations, let us know when it is posted.  We have articles on hundreds of PLoS One studies here, they publish a lot of fine work. My contention, which people continually misconstrue even though PLoS does not claim to peer review every study, is that anyone who thinks PLoS One articles are all peer reviewed has some damage in their prefrontal cortex.  PLoS started off because publishing was too corporate and now the contention seems to be, 'it is a 10 million dollar corporation, they must be right.'  Which is ironic.
    Hank, you really don't get it... PLoS ONE DOES NOT REVIEW PAPERS. Scientists (unrelated to PLoS) do. PLoS just handles the mechanics of it. That's why it's so productive.

    I've published at PLoS ONE twice, and reviewers were every bit as careful as elsewhere. Why shouldn't they be??? They get paid the same at PLoS as they do elsewhere (nothing).

    Hank, have you ever published at PLoS ONE? No? Than think twice before criticizing the reviews until you know something about them.

    Have you reviewed for PLoS ONE? If yes, from your comments, I have to assume you did a worse job than you did for other journals. Don't you know that's unethical? Why would you do it? just to prove a point that a bad article can pass through reviews? We already learned that from traditional publications...

    Because this acrimony gets dredged up again and again, here is what the publisher of PLoS One says about PLoS One:
     PLoS ONE was a radical concept when proposed by the PLoS Founders – a journal which would judge submissions only on scientific and methodological soundness, leaving any subjective determinations of impact, scope, or relevance to the post-publication phase.
    In other words, it is not peer reviewed, it is looked at by an editor.  It's a quality publication, as I have said all along, but PLoS cult members keep insisting it is peer reviewing what will be 24,000 articles this year - no one is peer reviewing 60 articles per day.  No one at other open access journals, including the many of the 25,000 open access journals that are also not peer reviewed, has any problem with not being called peer reviewed.  It is PLoS Biology envy, I think.
    @ Hank and his likes,

    I wish your skepticism about PLoSONE's peer review process was well founded. The truth of the matter is that, the peer review process is as rigorous as you would find anywhere, don't quote phrases from PLoSONE website that would support your preformed notions. Academic editors (AE) that are more than few hundreds are the ones handling the academic utility, futility and relevance of the submissions! The PLoS staff simply makes initial determination of suitability/eligibility of the article before it is passed over to various Academic editors for handling. Again over here each manuscript once cleared through the initial barrier is sent to numerous AEs at the same time and who so ever accepts it to handle then goes through the manuscript to make the initial decision about inviting peer reviewers!! Being one of the AEs, I spend ample time before I make determination whether or not the manuscript is worth the peer review. More interestingly, names of AE's handling the manuscript is printed with the manuscript so who would want to be tagged a 'dud' by accepting a worthless manuscript- esp. when the names would be attached forever with that publication? How many journals do that- quite a few only!!
    With 300+ AEs why can't you have 500+ articles in a year? Each one is handling no more than say 15- 20 articles/year won't you be able to handle 20 articles if you were an academic editor for a journal?? I would not question your efficiency or time management even if you say NO here!
    Other journals on the other hand are run by minuscule editorial boards of no more than 30-40 editors that would definitely get exhausted after say 600 manuscripts at the max.
    I did not mean to play simple maths here but may be you don't have insight into the statistics at PLoSONE!!
    Have a more broader outlook towards flourishing journal. Also I would want you to have a look at average time between submission and acceptance- would be somewhat surrogate for if the peer review was decent enough or no. Please don;t come up with a reasoning that the journal people might be egging on the manuscript just like that for sham- because if they were simply to make money they would want very high turnover and minimal peer review time.
    Hope you agree

    It's not skepticism, I love open access, I have had PLoS Biology as a badge on my homepage since Science 2.0 started in 2006.  But if Nature or Cell or Science were playing with the truth like this, the open access community would be all over it.  Honesty is honesty, they are not doing the same peer review and primarily they do not claim to be (or they did not - their editor left so what they claim now may be different), they only said an editor looked at it for technical merit.  That is not peer review.  And peer review after publication, which they also claim now, is not what people call peer review either, it is redefining something that has existed for 150 years.
    Did you at all read what I wrote above about the peer review? It nothing like post publication peer review!! What makes you believe that?
    But anyways you have preformed notions that you would likely not change in near future if at all! Yeah the editor did leave recently so what? Its better than some editor been kicked out like what has happened to the ace medicine journals- hope you don't need citations for those incidents?

    Sure, I read it.  It said that this one publication had achieved a miracle unheard of in 150 years of journal publication - being able to rigorously peer review as many articles each day as regular journals can only do in a month.

    Obviously you believe that.  Okay by me.  But their own press releases do not make that claim, they simply call it 'an open access journal'.  
    Just a correction to the number of AEs above it is close to 1000!!!! so does 10000+ articles/year appear more plausible to you?

    Right, when this was written it was only a mind-boggling 200 per day - already impossible.