Banner
    Scientists, Ye Pedantic Jerks
    By Tommaso Dorigo | May 13th 2010 09:32 AM | 12 comments | Print | E-mail | Track Comments
    About Tommaso

    I am an experimental particle physicist working with the CMS experiment at CERN. In my spare time I play chess, abuse the piano, and aim my dobson...

    View Tommaso's Profile
    Academics pay a lot of attention to the quality of their writings. It is generally a point of pride to publish flawless documents, and this is felt in scientific disciplines just as much as in literary ones. If I told you how much time the members of a scientific experiment such as CDF at the Tevatron or CMS at CERN (the ones I work in) spend in the review of their articles before these are sent to the publishers, you would be startled. Actually, let me just do it, since I promised it earlier this week.

    The thing works like this: you, alone or with a small group of researchers, produce a scientific analysis and write a draft detailing your result. If you have gotten to that point, the analysis has already withstood close scrutiny as you presented it several times at the relevant internal "physics" meetings, receiving annoying questions all along. Among the questions, 20% were probably constructive, 30% were meaningless, 40% were pedantic, and a further 10% were asked just to make you feel inferior. But you got to the bottom of it, presented the analysis for pre-approval, gave a collaboration-wide approval talk, and feel happily accomplished.

    Yet, you have not accomplished anything yet: the publish-or-perish rule dictates that you publish a written article, lest all your efforts will amount to nothing. And publishing a written article is a whole different matter from converging to an approved result. In fact, at this point, you have to get past a internal review committee of three peers. These are called "godparents" at CDF -a benign name indicating people who try to help, or an ominous one alluding to mafia bosses, depending to how you perceive it; in CMS they are called more neutrally "ARC" for "analysis review committee".

    The internal reviewers scrutinize both your result (although it has already withstood extensive checks, while your godparents were on leave fishing) and the text you produce. On the latter, they advise on the quality of writing, they enforce certain conventions (from important choices such as the mathematical notation, to less meaningful ones such as the use of the serial "Harvard" comma in lists of items, to masturbation-level ones concerning the color of lines in a plot). Once they are ready, the paper is released for collaboration-wide review.

    You would think that at this point -once your internal reviewers are happy- the article is close to being made public. Wrong! Any of the 600 (CDF) or 2500 (CMS) collaborators is now entitled to have a free shot at you, sending a detailed list of requested rewordings and corrections. I have oftentimes received, or alas, sent, lists of comments much longer than the paper itself. Authors need to answer each and every single comment received, and make sure that the paper is changed accordingly; unless they oppose the offered change, in which case they need to explain the reason of their denial, convincing the internal reviewers that they are right and the offending comment is wrong.

    After the first reading, a second draft incorporating the changes is approved by the internal reviewers and again sent around, and there again all the collaborators have a free shot. More nasty comments and requests for corrections pour on you, a new iteration follows, a new placet by the internal reviewers is waited for. At the end, the collaboration takes a live final reading of the whole article, where again everything may be rewritten. Whether an improvement is made or not at this final stage, is a quite tough question.

    Then, finally, the article is sent to the scientific magazine, where external reviewers will have a chance to further bother you. But this last step feels like a caress after a slap in the face. While the internal review may have lasted months, this last step is usually limited just by the speed at which the magazine finds reviewers and twists their arm into sending a positive judgement back.

    Now, if you think that the above baroque, surreal, ridiculous procedure is crazy, you might be right. But unfortunately if an academic is jealous of the quality of his or her writing, you well imagine how hard it is to get an agreement between a thousand or two of them. So there is little else to do but follow the rules, if you want to publish.

    I will not discuss further whether this procedure makes sense, but I wish to just point out here one of its striking positive qualities. It is extremely democratic! As a collaborator, you sometimes play the good guy -the one who tries to publish and gets all the heat- and sometimes play the reviewer, or the peer who sends nasty comments and corrections just for the sake of it. And further, your comment as a PhD student may be worth just as much as that of a senior professor -nobody knows everybody else, so if they do not know you, they must take you seriously!

    A different question, however, is whether the iter could be sped up. We live in an age of competition: the old days when CDF was the only experiment capable of playing with 1800-GeV collisions is long gone, and now DZERO competes with CDF. ATLAS and CMS, on the other side of the Atlantic, are sharpening their knives in the attempt at being the first experiment to claim evidence for this or that new physics effect. Can the procedure be made shorter ? Of course, it can. And I believe that all the experiments are working toward making their review procedures faster, while keeping the highest standards.

    One final point I wish to make is the following: the time wasted in the internal reviews is huge. I believe that the scientific output of any of the large collaborations I mentioned might increase by at least 20% if the main authors of a scientific result were held responsible of their drafts, and collaboration-wide reviews were canceled. But then, of course, you might be seeing your name appear on the first page of a publication using "centER-of-mass energy" instead than the British-kosher "centRE". The horror, the horror.

    (Be sure to understand one thing: I am not blaming anybody nor anything in particular for the above baroque procedures, or for the annoying reviews. I myself am a master-masturbator, since I have sat in internal review committees for over a dozen publications, significantly annoying a couple hundred people in the process; what is more, I have been for the past two years a member of one special group in CDF, called "Spokespersons Publication Review Committee" which really looks for nuisances in each and every publication draft that CDF produces; and in two years as a SPRG member, I have annoyed basically all the remaining colleagues within CDF. But I am not catholic, so I may jolly well cast the first stone even if I am a big sinner!)

    Comments

    Hank
    You would think that at this point -once your internal reviewers are happy- the article is close to being made public. Wrong! Any of the 600 (CDF) or 2500 (CMS) collaborators is now entitled to have a free shot at you
    This had me thinking the corporate world looked a lot better but then ...
    I have sat in internal review committees for over a dozen publications, significantly annoying a couple hundred people in the process; what is more, I have been for the past two years a member of one special group in CDF, called "Spokespersons Publication Review Committee" which really looks for nuisances in each and every publication draft that CDF produces
    ... sounded kind of fun.  Thanks for the detailed look behind the Wizard's curtain.
    Thanks for the detailed look upon this baroque, surreal, ridiculous, yet democratic procedure, which I did not know before. It reminds me not a few of the aspects of the theatre of absurd. Or theater?

    This sounds like an extended HII (High Impact) Review process. It is just as baroque in any industry where some kind of review discipline is enforced by government rule or worse, the finance deartment. Medical Instruments is where I did it. There were only a couple dozen reviewers though, never several hundred. Plus the auditors of course ...

    Now I know why I chose Theory.

    Interesting!

    LauraHult
    A different question, however, is whether the iter could be sped up.
    Our lives are mostly controlled by computers now, except for the little things like flushing a toilet (and I hear the Japanese are working the kinks out of that project), so why not just submit the whole danged thing to some uber-reviewer computer?  No personalities or egos to deal with, just cold, hard logic - or as close to logic as programmers can get.  ;)
    dorigo
    Such a thing does not exist yet, Laura. Otherwise we'd just stop colliding protons to learn stuff, and ask it!

    Cheers,
    T.
    Hank
    There was a movie a while ago called "Hollow Man" wherein the soon-to-be invisible hero does just that - sits in front of his PC and waits for the tell-tale Hollywood green success sign and when it happened I was the only one who started laughing.
    Democracy sure has side-effects!
    But apart from infuriating grammar comments, what's in essence undesirable about the process involving whole collaborations? Silly questions will be answered easily, clever ones will have something to offer. If reviewing the way the journals do it has sparked so much debate, why isn't this alternative preferable?

    dorigo
    Eleni, the process does make sense, if you take a detached stand: if all you care about is the science output, and the correctness of the results, then it makes a LOT of sense. However, scientists are human beings, and sometimes you are asked to take too much s**t from your colleagues. Moreover, competitions demands that you are quick to produce your results, and long review processes of course are a problem in that case.

    Cheers,
    T.
    "However, scientists are human beings, and sometimes you are asked to take too much s**t from your colleagues."
    ...That could be felt in the subtext :)
    It'd be interesting to know if the two "small" lhc experiments are faster only because of the number of collaborators or not.

    Greetings Tommaso,

    somewhere in your old blogsite I read a sentence something like "to speculate about elementary particle masses", why their masses are what they are. Not having a better idea I decided to put my comment here.
    The fact of the matter is that Pitkänen can calculate the elementary particle masses using p-adic thermodynamics for, if I'm not mistaken, fermions, and predict the mass of Higgs-boson too. 89 or 129 GeVs. So why don't you –colleagues- just take a look at his work??

    by the way the 450 GeV top quark is in his view a scaled-up version of a top quark (scaling also involving p-adics), with masses coming as half octaves; his prediction for its mass is 480 GeVs, 6 per cent difference.

    thanq,
    yours sincerely
    gábor fischer