I suppose the first is mostly correct, however Opera (for instance) published without the consent of all colaborators and the result was questionable.

The second one: My statistics are derived from a sample of one, so be sceptical: The paper about the impossible rotational symetry in quasicristals got published some 20 years ago although that current thinking at the time was that this is fundamentally impossible and completely crackpot idea. So I do not think that there is so much danger for paradigm shift research NOT being published because it is incorectly considered as crackpot. Moreover, even today crackpot papers can get published.

A 4th year undergrad in ENS Lyon, France.

I think peer review, flawed though it is, is absolutely essential to our field, even for experimental papers, which
have had a vigorous internal review already. There is a useful psychological effect of knowing that you're
going to try to get it into a peer reviewed journal, and a handy "time-stamp" on your paper: you have to get
it into fine shape for then. I fear that losing peer review would open the flood gates to crap papers on the arxiv:
some people would write any old crap, and signal to noise ratio would decrease substantially.

Often, having criticism from strict reviewers has made my papers better. Also, Experimental papers' review process
often ends up with slightly strange decisions as the result of horse-trading and compromise among the review boards.
A reviewer can cut through that, requesting what is scientifically necessary.

I was going to say that these are absolutely wrong but then I read them again and realised that they are absolutley right. What would be wrong is the possible implications that someone might read into them. For example, the obvious implication of quote 1 would be that papers written by large collaborations do not need to be submitted for peer review. The quote does not say it yet the responses already given suggest that people are assuming that this is the conslusion. It would be wrong and unnecessary because the only way to make the distinction between papers that need further review and those that dont is to ask a reviewer, which is what already happens.

Quote 2 is also true but the implications are less clear. It just shows that the current system is broken. How do SAS propose to fix it? Without knowing that it is hard to support the statement. It may be a trick where they then propose a solution and claim that people support it just because they agree with the quote. Probably I am being too cynical but the real point is that it is the conclusions about what should be done that they need to consult people about.

I have PhD in physics

1)
speaking of lhc experiments:
Internal review are very hard and so far we saw very few papers rejected by the referees (usally because "search "analysis were based on subset of luminosity). So I think is something that makes sense.
As somebody already said, ATLAS members should review CMS papers and viceversa. Find competent referrees will be soon difficult. (Tevatron-only scientisit they will probably disappear)

2)
one should not judge from his point of view (bias)
if the method is scientifically correct and robust and there is some kind of scientific interest
there is no reason why one paper should be rejected. (maybe one may wants to adjust the target. Not all papers are Nature-worth)
Even if the topic is very "bizare"

cheers,
postdoc in one of lhc experiments

1)
I think this is mostly true, for HEP.
What I would like to see is more of a group reviewing process. For instance if accounts on a website like arxiv were only given to verified Physicists and then users were allowed to rate and comment on papers in a blog-like format. I also think it is important to move away from anonymous peer review, if you believe something put your name on it.

2)
See 1). The above gives full disclosure, there would be no final 'accepted/rejected' stamp, but the ratings and comments could give you an indication of consensus.

Anon PhD student, with Masters in Physics.

Regarding the first point,

I'm a pheno postdoc so do not have direct experience of working in an experimental collaboration. However, I have now heard many stories form my experimental friends of how 'charismatic' experimental leaders can 'guide' the conclusions that an experiment draws. A group at the top often comes to the answer and any dissent can be effectively crushed through various political tactics.

I guess when I head the first story of this kind I was surprised. However, these get repeated from all experimental fields, LEP, SPS, Tevatron, astroparticle, dark matter detection... etc.

I'm not convinced that external review always provides an antidote but at the very least, it's an extra check.

Quote one assumes that large collaborations will always act in the best interest of science, and not just those of the lead scientist. Remember the story of the emperors new clothes? If the one person who sees that the result is not what the majority thinks it is they could suffer career consequences. (Even more so if the person is part of some minority group).

If the collaboration is large and run in an ethical way. If dissent from the majority will not equal being blacklisted then the first approach can work.

Quote two touches on a very real problem. All the big revolutions that solved the big problems came from people who were on the fringe. There is reason to doubt the works by Einstein or even Newton would make it past a modern style peer review. Einstein's work was published just because the editor of Annalen der Physik liked his ideas. Newton, we would all do well to remember, published his ideas in a book that was not peer reviewed.

If Philosophae Naturalis Principia Mathematica had been sent for a review before publication by notable scientist of his day it would have never been published. I mean "who is that upstart to dare introduce a new form of mathematics!" That's what may well have been said. Newton would have lost his tenure and the Lucasain professorship for lack of publications. Then went off to live in the woods and write very angry letters.

What could replace peer review?

The model that arxiv and Vixra demonstrate could replace peer review. Replace Arxiv's "endorsements" with a system that would limit publication to people with at least the equivalent of a Bachelors degree in science. Then allow open, public comments on the paper as is done at Vixra. The interested public can then read the comments and the paper and make up their own minds about weather it is worth further investigation or not. This solves what is, in my opinion, the big problem with peer review, the utter lack of transparency.

In my opinion, the first quote has some truth to it - it's hard for an external referee to do much checking of the correctness of a result, especially if it's published in a 4-page long PRL! On the other hand, external referees can have a role to play in assessing the presentation of the result - is it clear (to someone outside the experiment)? does it provide the necessary details for people "downstream" of the result (say, phenomenologists using the data for fits)?

Retired Ph.D. Scientist/Research Engineer
1) External review brings in a set of fresh, out of the group think questions that sharpens and clarifies any paper.
It is analogous to a court case where the facts are the same for both the defense and the prosecution. Defense takes those facts and presents a case that is the most logical and advantageous to the defendant. Internal reviews are similar and have a latent bias to get the paper out but they do make sure that the logic and the arguments are coherent. External reviews however are like the prosecution that takes the facts of the case and makes an argument against the defendant. Good external reviews are inherently adversarial and if you could clear it, it makes the paper really strong.
2)Crazy ideas such as that the earth is round, rotates, and moves are the stuff that develops and pushes out the boundary of science. Any new idea should:
a ) explain the existing data and observations, and
b) should provide extrapolation that could be observed.
If those two tenets are strictly followed it can be safely taken as a scientific breakthrough.