Georg, over at Lattice Points noted a piece about open science in Physics World:

The adoption and growth of scientific journals has created a body of shared knowledge for our civilization, a collective long-term memory that is the basis for much of human progress. This system has changed surprisingly little in the last 300 years. Today, the Internet offers us the first major opportunity to improve this collective long-term memory, and to create a collective short-term working memory — a conversational commons for the rapid collaborative development of ideas.

Physics World offers an example of the power of online collaboration, but I think the example backfires a little: the lone genius still beats the total knowledge of the crowd:

Another example of the power of online collaboration comes from chess. In 1999, Garry Kasparov, the greatest chess player of all time, played against the “World Team” — a single team consisting of thousands of chess players, many rank amateurs, which decided on their moves by vote. Kasparov won, but instead of the easy victory he expected, he got the most challenging game of his career, which he called “the greatest game in the history of chess”.

I also take issue with the article's claim that the main disincentives to scientists' participation in Wikipedia and science paper comment sites are cultural - that there are now rewards for commenting:

The problem that all these sites have is that while thoughtful commentary on scientific papers is certainly useful for other scientists, there are few incentives for people to write such comments. Why write a comment when you could be doing something more “useful”, like writing a paper or a grant proposal? Furthermore, if you publicly criticize someone’s paper, then there is a chance that the person may be an anonymous referee in a position to scuttle your next paper or grant application.

There are no rewards for Amazon reviews either, and yet you get many comments there. The bigger problem is time management: the internet is a notorious time drain. To use it successfully in your professional life requires enough self-control to be highly selective. The last thing I want to do during my workday is get drawn into a long online argument over at PLoS ONE that I keep checking obsessively every 30 minutes.

To write an intelligent comment on a technical paper takes some time. Commenting on each paper I read in a week would take up a non-trivial chunk of my time - which means I would have to be highly selective in the papers I comment on. But online commenting is supposed to be spontaneous, the instant thoughts of the community on some PLoS One posting. Here is the tension: online discussion is widely thought of as spontaneous, yet to have a substantive discussion at a very technical level, substantial enough to make it worth your work time, requires a level of effort that makes sponteneity nearly impossible.

Thus, Nature's failed experiment with comments on papers under peer-review - quick comments were insubstantial, and more substantial comments weren't worth the time they took to write.

I'm not talking about blogging and writing science for the public; this is about online tools that could be helpful to scientists in their professional work - tools that have the "potential to speed up the rate of scientific discovery," according to the piece.

Online databases, including open access to data and preprints have proven useful; group comments on science papers have yet to prove their value. If scientists find some new tool that will really speed up science, they'll use it - like OpenWetWare, which has an extremely useful section on experimental protocols (along with many other great tools).

The real key to open science is to create strong communities. These online communities will have to compete with the communities working scientists are already involved in. We swap ideas and expertise, get comments on our pre-print papers, and help with experimental protocols already, from colleagues that we meet in our departments, through seminars, and at conferences. If I have a question that I can't solve, I can fire off an email to any one of dozens of people I know personally, who have expertise that I lack.

Open online science has to prove that it's worth the effort to participate - that it provides something of value not available in the informal social networks scientists already participate in.