Reich’s book is apparently very well researched (I take Blume’s word for it, since material physics is not my field), but she draws exactly the wrong conclusion from the case study she so thoroughly investigated.
The biggest fraud that shook the scientific world wasn’t really the biggest (I would argue that Piltdown man, the fake missing link “discovered” in England in 1912 was much bigger), nor did it really shake the scientific world that much (I am a scientist, and I first read about this case through Blume’s book review). Still, it is an intriguing story in which a postdoc at Bell Laboratories, Jan Hendrik Schön, fabricated data concerning the properties of certain types of plastic (hence the title of Reich’s book) and got away with publishing papers in prestigious journals, including Nature itself, Science and several journals sponsored by the American Physical Society.
Jan Hendrik Schön. Credit: Bell Labs
Reich, who is a journalist with a background in both science and philosophy, takes the moral of the story to be that “It seems like little more than blind faith to insist that all activity carried out in the name of science will always be self-correcting.” Well, yes, but as Blume points out, no serious scientist (or philosopher, for that matter) actually believes that.
The peer review process that is at the core of science’s ability for self-correction consists of two phases. The first one is the rather institutionalized practice that every editor of a scientific (or other scholarly) journal follows: when an author submits a paper for publication, the editor reads it and sends it out to a minimum of two reviewers who are chosen because they are experts in the particular field to which the paper is pertinent. The reviewers (who are anonymous to the author) send more or less detailed comments to the editor who then makes a judgment as to the suitability of the paper for publication.
But it is the second, more informal and open-ended, component of peer review that is really crucial. The first step relies on the expert advice of a small number of people (the editor and the reviewers), and it is subjected to conflict of interests (maybe one of the reviewers knows the author and dislikes her on personal grounds; or they have been in direct competition for grants, so that the reviewer has an interest in keeping the author from publishing). But after the paper is out everyone in the scientific community can read it, cite it (or not), and criticize it at meetings or in print.
This second part of the peer review process is what really matters, because fraudulent papers in the long run end up in one of two categories: they are either completely forgotten because they didn’t really address an important topic at all (in which case the author gets away with the fraud, but there is no lasting damage to the scientific enterprise), or they are discovered because other people tried to replicate or build on the results and failed.
This is exactly what happened with Schön’s papers: they slipped through the first step of the review process, but spectacularly failed to pass muster during the second step. The failure of formal peer review in cases of fraud is not really that surprising, since the system is based on the assumption that authors are not just making stuff up.
The aim of peer review is to reject or correct papers that are deficient in their methodology, data analysis, or in how well the author’s conclusions are substantiated by the empirical findings. Making stuff up doesn’t fall into any of those categories. Indeed, the real culprits there are Schön’s senior co-authors, who should have paid more attention to what they were signing off on, especially considering that some of the claims made by Schön were groundbreaking (like the discovery of superconductivity in plastic!).
Yet, contrary to Reich’s conclusion, these stories actually validate the scientific enterprise as particularly effective at uncovering fraud, when it really matters (i.e., for contributions that make a difference to science, as opposed to just adding a line to the c.v. of an individual researcher). The scientific peer review process (both formal and particularly informal) therefore is a bit like what Winston Churchill said of democracy: it is far from being a perfect system, but it is a heck of a lot better than any of the alternatives.
Whatever works, as Woody Allen would say.
Comments