The word "all" is key - Science 2.0 readers surely agree that some academic research is crap - and I'm particularly sensitive to charges like the one in the Chronicle because a different loon, an influential friend of the Governor of Texas, managed to get like-minded regents appointed to my alma mater, UT-Austin. The new regents, with continued prompting from the loon, are trying to separate research faculty from teaching faculty, grade faculty solely on student evaluations (you know, those anonymous forms collected on the last day of class) and on the numbers of students in their classes. Whether the students show up or not, I guess. And whether they learn anything or not.
So it is worth noting, as I finish several days on a review panel that gave away a couple million bucks of taxpayers' money to economics and management researchers, the care and responsibility that the panel exercised to fund only proposals that would advance knowledge, boost the national economy, and maintain the nation's capacity to innovate.
It was sad that, in order to stay under budget and to honor the funding agency's wish not to open the door to appeals, we had to refrain from praising proposals that we all considered really good but could not fund. We had to emphasize negative comments when writing feedback to these proposers whose ideas were just under the cutoff point. I hope this will not discourage good scholars from trying again next year, or cause proposers (who know their proposals were pretty good) to lose faith in the process.
We tried to minimize these side-effects by funding the largest possible number of excellent proposals, and then ruthlessly slashing the budgets of those we funded. I mean, we took a pick-axe to those budgets. As long as there was enough money to get the lab work done, we figured, the researchers could report results at one conference instead of two, and could nurse their old computers for another year instead of buying new ones.
Not to say this was a perfect process. Panel members showed diverse personal biases about the weights that should be given to supporting students, to the sound theoretical grounding (versus the creativity) of hypotheses, to journal publication versus technology transfer, or to supporting early-career scholars versus funding scientists who had shown good performance on prior grants. We also funded ideas that might or might not work. (Hey, it's research. And I can't tell you what lines of research we funded, because it might give clues to the identity of reviewers.) We had to toss back a few intriguing research ideas that were couched in very poorly written proposals.
But it was a careful deliberation, and I believe it was the best that could be done.
Now some math: Travel plus five days of lodging and meals for eleven senior reviewers, plus a very modest honorarium for each of us, in order to evaluate 152 proposals. This is already a material fraction of (but was paid by the agency in addition to) the total pot of $2M. In other words, the process was almost too careful in its pursuit of quality, responsibility, and accountability.
Let no one say we wasted your tax dollars.