It was about month three when I first took a look at a numerical model in a journal paper. It was simplistic rubbish, full of assumptions. I came from companies making numerical models for the private sector, where a mistake in a physics answer was going to cost a company millions. And your company would be replaced if your tool created that mistake. Academic scientists were making models that were going to impact tens of billions of dollars in taxpayer money and no one seemed to mind simplistic Monte Carlo crap would get them fired in the private sector. (2)
Thankfully, models have gotten a lot better. There is a trickle-down effect that started, probably in particle physics, and has worked its way into numerous fields where "statistics" matter. Yes, there are still shoddy papers claiming to find effects from chemicals or food using data dredging and epidemiology, but in the physical and life sciences models are a lot more robust.
Not those made by everyone, though, and especially not if you are an activist pretending to do science. Which brings us to Mark Jacobson, a computer scientist at Stanford who got published in PNAS during that time when they were publishing rubbish like more people die during hurricanes with females names because of sexism, and they let Facebook manipulate news for readers as a live experiment without having any ethics concerns. (3)
So I was not surprised PNAS also published a paper which claimed America could switch to 100 percent renewables - wind, water and solar - with no issue at all. And his model showed it. I knew it was wrong without having to read it.
The climate activism community cheered. So much for claims that natural gas was a necessary bridge. So much for James Hansen believing nuclear or clean coal should be part of the mix. Renewables were ready right now, a Stanford global warming insider said so. The rich environmental groups - NRDC, Club Sierra, Greenpeace - cheered his work.
I'm a statistician. My motto is 'I haven't read your paper yet but I'm virtually certain your methods are flawed&your results are wrong.'— Stephen John Senn (@stephensenn) April 9, 2014
Ecowatch remains faithful. Is Mark Jacobson the Robert F. Kennedy, Jr. of computer science?
There was just one problem. His model was not very good. He lacked basic command of statistics.
A mathematician at National Oceanic and Atmospheric Administration and colleagues took their time doing a rebuttal. It went through exhaustive peer review, and eventually PNAS published it even though it made their original paper look weak. Clack et al. were really nice about their criticisms. Jacobson is on their side when it comes to renewable energy and cutting emissions, after all.
But they knew bad claims sowed distrust by the public in the work of everyone in science. So they simply wrote he and his coauthors had made an "error" even though no legitimate modeler could have done so. Jacobson responded by yelling that they were frauds, threatening legal action and sure enough he has now filed a lawsuit against PNAS for publishing it, claiming that because he is paid to make models, a paper showing his errors damages his reputation.
He felt damage? Cities had been taking action to switch, believing his work was valid. They didn't have the expertise to know his “no load loss” assertion was impossible and his confidence interval was a fever dream of optimism. Jacobson modeled the whole country. Who would do such a thing? In 2011's Science Left Behind with Dr. Alex Berezow (who has his own take on the fiasco here) I noted it would require transmission lines equivalent to all of the paved roads in America to make alternative energy work, and environmentalists had already blocked every large solar installation. How could Jacobson not know that and use a grid integration model that ignores simple facts of economics and politics?
We may never know. Politically sympathetic outlets like Daily Kos turned on him due to his unhinged behavior, as did many others. Only the kooky site Ecowatch has let him spout his conspiracy nonsense. Robert F. Kennedy, Jr., an anti-vaccine conspiracy theorist, is on their Board after all.
Will Jacobson get his $10 million? No. Will he get the retraction he demands? No. Will he get any settlement at all? No, I predict he will be dismissed from court.
And his reputation, suspect long before that paper, will be in tatters. Because you can't sue your way to competence.
(1) Okay, that part turned out to be true. While I defend all scientists from attacks by environmentalists, the plain truth is that if I needed corporate scientists to write, they would still be waiting for approval 11 years later. And their marketing group would want pre-approval on anything. And they will never, ever buy an ad, they will instead hire Ogilvy (insert your favorite agency there) for tens of millions of dollars who will assign some junior account executive to write me and ask if I have ever heard of Unilever (insert your favorite large company there) and because I write about science, would I like to cover their client for free?
Every company feels this pain. People want a great product for free. Before this began, SEED media created Scienceblogs.com, a valuable launching pad for science blogging, home to everyone from Carl Zimmer and Chris Mims to Chris Mooney and a lot more. That is now gone though I offered to pay the costs for the servers indefinitely to keep it running. Likewise, the company behind Gothamist, which started after Science 2.0, just announced they are shutting down despite having 15 million pageviews, 2 million social media followers and a 500,000 strong newsletter. I will pay for their servers too.
(2) In the private sector, I can't recall how many times an academic told me their tool was faster and more accurate than Ansoft or Optimal. And it turned out to be command line only, or it only worked on their test suite of problems and any new model created was all over the place.
There is an old saying that goes "Everyone believes the data except the collector. No one believes the model except the modeler" and it remains wise to be skeptical.
(3) In 2014, I criticized PNAS in the Wall Street Journal for publishing a paper by Tyrone Hayes (who claimed an herbicide was turning boy frogs into girl frogs) which had been hand-walked past peer review by a friend of his who is an NAS member. No data included, no skepticism by an editor, it just went out and then appeared in the New York Times (naturally) as yet another evil company ruining Gaia. When the EPA decided to investigate the herbicide, Hayes claimed all of EPA was conspiring against him with the manufacturer of the herbicide, Syngenta, and so he was refusing to turn his data over. The paper was never retracted. It was also never used by EPA, which exonerated the compound in a special panel and then re-registered it.
After my article, PNAS changed their policy. They didn't sue me or the Wall Street Journal for damaging their reputation. They set out to fix it.