A recent review in the British Journal of Nutrition concluded that the nutritional quality and safety of organic food was higher than conventional food. Fruits, vegetables, and grains, organic versions were better in all ways than conventional farming, they determined.

Organic food had fewer pesticides, a much different result than other studies, and also had more important nutrients, also a much different result than other studies.

You always want to try and take each paper on its merits but when something is dramatically in defiance of the science consensus, you also have to look for other factors that might be involved. And this has some red flags. To start with, one of the designers of the new review is Charles Benbrook of Washington State University, a noted anti-GMO activist with a history of agenda-driven claims and poorly designed studies. That raises eyebrows outside the environmental community. Anti-science activists and sites like Mother Jones love his work, but he is an economist, not a biologist. Nothing wrong with economists talking about science, but if environmentalists and Mother Jones don't let economists overturn climate studies, why would they endorse it in biology? That's a mystery.

This new work is right in line with his 2013 paper which claimed organic milk had more omega-3 fatty acids than regular milk. Why is a study of pesticides done under EPA supervision disqualified by activist front groups if a pesticide company has to pay for it (required by federal law), but a study endorsing organic milk that this author paid to have published, that was funded by an organic milk company, remains free of concern about neutrality? Another puzzle. 

Ordinarily, talking about individuals rather than issues is a personal no-no, it is the kind of tactic environmental groups use, but when someone is engaged in chronic anti-science behavior, it becomes part of the issue. He wrote a blog about this new paper, which contains various logical fallacies, among them that other reviews which used many of the same studies found many of the same results. I should hope they did, because one of the reviews he lauds happens to be what he modeled this one after: "The literature search strategy and meta-analysis protocols used were based on those previously published by Brandt, et al."

Same method, just a wider net to include more papers. That can be problematic.

So what results did they find? That pesticide residues were 3-4X more likely in conventional foods than organic.  Now, that is not really a problem. If you bought vegetables and fruits anywhere at any time, you should wash them. There are a lot of reasons that a review with such a broad reach can find issues. 

"What do you bet they didn't control for soil, sunlight, storage, or shipping methods? And that just stuff that starts with S," Norm Benson asked on Twitter.

Well, they didn't, we know that by reading the paper. But they don't consider that the samples in the studies they selected may be causing odd results. Instead, they explain it away that organic farmers don't use synthetic pesticides.

Did this economist reinvent toxicology? Do toxic organic pesticides dissolve into rainbows when they are placed on a delivery truck? How can there so many fewer residues on organic food when it is known organic growers simply use different pesticides, not fewer?

We can thank meta-analysis for their result.

Image: Organic food will also make you prettier, if we do the proper unweighted random effects meta analysis. Credit: Shutterstock

In an unweighted random-effects meta-analysis, cows can whinny and sideways can be back and forth - if I include large outliers in the samples.

A total of eight different meta-analyses were undertaken. The protocols used for the standard weighted and unweighted meta-analyses were based on the methodologies described by Palupi et al.(29)and Brandt et al.(20), respectively.

They used a lot of papers, that is a good thing if there is actually a large body of knowledge and it is rigorous, but in even the most controversial toxicological issue, the EPA will end up disqualifying all but about a dozen papers due to lack of underlying data being included or methodological concern. In a review, they look at no data, of course, and 343 papers becomes the problem rather than the solution when the methodology is flawed.

Meta-analysis, as everyone with statistics knowledge knows, can boost the strength of systematic reviews when done properly but easily suffers from bias unless the researchers are truly interested in controlling eligibility criteria and methodological quality. Without controlled eligibility, it's easy to find any pattern you want. With Web of Knowledge search terms like 'organic' and 'biodynamic', it's really easy to skew the inclusion. Then they synthesized their dramatically different studies using a random effects model. 

In general, prospective studies are stronger than retrospective but a prospective study is not going to find organic food is more nutritious and that conventional food is toxic, so they ended up using claims by farmers about what they used as data. By using random effects, they are recognizing that the observed effects and sampling variability are very different, but then they take them all on face value anyway and average them out. Since the random effects are unweighted, credibility and confidence intervals take a dive when there are large sample outliers - like happens with the papers in this review.(see DOI: 10.1177/014920639902500602 for more on statistics)

An example, since Washington State University goes out of their way to highlight it. One of the studies used is a 2010 paper comparing the 'sensory' quality of organic and conventional strawberries, among other things related to how much more awesome organic food is. Sensory quality is not a scientific parameter but it got used anyway. Naturally, the Washington State University scholar behind that finding, John Reganold, declared this new review using his work "an impressive study" in their press release. It would be like a Pepsi official saying he was impressed by a blind taste test finding Pepsi tastes better than Coke. I believe he is impressed, but it's not science.

Reganold runs an organic farm and this year he won National Resource Defense Council's (NRDC) Growing Green Award. When an advocacy group is giving you an award, it's not because of your knowledge of science, it's because you write articles claiming organic farming is only not viable when you don't include 'value' such as the intangible benefit to Gaia, which he has also done.

The review also concludes that by simply choosing organic you get antioxidants equivalent to two extra portions of fruit and vegetables per day. They mean things like Vitamin E. Organic food is, in their minds, literally a Miracle Vegetable.  How did those studies measure enough product to be sure antioxidants were really higher in a large enough sample? They didn't, that is why more neutral studies have never found that result but a review can if they are determined to do so. They include papers that would be disqualified in many cases. 

Would that extra Vitamin E even make a difference? Well, we know there are lots of mitochondrial pathologies but we also know that if organic fruits and vegetables prevented those, people would never have gotten any of the diseases we know they have gotten throughout history. 

As Intrepid Wanders pointed out on Twitter, these same conclusions seem to be written over and over, with just a few changes. Györéné, Varga and Lugasi in 2006 wrote 

Organic crops contain a significantly higher amount of certain antioxidants (vitamin C, polyphenols and flavonoids) and minerals, as well as have higher dry matter content than conventional ones. Moreover, there is a lower level of pesticide residues, nitrate and some heavy metal contaminations in organic crops compared to conventional ones

while the new paper says 

the concentrations of a range of antioxidants such as polyphenolics were found to be substantially higher in organic crops/crop-based foods ... Additionally, the frequency of occurrence of pesticide residues was found to be four times higher in conventional crops, which also contained significantly higher concentrations of the toxic metal Cd.

It's understandable that there are only so many ways to say the same thing, but this is clearly a bunch of people reaffirming each other. I recently interviewed Frederick Crane, Professor Emeritus at Purdue and the man who discovered Coenzyme Q, which launched the whole field of mitochondrial antioxidation research. In our phone calls and pages of emails he never once said that eating a fruit grown using an organic pesticide rather than a synthetic one would make anyone live longer. Or that any fruit would make anyone live longer.

I think most people are looking for a compelling argument for organic food. If I had my way everything my family eats would be killed, grown, cleaned and cooked by no hands except mine. I'm certainly no fan of added chemicals in food production but I am even less a fan of starving children in the third world and the "let them eat kale" elitism of clueless Whole Foods shoppers. In the race to make more food with less strain on the environment, science is winning by a long shot. American farmers have "dematerialized" food production in a way that is an example for the entire world.

That said, we're going to have to keep on looking for a compelling argument for the organic food process as a viable mass food source - because this review isn't it. 

Citation: Barański M, Srednicka-Tober D, Volakakis N, Seal C, Sanderson R, Stewart GB, Benbrook C, Biavati B, Markellou E, Giotis C, Gromadzka-Ostrowska J, Rembiałkowska E, Skwarło-Sońta K, Tahvonen R, Janovská D, Niggli U, Nicot P, Leifert C, 'Higher antioxidant and lower cadmium concentrations and lower incidence of pesticide residues in organically grown crops: a systematic literature review and meta-analyses', Br J Nutr. 2014 Jun 26:1-18 doi:10.1017/S0007114514001366