The Environmental Working Group (EWG) says that it "helps protect your family from pesticides."

The purpose of this Applied Mythology post is to "help protect your family from dangerously misleading information from the EWG."

Each year since 1991, the USDA has been publishing the results from a large-scale pesticide residue monitoring program called the PDP.  Each year, a different set of crops is chosen and samples are purchased from regular stores and tested. Year after year, the results of those studies confirm the safety of the food supply. Year after year the EWG misrepresents the data to say otherwise.

To understand what that is like for the people who farm those crops, consider this analogy:

What if you were taking a college course that was critical for your graduation, but your entire grade was going to be based on a giant group project involving thousands of other students you didn't even know and with whom you could not even communicate.  Some of those student's test results would be chosen at random and the grade for everyone in the class would depend on how they did. When the grading is done, you find out that the class score was over 99% - a clear A+!  Then,  someone who doesn't really understand the topic of the class, or chooses not to,  re-grades the test and tells your potential future employers that you got a D, and many of them believe the incorrect grade.

This is much like what farmers have been experiencing for years. They grow a crop as best they can, and use pesticides only as necessary and within the strict rules established by the EPA. Much of what they use are pesticides with very low toxicity.  In years that their crop is selected for the PDP, random samples of their commodity are purchased in stores, including examples coming from other countries.  

They are taken to federal and state laboratories and scrutinized for trace residues of hundreds of different chemical pesticides. When the data is finally published (usually two years later), the highly qualified experts of the USDA, EPA and FDA conclude that the system is working and that consumers should confidently purchase and eat the crop without concerns about residues.  In fact, studies show that the anti-cancer benefits of eating things like fruits and vegetable far, far outweigh and minuscule risk associated with pesticides.

Then each year, the EWG takes advantage of the transparent availability of the USDA-PDP data, but then performs their own "analysis" which experts have rejected as utterly anti-scientific.  They generate an incorrect "grade" for the crop and post it as part of their "Shopper's Guide," and on their notorious "Dirty Dozen List."  The grower's virtually perfect grade gets forgotten and what is passed along by an un-critical press and blogosphere is the distortion that the crop is "dirty."

Many consumers believe this and heed the EWG's suggestion that they need to buy organic versions of that crop (the actual agenda of the EWG is the promotion of organic and also their own fundraising). Worse still, there is some evidence that this disinformation causes consumers to purchase and eat less produce. At a minimum, many consumers feel guilty for not buying organic.

As you can imagine, this is very frustrating for farmers. Some have joined in groups which are trying to get out a much more accurate interpretation of the data which is to say that the PDP confirms the that pesticides are well regulated and that the farming industry is doing a very good job.  They want to reclaim their rightful A+!

What Does The Data Really Say?

I decided to do an independent analysis of the latest PDP data (for growing year 2011, released earlier this year). The information is freely available from a USDA web site, but using it is a non trivial exercise. The zipped file expands to 92MB because it contains 2.2 million rows of information covering each of the hundreds of pesticides or metabolites looked for in each of the thousands of food samples. 1.75 million of those are for fruits and vegetables.

Fortunately, even using the extremely sensitive analytical techniques available today, less than 1% of these rows are cases where some detectable residue was found.  I'd be happy to email you the 15,450 row Excel table left after eliminating all the non-detects.

To understand the significance of each detection it is necessary to know what the chemical is and what "tolerance" the EPA has established for it on each crop.  The tolerance is a very conservative threshold for how much residue represents an acceptable margin away from any health risk.

 It is based on the best data and risk assessment tools available to EPA.

 Not surprisingly, the tolerances for different chemicals are very different based on the details of their toxicological profile.  I've plotted the distribution of all the detections relative to crop/chemical-specific tolerance in the graph below.

For all 21 commodities tested from 2011, there were only 0.18% of residues found which were higher than the EPA tolerance.  In fact fully 1/2 of the detections were of levels more than 100 times lower than the already conservative tolerance.

There are differences between crops and between country of origin, but they are only between good and very good.  Snap peas were the "worst" example, particularly those imported from Central America (see above), but they still had 94% of detections below tolerance.

 The few that were above are not particularly scary either (you can see the detail in the complete analysis I posted on SCRIBD).  Many crops had a "perfect score" of keeping all the residues below the tolerance (see graph to the right)

Quite appropriately, one of the "cleanest" crops was pear baby food.  When EPA sets the tolerance for baby food it is even more conservative than ever.  In this case all the detections were below tolerance and more than 99% of them were 10 times or more lower than the tolerance (graph below).

The people collecting samples for the PDP  do track whether the sample had an organic claim.  For most crops the number of organic samples is too low to make a meaningful comparison, but for pear baby food, 11.5% were organic.  Interestingly, among those 67 samples, there were 101 pesticide residues detected, only 33 of which are for the organically approved insecticide, Spinosad.

The rest were for synthetic pesticides including some that are applied after harvest (such as DPA which prevents scald in storage). As with the conventional samples, these residues were at such tiny levels as to be of no concern, but for this and other crops, choosing organic does not guarantee "no pesticide residues,"  instead the same risk assessment process suggests safety for both the organic and conventional options.

How Does The EWG Ranking Compare to One Based on Science?

What I have been presenting is an analysis that pays attention to what the chemical is, what levels are found, and what the EPA has concluded from its risk assessment process.  The EWG's ranking ignores all of those factors.  

I've taken the EWG's ranking (higher numbers are supposedly "cleaner") and  compared it with a tolerance-based measure which is the percent of the detections that are not even as much as 1/10th of the tolerance (again, high number = cleaner).  Not surprisingly, there is really no correlation between these two approaches (see below).

Again, none of these examples are really problematic, but cauliflower, which EWG calls part of the "clean 15" and ranks as number 34 in their list has the has more detections over 1/10th of the tolerance than other crops.  Apples, which are the worst according to EWG have 92% detections below 1/10th of tolerance - more than a great many other crops.

Canned beets, for which not even one detection was noted among 756 samples from 2011, doesn't appear on EWG's "Clean 15" list or in the list at all. Again, the real "grades" are all "A's," just to different degrees.  It's like Lake Wobegon - all the crops are above average.

What is the take home message from all of this? Eat more fruit and vegetables! And don't worry about whether it is organic or not. The fact is that we know less about what is on organic produce than on conventional.

Full analysis posted as An Independent Analysis of the 2011 PDP Data on SCRIBD.