Fake Banner
    How Wrong Is The Latest "Dirty Dozen" List?
    By Steve Savage | May 16th 2013 08:10 PM | 9 comments | Print | E-mail | Track Comments
    About Steve

    Trained as a plant pathologist (Ph.D. UC Davis 1982), I've worked now for >30 years in many aspects of agricultural technology (Colorado State...

    View Steve's Profile

    The Environmental Working Group (EWG) says that it "helps protect your family from pesticides."

    The purpose of this Applied Mythology post is to "help protect your family from dangerously misleading information from the EWG."

    Each year since 1991, the USDA has been publishing the results from a large-scale pesticide residue monitoring program called the PDP.  Each year, a different set of crops is chosen and samples are purchased from regular stores and tested. Year after year, the results of those studies confirm the safety of the food supply. Year after year the EWG misrepresents the data to say otherwise.

    To understand what that is like for the people who farm those crops, consider this analogy:

    What if you were taking a college course that was critical for your graduation, but your entire grade was going to be based on a giant group project involving thousands of other students you didn't even know and with whom you could not even communicate.  Some of those student's test results would be chosen at random and the grade for everyone in the class would depend on how they did. When the grading is done, you find out that the class score was over 99% - a clear A+!  Then,  someone who doesn't really understand the topic of the class, or chooses not to,  re-grades the test and tells your potential future employers that you got a D, and many of them believe the incorrect grade.

    This is much like what farmers have been experiencing for years. They grow a crop as best they can, and use pesticides only as necessary and within the strict rules established by the EPA. Much of what they use are pesticides with very low toxicity.  In years that their crop is selected for the PDP, random samples of their commodity are purchased in stores, including examples coming from other countries.  

    They are taken to federal and state laboratories and scrutinized for trace residues of hundreds of different chemical pesticides. When the data is finally published (usually two years later), the highly qualified experts of the USDA, EPA and FDA conclude that the system is working and that consumers should confidently purchase and eat the crop without concerns about residues.  In fact, studies show that the anti-cancer benefits of eating things like fruits and vegetable far, far outweigh and minuscule risk associated with pesticides.

    Then each year, the EWG takes advantage of the transparent availability of the USDA-PDP data, but then performs their own "analysis" which experts have rejected as utterly anti-scientific.  They generate an incorrect "grade" for the crop and post it as part of their "Shopper's Guide," and on their notorious "Dirty Dozen List."  The grower's virtually perfect grade gets forgotten and what is passed along by an un-critical press and blogosphere is the distortion that the crop is "dirty."

    Many consumers believe this and heed the EWG's suggestion that they need to buy organic versions of that crop (the actual agenda of the EWG is the promotion of organic and also their own fundraising). Worse still, there is some evidence that this disinformation causes consumers to purchase and eat less produce. At a minimum, many consumers feel guilty for not buying organic.

    As you can imagine, this is very frustrating for farmers. Some have joined in groups which are trying to get out a much more accurate interpretation of the data which is to say that the PDP confirms the that pesticides are well regulated and that the farming industry is doing a very good job.  They want to reclaim their rightful A+!

    What Does The Data Really Say?

    I decided to do an independent analysis of the latest PDP data (for growing year 2011, released earlier this year). The information is freely available from a USDA web site, but using it is a non trivial exercise. The zipped file expands to 92MB because it contains 2.2 million rows of information covering each of the hundreds of pesticides or metabolites looked for in each of the thousands of food samples. 1.75 million of those are for fruits and vegetables.

    Fortunately, even using the extremely sensitive analytical techniques available today, less than 1% of these rows are cases where some detectable residue was found.  I'd be happy to email you the 15,450 row Excel table left after eliminating all the non-detects.

    To understand the significance of each detection it is necessary to know what the chemical is and what "tolerance" the EPA has established for it on each crop.  The tolerance is a very conservative threshold for how much residue represents an acceptable margin away from any health risk.

     It is based on the best data and risk assessment tools available to EPA.

     Not surprisingly, the tolerances for different chemicals are very different based on the details of their toxicological profile.  I've plotted the distribution of all the detections relative to crop/chemical-specific tolerance in the graph below.

    For all 21 commodities tested from 2011, there were only 0.18% of residues found which were higher than the EPA tolerance.  In fact fully 1/2 of the detections were of levels more than 100 times lower than the already conservative tolerance.

    There are differences between crops and between country of origin, but they are only between good and very good.  Snap peas were the "worst" example, particularly those imported from Central America (see above), but they still had 94% of detections below tolerance.

     The few that were above are not particularly scary either (you can see the detail in the complete analysis I posted on SCRIBD).  Many crops had a "perfect score" of keeping all the residues below the tolerance (see graph to the right)

    Quite appropriately, one of the "cleanest" crops was pear baby food.  When EPA sets the tolerance for baby food it is even more conservative than ever.  In this case all the detections were below tolerance and more than 99% of them were 10 times or more lower than the tolerance (graph below).

    The people collecting samples for the PDP  do track whether the sample had an organic claim.  For most crops the number of organic samples is too low to make a meaningful comparison, but for pear baby food, 11.5% were organic.  Interestingly, among those 67 samples, there were 101 pesticide residues detected, only 33 of which are for the organically approved insecticide, Spinosad.

    The rest were for synthetic pesticides including some that are applied after harvest (such as DPA which prevents scald in storage). As with the conventional samples, these residues were at such tiny levels as to be of no concern, but for this and other crops, choosing organic does not guarantee "no pesticide residues,"  instead the same risk assessment process suggests safety for both the organic and conventional options.

    How Does The EWG Ranking Compare to One Based on Science?

    What I have been presenting is an analysis that pays attention to what the chemical is, what levels are found, and what the EPA has concluded from its risk assessment process.  The EWG's ranking ignores all of those factors.  

    I've taken the EWG's ranking (higher numbers are supposedly "cleaner") and  compared it with a tolerance-based measure which is the percent of the detections that are not even as much as 1/10th of the tolerance (again, high number = cleaner).  Not surprisingly, there is really no correlation between these two approaches (see below).

    Again, none of these examples are really problematic, but cauliflower, which EWG calls part of the "clean 15" and ranks as number 34 in their list has the has more detections over 1/10th of the tolerance than other crops.  Apples, which are the worst according to EWG have 92% detections below 1/10th of tolerance - more than a great many other crops.

    Canned beets, for which not even one detection was noted among 756 samples from 2011, doesn't appear on EWG's "Clean 15" list or in the list at all. Again, the real "grades" are all "A's," just to different degrees.  It's like Lake Wobegon - all the crops are above average.

    What is the take home message from all of this? Eat more fruit and vegetables! And don't worry about whether it is organic or not. The fact is that we know less about what is on organic produce than on conventional.

    Full analysis posted as An Independent Analysis of the 2011 PDP Data on SCRIBD.  

    Comments

    Great work, and thank you taking the time to provide the insight. Sadly, good news, too frequently, isn't "news" at all to today's science media.

    sdsavage
    Thanks Gori,  I enjoy big data sets
    Steve Savage
    As an executive working for a national produce company, I am in full agreeance with your independent research. Why the EWG has decided to put a cross-hair target on fresh produce is beyond imagination - it's one of the few things left which give real nutrition.

    sdsavage
    Ralph,
    I often get to attend the PMA and this year I went to United Fresh.  What I always see is an industry that is working very hard to deliver delicious and healthy foods to people.  The school lunch programs these organizations support are great.  And then EWG comes along and tries to undo all of that.  I really wonder if violates some law to misuse government data like that.  It certainly is unethical.
    Steve Savage
    Hank
    And they create new organizations and get celebrity endorsements and give money and endorsements to the politicians most likely to agree with their anti-food agenda. The Food Policy Action group got a Top Chef, a union lobbyist and an animal-rights activist together to rate politicians.  No shock at all this "nonpartisan" group still put all Republicans at the bottom of their list and all Democrats at the top of 'food' policymakers.

    Who runs it?  The same person who runs EWG.
     
    I do find this encouraging as organic seems to price too many people out of fresh produce and is not giving them any benefit either
    I mostly shop at Costco and don't bothr w/ their organic offerings
    Thank you Mr. Savage for loving large data bases and putting it into enjoyable reading

    sdsavage
    Michael,I am a sucker for large databases.  I'm working on something based of FAOStats which is a fantastic resource for data nerds like me.

    I also shop at Costco.  They do long-term contracts with suppliers instead of playing the spot market.  They also properly place produce in a cold room,  at least for part of what they sell here in the San Diego area.  I also avoid their organic offerings when I can.  Even my favorite retailer panders to that market.

    Steve
    Steve Savage
    Thor Russell
    What is the story with the article and reported study I have seen mentioned?
    http://www.mnn.com/green-tech/research-innovations/stories/herbicide-could-make-deadly-bacteria-more-dangerous-says-new#
    (I havn't spent any time looking at it, you may already know etc)


    Thor Russell
    sdsavage
    Thor,The study mentioned there looks like a clear example of "agenda science" - doing an experiment designed to show something negative but which bears no resemblance to reality.  They did a direct exposure of the herbicide to bacteria and meat.  When glyphosate is used as a herbicide it is often months before the crop is harvested and fed to animals so any residues would be minuscule between metabolism and dilution in the growing plant.  Also, glyphosate is phloem mobile and moves down to the roots more than up to the top.  In any case, to have been registered, the fate of the glyphosate in the plant over time had to be tracked in detail.  These researchers just put the glyphosate directly on the bacteria at levels that don't mean anything vs reality. 
    Steve Savage