The first question should be, how could such a thing happen? I don't ingest pesticides, and neither should you. People don't ordinarily do that, you'd have to really try to do that, unless you live out in the fields. In its coverage of this story, Reuters finds someone to speculate "that the pesticides probably drifted from crops through the air, and that’s how pregnant women were exposed."
That someone would be Dr. Philip Landrigan, epidemiologist and pediatrician at Mount Sinai in New York, who is the go-to guy for this sort of thing. Everyone from the New York Times to the US Senate Committee on Environment and Public Works has quoted him, implicating pesticides in everything from ADHD to cancer to autism.
But that is not really how pesticide application works. We can forgive a New York City pediatrician if they have never visited a farm, or for not knowing that people also use pesticides in and around their homes, sometimes four or more times per year. Speculation aside, the public will be concerned about toxicological harm to kids because there is a template for concern - 50 years ago thalidomide, the nausea drug for pregnant women, clearly caused harm in babies. Every environmental fund-raiser compares their scary chemical of the week to it even today.
But the authors of the paper in Environmental Health Perspectives don't even know if any of the mothers actually breathed in pesticides or were exposed to them in any way. All they know is that mothers of kids with an autism spectrum diagnosis in the agricultural Sacramento region were more likely to live near a farm that used pesticides. That's common in this region, I live here, and I am within 2 miles of a farm even though I am in a city of 50,000 people. Yet their results counter-intuitively found that living closer to pesticide-using farms was better. The Odds Ratios for people who lived closer (1.25 KM) to farms were weaker than for people who lived farther away (1.5 KM). Weak associations that make no sense often mean it is coincidence that everyone but the authors can see.
So given that first weakness, why did UC Davis let a graduate student imply the science was settled? In their press release, lead author Janie Shelton declares, "This study validates the results of earlier research that has reported associations between having a child with autism and prenatal exposure to agricultural chemicals in California."
That's a bold claim and so it requires bold evidence. Let's get to it!
Agriculture in California is a $38 billion industry so almost everything is known about it, including pesticide use. Farming as logistics, technology and science is exhaustively examined on a constant basis because the margins are thin. The Childhood Autism Risks from Genes and Environment (CHARGE) study at U.C. Davis was created to look for causes of autism, so it has followed 1,600 kids with Autism Spectrum Disorder since 2003. Since the participants are in one California location, and pesticide data is also available, they matched younger kids in the study to a pesticide map.
Result: more autism among kids close to farms. So much for healthy living outdoors, right? But there is a problem - they didn't actually measure anything, they used a very small sample size and a proxy. We all know what a proxy is; it is an agent for another thing. In epidemiology, though, its use can be tricky. That discipline often uses interviews or surveys, like this one did, which are not all that reliable in small samples. Using a proxy based on questionnaires to connect autism and pesticides via farms is a bit of a red flag unless things are well controlled. Using a logistic regression to try and accurately control for confounding factors can be tough for scientists, in the hands of people with far less statistical expertise, it is wrong more often than right. Statistics experts by now assume the worst.
Proxies clearly have value. If someone is dead or unavailable, a proxy must be used. But they can be misused. To put that into context, a proxy was also used to recently show:
I'm a statistician. My motto is 'I haven't read your paper yet but I'm virtually certain your methods are flawed&your results are wrong.'— Stephen John Senn (@stephensenn) April 9, 2014
(1) Zebras evolved stripes due to flies
(2) Famous paintings are proof of climate change
(3) Bicep size correlates to political conservatism.
(4) A lack of vitamins is linked to more autism
That fourth one is interesting because it also used The Childhood Autism Risks from Genes and Environment (CHARGE) study. Retrospective studies, like finding people with autism and then finding statistical matches to other stuff, is what a lot of what epidemiologists do. Unfortunately, that's not good science.
Using a demographic method like that, we can find a real nexus for autism, a place that is far worse than farms: It is being in wealthy Los Angeles neighborhoods with lots of pediatricians nearby. That is such an autism causer that kids who moved there and had no diagnosis before suddenly caught it.
That's not to trivialize autism or pesticides, I mention it in order to not let science be trivialized. I grew up on a farm, I respect pesticides in a way the anti-science fundamentalists at Pesticide Action Network can't. And public health is serious business. But because it is serious business, we want to make sure that the public outrage machine is only geared up when it is warranted. When thalidomide was found to be a problem, getting action was easy, but today those kinds of claims are launched once a week and the public quickly becomes jaded. That means they won't know real science from environmental bombast.
Landrigan, quoted in the Reuters article, says the study's two biggest weaknesses, that they used a proxy and didn't actually measure anyone's exposure, lead him to counter-intuitively believe their findings are too conservative, so look for his reliable anti-pesticide stance to get him profiled in The New Yorker some time soon. But rational people can't take that seriously.
I wrote in the title this has 3 weaknesses. In summation, they are:
(1) They didn't measure any pesticides in mothers or their homes.
(2) The data is inconsistent. Their odds ratios were weak, and that means there is a strong likelihood they are overstating the power of chance events.
(3) The sample size is too small. 468, in this case.
The paper itself concedes it is only "exploratory" so why does the lead author declare "This study validates the results"? That brings us, by proxy(!), to a fourth weakness.
The principal investigator of the study is Irva Hertz-Picciotto, a professor and vice chair of the Department of Public Health Sciences at UC Davis. She also happens to be on the Advisory Board of Autism Speaks and on the board of an anti-chemical advocacy group Healthy Child, Healthy World. That doesn't invalidate the work, obviously, but by proxy, when a person conducting a study that links two advocacy causes together fails to disclose their high-level involvement in advocacy groups for those causes, it has to be considered that there is something happening that is not the impartial science that the public expects of the science community.
What would environmentalists say if a study claiming pesticides boosted IQ happened to be done by someone on the advisory board of a pro-pesticide advocacy group?
The study would still have to be taken on its merits, just like this one must be, but it's a good reason to use some skepticism about its motivations. Taken on its merits this doesn't tell us anything more about autism than that study claiming vitamin supplements would prevent it did.
Citation: Janie F. Shelton, Estella M. Geraghty, Daniel J. Tancredi, Lora D. Delwiche, Rebecca J. Schmidt, Beate Ritz, Robin L. Hansen, Irva Hertz-Picciotto, 'Neurodevelopmental Disorders and Prenatal Residential Proximity to Agricultural Pesticides: The CHARGE Study', Environ Health Perspect; DOI:10.1289/ehp.1307044