It may be harming trust in epidemiologists and peer review as well.
It's easy to criticize peer review, because the flaws are obvious. It's also the best we've got, so we have to put up with claims that a weedkiller turns frogs gay or that men with short index fingers are nicer to women in order to get 1,000 great studies. One problem that only became evident to most in 2020 - epidemiology peer review can be almost as bad.
If I gather a group of astrologers and ask them to peer review an astrology paper saying that our location in the cosmos the day we were born shaped our personalities, then astrology has been peer-reviewed. It's the scientific equivalent of buying swampland in Florida, where a group colludes to purchase useless land from each other in turn, and the last one sticks the bank which provided the loan with the land while the whole group splits the profit.
Rather than being a hard science the way physics is, epidemiology is looking for correlation - X may be more likely to cause Y or prevent Z. Any results are considered exploratory, meaning they might merit a science follow-up. The problem is that correlation can be prone to bias and even manipulation.(1) It is so easy to manipulate there is a spurious correlations website, where you can randomly get two curves going the same way and "suggest" a link between them. If you want to claim bacon is as dangerous as mustard gas and plutonium, as International Agency for Research (IARC) on Cancer did, you simply have to ignore dose, which no serious scientist would do. Instead, IARC and others with a goal that is not informing the public lump in studies with doses across five orders of magnitude to create correlation. One piece of bacon is the same as eating 10,000 using their methodology. Obviously you know that is crazy, but they are looking for hazard, not risk from a hazard. (2) It's like issuing a paper saying sharks can kill you. If you live in Utah, your risk is zero but IARC leaves that part out.
Bias can happen when a group wants to find something instead of doing true exploratory research. And believing China as part of a desire to see what they wanted to see about pandemic lockdowns may be why a team concluded that the only effective way to mitigate COVID-19 was shutting down the world.
A more recent look at the same data the original paper used found it was easy to be fooled. Governments are using dynamic models to make potentially crippling decisions (3) but policy makers often don't understand the limits of models. And can't ask the people running the simulations. There is a saying that goes "Everyone believes the data except the collector, no one believes the model except the modeler" but COVID-19 has thrown that out of whack along with everything else. Everyone is believing these models, even if anyone can see using a few weeks in March to declare that total lockdown - China literally welded people's doors shut- is the only thing that worked is suspect. Social distancing, masks, etc. were not enough, they concluded.
Except in the new paper the authors re-ran the code using the repository data and got different results. Imagine if you were building a bridge and the modeler said it was safe. Then another modeler ran the same simulation using the same data and results showed the bridge would collapse. Which do you trust? You don't build the bridge until you know. In the real world, you have measured data, that is how you calibrate models, but the original paper didn't have enough data to be meaningful, and the data it did have still only found that lockdowns worked in 10 of 11 countries. Viruses don't know political boundaries, so it is impossible that a containment strategy that fails in one country but works in another is really a strategy. The model is just ignoring confounders.
No one is saying they intentionally made modeling choices to limit sensitivity issues and therefore concerns about parameters (and conclusions) but shouldn't that have been flagged in peer review? Well, yes, but we can't blame the system. Peer review is a thankless job. Corporations like the owner of Nature want scholars to do it for free for their billion dollar multinational conglomerate. It is little wonder so many studies with methodologies subject to question get published when the subset of reviewers is limited to people who are willing to do it, have time to do it, and are already in the tribe.
We can blame government panels and the journalists that enable them for promoting papers that match confirmation bias. Both have become so highly politicized that it is difficult for the public to trust either. Media won't be the trusted guides they claim to want to be when the New York Times routinely endorses acupuncture, astrology, ghosts, and organic food - because their readership believes in that nonsense. How can we trust them on science, when their readership is overtly against it in so many other areas? They're not alone, obviously.
That means it is up to other science groups to find the flaws in papers that slipped past peer review but that is a challenge when COVID-19 has resulted in 15,000 papers so far. Will corrections get the same attention as original claims did? There will be journalistic and academic scrutiny if Trump mentions a peer-reviewed paper, but otherwise almost never, journalism is instead often about rewriting the latest claims in press releases. And that is hurting us all.
(1) Due to epidemiologists undermining epidemiology, if you are an insider at the National Institute of Environmental Health Sciences(NIEHS), your credibility as a trusted guide for the public is suspect. The last boss there routinely gave talks at organic industry events, she was overtly anti-science, and they publish an internal journal that is more like a conspiracy fanzine opposing science. You do not even get hired inside there unless you have proven that you hate chemicals, toxicology, and biology. Basically, you hate evidence and think only statistical correlation matters. It's the same at Harvard School of Public Health, which created a whole industry using food frequency questionnaires to match various products to either early death or greater longevity. At IARC, mentioned above, an insider was busted collaborating with trial attorneys for contracts before analyses of chemicals they were involved with had even been concluded. Where did Chris Portier, PhD, get the credibility to get into IARC? By getting hired by the organic industry activist running NIEHS.
(2) Yet when they are drumming up journalist interest for their latest claim about how the modern world is killing us, they use the word risk liberally in their media kits - 38 times in that one, until they got criticized by me and then remove them. Here are more examples of them using risk when they only searched for hazard. An odd thing to do for a group that claim to be the best epidemiologists in the world.
“An analysis of data from 10 studies estimated that every 50 gram portion of processed meat eaten daily increases the risk of colorectal cancer by about 18%.” And “studies suggest that the risk of colorectal cancer could increase by 17% for every 100 gram portion of red meat eaten daily.” And more, despite not calculating any risk.
(3) Not China, they are taking the pandemic to the bank by ignoring it while American Exceptionalism makes us desire to turn gunshot deaths into COVID-19 ones so we can be the hardest hit by coronavirus as well.
- Should Any Current COVID-19 Estimates Be Used To Make Policy Decisions?
- New Study: Beef In A Healthy Diet Does Not Increase Risk Of Heart Attacks
- IARC Creates A 'Get To Know A Scientist' Series- Now They Just Need To Hire One
- Who Is Surgisphere And How Did They Get Into A Lancet Hydroxychloroquine Paper?
- Coronavirus Treatment Retraction Shows The Life Sciences Are Too Often ‘A House Built On Sand’