How a Faulty CDC Study Led to Bad Journalism, False Panic, and Bad Legislation

Here in the United States, we rely on research from many organizations, both private and governmental, to inform us on public health issues and guide our legislators in crafting laws to benefit the public good. This can mean anything from local bans on hydrogenated oils to wider-scale regulation like tobacco labeling requirements. These laws are always backed by empirical evidence that seems to demonstrate a need for change to benefit public health.

While the two examples I gave you are pretty straightforward, this is not always the case. Study data can be difficult to decode in ways that give us practical information, and this often leads to misreading of study results and faulty conclusions for those of us who aren’t scientists or researchers. Add to this the inevitable incidence of studies that use faulty methodology, too-small sample sizes, or yield results that can’t be reproduced, and you begin to see that the world of actionable study data is anything but clear cut. As we’ll see, this can cause big problems.

The Faulty CDC Study on Farmers and Suicide

This was the case last December when a Centers for Disease Control and Prevention study (take a look and see how difficult these things can be to read and interpret) caught the attention of the national media. The faulty CDC study reported that farmers had the highest rates of suicide in the country by occupation, and the story caught fire. Soon it was found on such reputable news sources as The Guardian, NPR, and The New York Times. The public was justifiably alarmed, and this put pressure on lawmakers to address the issue in the United States farm bill, one of the biggest, most important pieces of omnibus legislation there is. Because of this, the current draft of the 2018 Farm Bill includes mental health provisions for farmers.

There’s just one problem: The study data was badly misleading. Farmers likely don’t have the highest rates of suicide by occupation, and the funds allotted for mental health in the farm bill likely won’t go to those who actually need them. After questions were raised about the methods and language used in the study, the CDC retracted the data and backed off their findings concerning farmers and suicide. By this time, the public was badly misinformed, and the study data was accepted as truth by many.

How did this happen?

Misleading Claims and Bad Data

Put simply, the faulty CDC study lumped suicides among farmers in with a category of workers known as “farming, fishing, and forestry,” which does not actually include farmers, who are classified as managers. This group includes farm laborers, who do have high suicide rates, but not farm owners or managers. This resulted in data that tells us nothing about the suicide rate among farmers.

When the study abstract falsely claimed that farmers were killing themselves at record rates, legislators sprang into action and drafted legislation to protect farmers, but did nothing to help those in the “farming, fishing, and forestry” category. Those that possibly need the help the most were completely left out of the bill as a direct result of bad data, and millions of dollars potentially wasted. To top it all off, media outlets that initially reported on the story did not take down the stories based on bad data. Instead, they inserted disclaimer paragraphs in the middle of the stories and left the misleading headlines intact. After all the confusion has seemingly been cleared up, this bad data is still circulating, and people are still reading it as solid research.

Reader Beware

This is a cautionary tale. It’s a reminder to always take study data with a healthy degree of skepticism and be wary of any data that makes serious claims about anything. This is not to say that peer-reviewed research in general should be discounted in any way. Research like that done by the CDC is invaluable for our health, as well as the advancement of medicine and society as a whole. There’s no reason to believe the CDC would intentionally skew results to any particular outcome.

Sometimes mistakes are just going to happen — that’s the way science works. Basing our actions on empirical evidence is in no way a guarantee that we’ll always make wise choices, but it’s the best option we have for understanding and improving the world we live in. Be skeptical; fact check, and don’t believe everything you read.