:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/46850454/9570395361_c3ac992ab1_k.0.0.jpg)
- Intestinal worms are a massive health problem in low- and middle-income countries, afflicting up to a quarter of the world's population.
- Over the past 20 years, many organizations have pushed large-scale "deworming" initiatives, which often involve giving every child in a school deworming pills on a regular basis.
- These programs — largely inspired by an influential study that ran in the late '90s in Kenya — were thought to not only treat children, but also improve their school performance and overall health. In other words, very inexpensive pills seemed to be "one of the most potent anti-poverty interventions of our time."
- Researchers have been calling these promises into question. Most recently, a group of epidemiologists in London replicated and re-analyzed the results of that Kenya trial, and uncovered a number of flaws and exaggerations.
- They aren't alone; for years, other research groups have been wondering whether mass deworming schemes are really the panacea some make them out to be. The high-stakes argument over their effectiveness has been dubbed the "worm wars."
- And the worm wars aren't over. Many folks who work in development are pushing back on the latest debunking, saying the analysis was unfair.
Deworming schemes have been called one of the most important health interventions ever
Worms are terrible for human health. These parasitic bugs usually live in people's stomachs and intestines, stealing away nutrients from food and leaving their hosts weak, malnourished, even cognitively impaired. Left untreated, worms can cause organ damage and internal bleeding, and bring on other diseases. It's not surprising that people with worms — an estimated quarter of the world's population — may find going to school or working difficult.
Deworming schoolchildren with effective and inexpensive drug treatments has been called one of the most important health interventions ever. The World Health Organization, the Gates Foundation, and the World Bank all back large-scale deworming initiatives.
Even Cherie Blair, wife of the former British Prime Minister Tony Blair, put her force behind the effort to "deworm the world." In an infamous incident at Davos in 2008, she reportedly dressed up as an intestinal worm and chased around people who pretended to be schoolchildren.
Many of these initiatives have been based on an influential study from Kenya
The push to deworm everybody has been influenced in a big way by the results of a study conducted in Kenya in 1998 and '99.
The researchers, economists Edward Miguel and Michael Kremer of Berkeley and Harvard, looked at what happened when schools phased in mass treatment with deworming drugs and worm-prevention education programs. They found that these initiatives reduced absenteeism by 25 percent and improved kids' health and school participation.
Most amazingly, the results also seemed to spread to neighboring schools where children weren't treated, presumably because the drugs interrupted worm transmission from child to child.
This study helped make deworming drugs among the most popular in the world, and influenced national and global policies and programs designed to address this pervasive health problem.
When researchers re-analyzed the Kenya study, some of the results fell apart
The only trouble is, the foundational study had some major flaws. That's what a group of independent researchers and epidemiologists at the London School of Hygiene & Tropical Medicine just found when they replicated and reanalyzed its data. (Their results can be found in two open-access papers — here and here — published as part of a collection this week in Oxford's International Journal of Epidemiology.)
What they found, as Ben Goldacre nicely summarized at BuzzFeed, was that some of the original data was missing and a lot of the available data had errors. Overall, this meant that some of the deworming schemes' effects were exaggerated.
Specifically, in one paper, they replicated the original study, using data provided by the authors to re-analyze the results according to their original methods. They found that while worm infections decreased, some of the other supposed benefits didn't happen: Most notably, the indirect effect on neighboring schools vanished. They also found little evidence for an effect on anemia — an important supposed health improvement in the original study.
In the second paper, which re-analyzed the original study results using different statistical methods, they uncovered "a substantial amount of missing data." And the effect of deworming on school attendance seemed to differ depending on what statistical analysis they used. As they wrote, "We find that the study provides some evidence, but with high risk of bias" that the school-based programs improved attendance and there was "no evidence of effect" on school performance.
Based on these results, there's no doubt that deworming treatments work. The re-analyses showed that large-scale programs decreased worm infections. But the papers call into question exactly how effective the programs really are on all the other measures they promise to help, such as attendance and exam performance.
The authors of a related commentary, also published in the International Journal of Epidemiology, note that one of the most important points these papers make is that giving out deworming drugs in the original Kenya study was part of a "complex health education and drug treatment intervention."
In other words, it wasn't only the drugs that may have had an effect on kids. "The intervention schools also received regular public health lectures, wall charts and teacher training on worm prevention," the researchers wrote. The replication study showed that these health-promotion activities may have had as much of an impact as giving kids deworming drugs.
So, they conclude, if the effects on school attendance and performance are not clear, nor are the gains in nutritional status and cognition, the development community has some soul searching to do: "Without these effects it seems implausible that deworming itself would have an independent effect on school attendance or economic development."
Some disagree with the deworming debunkers
The re-analysis of this one influential study is in no way the final word on whether or not we should continue these efforts to deworm the world.
Criticisms have been popping up from folks in the global health and development communities. Some have noted that the statistical methods used to re-analyze the Kenya study were unfair.
Others say that the re-analysis only seemed to strengthen the findings of the original study, since some of the outcomes measured didn't actually change all that much.
Chris Blattman, an associate professor at Columbia University, put it this way:
The bottom line is this: both sides exaggerate, but the errors and issues with the replication seem so great that it looks to me more like attention-seeking than dispassionate science. I was never convinced that we should deworm the world. There are clearly serious problems with the [Kenya] Miguel-Kremer study. But, to be quite frank, you have throw so much crazy sh*t at Miguel-Kremer to make the result go away that I believe the result even more than when I started.
Here, he charges, the replicators unfairly picked apart the Kenya study, parsing it using a bunch of methods that not even the best-designed study could have withstood.
So where does this leave us?
Whether or not you think the Kenya reanalysis is fair, it's important to note that these criticisms of deworming studies haven't come in isolation.
People have been debating the efficacy of deworming for years. Starting in 1997, the Cochrane Group, which analyzes all the best available research on clinical questions to come to more fully supported conclusions, has been noting that there are serious problems and holes in the evidence base on deworming programs.
The latest Cochrane review on the subject, which included the findings from the new re-analyses as well as more than 40 other trials, comes to a similar conclusion as the Kenya study replicators: "Treating children known to have worm infection may improve weight gain but there is limited evidence of other benefits. For routine deworming of schoolchildren in endemic areas, there is quite substantial evidence that deworming programs do not show benefit in terms of average nutritional status, haemoglobin, cognition, school performance, or death."
This means there's just no good evidence that deworming programs improve many of the things they promise to, such as kids' nutrition and ability at school, or that they reduce rates of death. According to all the best available research, the Cochrane authors say, these schemes just aren't having the effect on overall health and education that some hoped they would.
This is an important finding, given that it's based on many studies, and policies and programs should be designed to reflect the weight of evidence, not just one study that has a result people like.
If it turns out that they're right, perhaps this shouldn't be surprising. We want solutions to complex problems so badly that we might be inclined to glom onto something that looks like an easy fix.
As one of the authors of this latest reanalysis, Calum Davey of LSHTM, told the Guardian of the original Kenya study, "I can’t speak [on] why it was so influential. Partly I think it was because of the optimism built on the findings — a golden bullet for some problems like attainment and education in low-income settings."
