A new, huge review of gun research has bad news for the NRA

Editor’s note, May 26, 2022: This story was originally published in 2018, and the statistics included may not be the most recent available. For Vox’s latest coverage of America’s gun violence epidemic, see our explainer.

For decades, the federal government, with the support of the National Rifle Association, has made it very difficult to answer a question at the heart of American public health and safety: Does gun control work?

The answer is hugely important given that guns killed nearly 39,000 Americans in 2016 alone. But after research on gun violence in the 1990s found that firearms do not — contrary to NRA talking points — make people safer, the group backed a federal funding freeze on gun policy research.

But studies have gone on — just without federal funding. And on Friday, a nonpartisan think tank, the RAND Corporation, released the results so far of its Gun Policy in America initiative, a two-year dive into the research on gun violence and the laws trying to curtail it.

RAND’s extensive report does not make any sweeping declarations about gun policy. It does, however, make clear that gun control research is very limited, calling on Congress to lift the NRA-backed funding freeze. It argues that this freeze has, by making it difficult to conduct better studies, led to a confusing empirical environment, where it’s easy for groups on both sides of the debate to cite shoddy work that supports their prior beliefs.

“The studies that have been done often reach opposite conclusions to each other,” Andrew Morral, the head of RAND’s gun policy initiative, told me. The lack of thorough research, he added, “creates this kind of fact-free environment in which people can cherry-pick any study that happens to support what their priors are on the effects of the law.”

Morral’s team spent two years reviewing US-based studies published over the past several decades, pulling out the most rigorous to try to find some “incontrovertible truths.” RAND concluded that, first and foremost, far more research is necessary. “Many of the matters that people disagree on when they disagree on gun policy have not been rigorously studied in ways that produce reasonably unambiguous results,” Morral said.

But there were some things that could be gleaned from the available evidence. While RAND as a nonpartisan group avoided any sweeping policy conclusions in its analysis, its review does seem to point in a direction, based on my own reading: More permissive gun policies lead to more gun deaths, while more restrictive policies lead to fewer gun deaths. Coupled with other evidence in this area, that supports the idea that more guns lead to more gun deaths.

Given that America is dealing with an immediate gun violence problem, as mass shootings and deaths pile up, the report is worth taking seriously. At the very least, there’s enough evidence to suggest that the federal government should stop refusing to fund research on policies that really could work to save thousands of lives every year.

The available research points in one direction

RAND’s report does not come out in favor of more or less gun control. Instead, the team compiled the best research that’s available so far into charts and in-depth evaluations — the result of a review of dozens of studies, focused on 13 policies and eight outcomes. Here are the overall findings, which only included studies that met RAND’s rigorous standards:

The RAND report emphasizes that much of the research on gun policy is still in its infancy. You can see that in the chart above in all the white and gray space — we still don’t have answers to a lot of important questions when it comes to gun policy, including the effects on defensive gun use, hunting and recreation, and police shootings.

But the answers we do have point in one direction. On the gun control front, there’s moderate evidence that background checks reduce suicide and violent crime, limited evidence that prohibitions associated with mental illness reduce suicide, moderate evidence that those prohibitions reduce violent crime, and supportive evidence that child-access prevention laws reduce suicides and unintentional injuries and deaths.

Meanwhile, there’s limited evidence that concealed carry laws increase violent crime and unintentional injuries and deaths. And there’s moderate evidence that “stand your ground” laws — NRA-backed measures that expand when someone can use a gun or other weapons to defend himself — increase violent crime.

If you put this all together, it suggests that restrictive laws seem to lead to fewer gun deaths, while the permissive laws seem to lead to more gun deaths.

That’s not enough for nonpartisan researchers at RAND to make sweeping conclusions, but it’s certainly suggestive — particularly for policymakers interested in finding solutions to America’s gun violence problem as soon as possible.

Still, the research is far from ideal.

“I don’t want to claim a general truth from this yet,” Morral said. “There’s been so little research that we’re really at the mercy of a few studies. If we could fill every cell in the empty table that we put up, I don’t know if we would see the same thing. But it’s possible.”

It’s worth pointing out here that RAND’s standards are fairly high. For example, it deemed an oft-cited study from Missouri “inconclusive.” That study found that when Missouri repealed its handgun licensing law in 2007, firearm homicides — but not other types of homicides — rose significantly. That certainly suggests that the law led to an immediate increase in homicides.

But RAND’s team was more cautious. For one, Missouri also passed its “stand your ground” law in 2007. Could that explain the rise in homicides, given the evidence that “stand your ground” laws increase violent crime? RAND’s team wasn’t convinced that the Missouri study completely ruled that out, given some of its methodological limitations.

RAND also excluded foreign studies in its official synthesis (although it did include a few foreign studies in brief essays). Some of these studies have compelling evidence for gun restrictions, but RAND’s team argued that the US is too unusual for those other findings to cleanly apply.

Another major area of limitation was the way the science itself was done. Studies try to find what’s known as statistical significance — a mathematical test that helps researchers understand if the effect they found is real or the result of a statistical anomaly. In some cases, this can make it difficult to detect small effects in a scientifically reliable manner.

But in gun policy, a small effect could actually be huge, RAND noted: “For example, a 3-percent reduction in firearm deaths corresponds to 1,000 fewer deaths per year nationally.”

This problem could be overcome with better, more rigorous research. But because the studies were often dealing with limited data and methodologies, many of them couldn’t work through these problems.

Overall, the research that RAND included in its analysis — and deemed reliable — just did not answer many questions to the team’s satisfaction, even if the evidence appears suggestive to you or me.

That’s not to say RAND itself made no conclusions. The think tank found supportive evidence for child access prevention laws reducing suicide and unintentional injuries and deaths. And Morral agreed that the evidence is stronger on background checks and prohibitions associated with mental illness than other gun policies.

But Morral and RAND weren’t comfortable reaching the conclusions that I’ve made after seeing their findings.

Experts on both sides are split on key issues

Unsatisfied with the state of the research, RAND tried to find other sources for answers. So it turned to gun policy experts.

“Given that there’s a fair amount that’s not been studied but matters in these gun policy debates, we believed that policymakers and the public tend to turn to experts — researchers, but also policy analysts at advocacy organizations,” Morral said. “And so we wanted to understand, in the absence of a real consensus around lots of the potential effects of these policies, what do the experts from these two different sides believe those effects are?”

RAND surveyed experts about 15 policies, splitting them into two camps: those who support more restrictive gun policies, and those who support more permissive policies. The full results are available over at RAND’s website, which compiled them into interactive maps.

Here are the survey’s overall findings for firearm homicides:

There’s some obvious disagreement here. Those in the more restrictive camp said that restrictive policies would reduce homicides and more permissive measures would have the opposite effect. Those on the more permissive side said that restrictive policies would have little to no effect, while permissive policies would reduce homicides.

And if you implemented all 15 policies included in the survey, both sides said that gun homicides would fall.

One caveat to the survey results: “These are really back-of-the-envelope calculations,” Morral said. So the exact percentages in the maps should not be taken too literally, but the general trends do at least give some guidance for where experts land on these issues.

There were some big points of agreement: Both sides agreed on the effects of forcing prohibited possessors to give up firearms, expanded mental health prohibitions, and a child access prevention media campaign, concluding that such measures would reduce gun deaths overall.

Another point of agreement: gun suicides, which killed 59 percent more people in the US than gun homicides in 2016. Both sides said, on average, that more restrictive gun policies would reduce firearm suicides — saving at least thousands of lives annually.

The survey also exposed why the different sides disagree on these issues. According to RAND, it’s not that both sides have totally different values — that the permissive side, for example, believes that gun rights are important even if they increase firearm homicides.

Instead, both sides generally agree that the top concerns are public health and safety outcomes. It’s just that they disagree on which policies are more effective at doing that — the permissive side genuinely believes that more permissive gun policies could save lives, while the more restrictive side genuinely believes that more restrictive policies could save lives.

One reason for the difference may be conflicting beliefs about what RAND characterized as a substitution effect. The report explained:

The median respondent with more-permissive regulatory preferences indicated that if a policy successfully reduced a state’s firearm suicides, 90 percent of the prevented suicides would still end as a suicide by some other means. In contrast, the median respondent with more-restrictive regulatory preferences responded that 20 percent would still end as a suicide. They produced the same stark differences in expected substitution of means for firearm homicides: 90 percent for those favoring permissive policies, and 20 percent for those favoring restrictive policies. Those who favor permissive policies view reductions of firearm suicides and homicides as largely futile efforts because these outcomes will continue, largely uninhibited, through other means.

There is some evidence for the more restrictive side here. Consider suicide: It tends to be fairly impulsive, with people often deciding to kill themselves and acting on it within hours. But if they don’t have a gun around, maybe they’ll give up on it altogether. Or if they reach for other means, those are simply going to be much less deadly than a firearm — while the death rate of suicide attempts with guns is above 90 percent, the death rate with other means can drop to below 10 percent.

Regardless, disagreement over the substitution effect helps explain why smart people on both sides can have such different views on gun policy.

More guns, more gun deaths?

Outside of RAND’s research, there has been a lot of evidence that there is a correlation between the availability of guns and more gun deaths.

The research, compiled by the Harvard School of Public Health’s Injury Control Research Center, is pretty clear: After controlling for variables such as socioeconomic factors and other crime, places with more guns have more gun deaths.

“Within the United States, a wide array of empirical evidence indicates that more guns in a community leads to more homicide,” David Hemenway, the Injury Control Research Center’s director, wrote in Private Guns, Public Health.

For example, a 2013 study, led by a Boston University School of Public Health researcher, found that, after controlling for multiple variables, each percentage point increase in gun ownership correlated with a roughly 0.9 percent rise in the firearm homicide rate.

This chart, based on data from researcher Josh Tewksbury, shows the correlation between the number of guns and gun deaths (including homicides and suicides) among wealthier nations, demonstrating that the US in particular has way more guns and gun deaths than its developed peers:

There’s good reason to believe, too, that guns are to blame here. As a breakthrough analysis by UC Berkeley’s Franklin Zimring and Gordon Hawkins in the 1990s found, it’s not even that the US has more crime than other developed countries. This chart, based on data from Jeffrey Swanson at Duke University, shows that the US is not an outlier when it comes to overall crime:

Instead, the US appears to have more lethal violence — and that’s driven in large part by the prevalence of guns.

”A series of specific comparisons of the death rates from property crime and assault in New York City and London show how enormous differences in death risk can be explained even while general patterns are similar,” Zimring and Hawkins wrote. “A preference for crimes of personal force and the willingness and ability to use guns in robbery make similar levels of property crime 54 times as deadly in New York City as in London.”

This is in many ways intuitive: People of every country get into arguments and fights with friends, family, and peers. But in the US, it’s much more likely that someone will get angry at an argument and be able to pull out a gun and kill someone.

RAND takes a more skeptical view. The report argued that it can be hard to disentangle the relationship between more guns and more gun deaths: Is it the abundance of guns leading to more gun deaths, or are people seeing a lot of violence in their communities and reacting to it by stocking up on guns to protect themselves? RAND concluded that there’s just not enough in the research it reviewed to solve this chicken-or-egg scenario.

Still, when RAND’s findings are coupled with other studies we have, there’s some suggestive evidence that easy access to guns is the cause here.

Take the research on gun control measures more broadly, including the foreign studies that RAND left out. A 2016 review of 130 studies in 10 countries published in Epidemiologic Reviews, for example, found that new legal restrictions on owning and purchasing guns tended to be followed by a drop in gun violence. That’s a very strong indicator that it’s the guns that are causing the deaths, since restrictions on guns were followed by a drop in violence. And it comes from very big, reliable meta-analysis.

But as RAND points out, things are less clear when trying to gauge how well specific policies would work in more exact terms, particularly in a US context.

America has a ton of guns — by far more than any other country in the world; it’s an open question how you go about reducing or restricting a stockpile so large.

Would people react violently to a mandatory buyback program similar to the one Australia used to confiscate guns? Would a universal background check system have a big impact, given that people could potentially get around it in the underground markets by simply trading the many firearms already in circulation? If America required licensing and registration for guns, how would it ensure that people who already own weapons get the proper license and registration filed?

The research — including some of RAND’s findings — suggests that these three policies would likely reduce gun deaths to some extent. The question is just how big the effect would be and which policies would be the most effective — a very important question if the US is ever going to bring gun violence down to European levels.

We need more research

All of that is to say that more research is needed. That’s why it’s so bad that the federal government, typically a major facilitator of studies, has for decades refused to fund the research.

Stateline staff writer Michael Ollove, citing a recent study in JAMA, summarized the problem: “The study said that gun violence research received a paltry 1.6 percent of the [federal] funding ($22 million) that would be predicted ($1.4 billion) based on the number of deaths caused by guns — 36,252 in 2015, according to the CDC.”

In the absence of the federal government, RAND is stepping in to try to help out researchers. It put up a database of gun laws, which it hopes will help researchers conduct more accurate analyses of the effects of such laws. It also put out a lengthy, wonky set of recommendations for improving the methodology of research into this area. (Be warned: Unless you’re a statistician or pretty familiar with statistics, much of it will go over your head.)

This all goes to addressing one of the key problems with the lack of federal funding: Since it can be difficult to get money for gun research, it’s hard to build up correct data sets and thorough methodologies when conducting studies. After all, these are things that are not only time-consuming but financially costly as well — it’s not cheap to run big surveys across the country.

“We have not been collecting some of the right data to do these analyses well,” Morral said. So researchers often rely on limited data just because it’s what’s available.

As an example: Several studies looking at the effects of concealed carry permits looked at firearm homicides in general. But what a researcher should ideally want to look at in this area is the effect of firearm homicides outside the home, since that’s where people really use those permits. The data, however, often can’t be broken up like that.

Federal funding could go to surveys and on-the-ground research that could help address the gaps.

The funding is something, however, that pro-gun groups like the NRA have worked against for years. And so far, their tactics have worked — keeping the federal freeze in place. So although there’s evidence that some gun control measures could work, we by and large remain blind to what specific solutions would work best and what all of their effects would be.

Will you support Vox today?

We believe that everyone deserves to understand the world that they live in. That kind of knowledge helps create better citizens, neighbors, friends, parents, and stewards of this planet. Producing deeply researched, explanatory journalism takes resources. You can support this mission by making a financial gift to Vox today. Will you join us?

One-Time Monthly Annual

$5/month

$10/month

$25/month

$50/month

Other

$

Yes, I'll give $5/month

Yes, I'll give $5/month

We accept credit card, Apple Pay, and Google Pay. You can also contribute via