What does Missouri show about the benefits from universal background checks?: Nothing really. Notes on the forthcoming Journal of Urban Health study by the Bloomberg School of Public Health

18 Feb , 2014  

There has been some attention given yesterday and today about a study coming out from the Bloomberg School of Public Health at Johns Hopkins University:

Repeal of Missouri’s background check law associated with increase in state’s murders

Missouri’s 2007 repeal of its permit-to-purchase (PTP) handgun law, which required all handgun purchasers to obtain a license verifying that they have passed a background check, contributed to a sixteen percent increase in Missouri’s murder rate, according to a new study from researchers with the Johns Hopkins Center for Gun Policy and Research.

The study, to be published in a forthcoming issue of Journal of Urban Health, finds that the law’s repeal was associated with an additional 55 to 63 murders per year in Missouri between 2008 and 2012. State-level murder data for the time period 1999-2012 were collected and analyzed from the Federal Bureau of Investigation’s (FBI) Uniform Crime Reporting (UCR) system. The analyses controlled for changes in policing, incarceration, burglaries, unemployment, poverty, and other state laws adopted during the study period that could affect violent crime. . . . .

The study has gotten breathless coverage from Jonathan Amos at the BBC, Robert VerBruggen with Real Clear Policy, Julia Dahl at CBS, Chelsea Coatney at the PBS Newshour, Steve Benen at MSNBC, and Niraj Chokshi at the Washington Post. The staid Roll Call has a headline “Study Shows Gun Control Works.” Others such as Piers Morgan tweeted about it.

Bottom line: There are a lot of problems with how the tests here are conducted, particularly cherry picking one state when many states have these laws, but even if one accepts the way all those decisions were made, the change in murder rates don’t change what people think that they do. The results are more complicated than simply looking at what the average murder rate before and after rescinding the Missouri law. While it is true that the murder rates in Missouri rose 17 percent relative to the rest of the US after the law was changed, it had actually increased by 32 percent during the five years prior to the change. The question is why the Missouri murder rate was increasing relative to the rest of the US at a slower rate after the change in the law than it did prior to it. The period 1999 to 2012 is picked because that is the period of time examined in Webster’s study. The data is Missouri is available here and for the US here. The US data was net of the numbers for Missouri.

Screen Shot 2014-02-20 at  Thursday, February 20, 2.07 PM

More detailed discussion on how to do empirical tests: You can’t do a study like this on one state over time. There are 17 states with “universal background checks” (see table below for laws up through beginning of 2007). You can’t just pick one state. Let me give you an example. You flip a coin 20 times — ten heads and ten tails. If you specifically picked just five heads from the sample, could you conclude that the coin was biased? Presumably not. There is research on these universal background checks across all the states. Indeed, the third edition of More Guns, Less Crime provided one study on this, and, unlike the Webster study, it shows no benefit in terms of murder rates from these laws. The question the media should ask is: why pick one state when there are so many states with this law? Not only isn’t it the right way to do research, it is pretty obvious to anyone who has looked at the national data that Webster picked that one state to report because it was the one that gave him the result that he wanted. Do studies that collect data on every state take a little more time? Sure, but it isn’t that much more difficult and without it you can’t really determine anything.

Note that the results from More Guns, Less Crime show a slight 2 percent increase in murder rates from “universal background checks,” but the result was not statistically significant.

It is problematic to look at one state for another reason. It is essentially impossible to disentangle all the changes that are occurring on when you are looking at only one experiment. Take another example that has been hotly debated: the Supreme Court’s 1963 Miranda decision requiring that defendants have to be read their rights. Some claim that decision increased crime rates during the 1960s because it made it harder to convict criminals, but there is no way we can really tell. For example, during that term there were three other Supreme Court decisions just on crime issues. All one knows is that the crime rate when up after the decision, but you can’t even tell if it was the Miranda decision or one of the other cases that was responsible, let alone all the other changes that were occurring at that time. The point of using national data, as was used in More Guns, Less Crime, is that by using lots of different states that change their laws in lots of different years you can begin to disentangle these different effects. Unfortunately, you can’t do that with the Miranda ruling because all states were changed in the same way at the same time. With the current research only looking at Missouri, you are producing biased estimates because you are not able to properly disentangle all the changes that were occurring in Missouri.

Other questions are why the paper only examines murder rates and not any other type of violent crime or why only murder rates from 1999 and later are examined. Again, the answer is clear: none of the other violent crime rates, including robbery, showed the change that Webster desired. As to the years studied, one would think that since Missouri’s law went into effect in 1981 researchers would study the impact on crime both when the law was adopted and when it was repealed. Yet, again, the answer for why this wasn’t done is obvious.

For those interested, More Guns, Less Crime also has information on licensing. If you would want to play the game of pointing to one state on the other side of the issue, see Massachusetts for an example (see also here).

Screen Shot 2014-02-18 at  Tuesday, February 18, 1.18 PM
(Click on screen shot to make it bigger.)

, ,

12 Responses

  1. […] Cherry picking data is awesome! Also, debunked: crimepreventionresearchcenter.org/2014/02/what-does-missouri-show-about-the-benefits-from-universal-… […]

  2. […] 17 states have mandated universal background checks. The rest of the data is discussed here: What does Missouri show about the benefits from universal background checks?: Nothing really, The fo… NRA Lifetime Member – ΜΟΛΩΝ ΛΑΒΕ. Reply […]

  3. […] Gun rights advocate John Lott pushes back: […]

  4. […] it is true that the murder rate in Missouri rose 17 percent relative to the rest of the U.S. in the five years after 2007, it had actually incre…. The question is why the Missouri murder rate was increasing relative to the rest of the United […]

  5. […] This on the other hand, does count as mathematical malpractice. The gun control advocates are hyping a Missouri study that shows a rise in murder rate after a change in the gun control laws. However, in doing so they are ignoring data from 17 other states, data on all other forms of violent crime and data from Missouri that showed a steep rise in the murder rate before the laws were changed. They are picking a tiny slice of data to make a huge claim. Disgraceful. And completely expected from the gun-grabbers. […]

  6. Topher says:

    The author misses one important point the study, they ran a regression to control for other factors that could change the handgun homicide rate. There’s more science here than you’re giving credit. Sure, questions should be asked here but not total dismissal based on hating the results.

  7. johnrlott says:

    Dear Topher:

    Please note that your point does respond at all to what was posted above. Our post here didn’t discuss the weaknesses in what factors that they tried to account for or didn’t account for. For example, if you disagree, please explain how the variables that they controlled for protects against the concerns over sample selected. Why not look at when the law was adopted in 1981? Shouldn’t it have had the opposite effect then? Why not look at any other states that adopted similar laws? To pick one state for just one of that state’s changes and then to only look at the simple before and after average isn’t serious research.

    • Topher says:

      You’re also mixing apples and oranges, the study uses gun homicide rate, yet I see in your graphs and texts you’re using the total murder rate. This type of mix-up causes confusion, hurts debate. When you do this I wonder who is analyzing these reports. Someone needs to get the basic facts correct is all. I don’t think the study is a home run for background checks, but I certainly didn’t get that from reading this analysis. Can you at least understand how even if the hand gun murder rate went down, we COULD have a correlation to prove that the repeal of the law caused an increase in that rate? This is what advance statistics allows for, it’s science.

  8. […] on. This then makes for an apple to oranges comparison already.  No doubt they were influenced by this piece and others who have been echoing this […]

  9. […] What does Missouri show about the benefits from universal background checks?: Nothing really – http://crimepreventionresearchcenter.org/2014/02/what-does-missouri-show-about-the-benefits-from-uni… […]

  10. […] this cabal uses intentionally skewed “data” from the Michael Bloomberg-funded Johns Hopkins Center for Gun Policy and Research to provide a […]

Leave a Reply

Your email address will not be published. Required fields are marked *