What does Missouri show about the benefits from universal background checks?: Nothing really. Notes on the forthcoming Journal of Urban Health study by the Bloomberg School of Public Health

Feb 18, 2014 | Featured

There has been some attention given yesterday and today about a study coming out from the Bloomberg School of Public Health at Johns Hopkins University:

Repeal of Missouri’s background check law associated with increase in state’s murders

Missouri’s 2007 repeal of its permit-to-purchase (PTP) handgun law, which required all handgun purchasers to obtain a license verifying that they have passed a background check, contributed to a sixteen percent increase in Missouri’s murder rate, according to a new study from researchers with the Johns Hopkins Center for Gun Policy and Research.

The study, to be published in a forthcoming issue of Journal of Urban Health, finds that the law’s repeal was associated with an additional 55 to 63 murders per year in Missouri between 2008 and 2012. State-level murder data for the time period 1999-2012 were collected and analyzed from the Federal Bureau of Investigation’s (FBI) Uniform Crime Reporting (UCR) system. The analyses controlled for changes in policing, incarceration, burglaries, unemployment, poverty, and other state laws adopted during the study period that could affect violent crime. . . . .

The study has gotten breathless coverage from Jonathan Amos at the BBC, Robert VerBruggen with Real Clear Policy, Julia Dahl at CBS, Chelsea Coatney at the PBS Newshour, Steve Benen at MSNBC, and Niraj Chokshi at the Washington Post. The staid Roll Call has a headline “Study Shows Gun Control Works.” Others such as Piers Morgan tweeted about it.

Bottom line: There are a lot of problems with how the tests here are conducted, particularly cherry picking one state when many states have these laws, but even if one accepts the way all those decisions were made, the change in murder rates don’t change what people think that they do. The results are more complicated than simply looking at what the average murder rate before and after rescinding the Missouri law. While it is true that the murder rates in Missouri rose 17 percent relative to the rest of the US after the law was changed, it had actually increased by 32 percent during the five years prior to the change. The question is why the Missouri murder rate was increasing relative to the rest of the US at a slower rate after the change in the law than it did prior to it. The period 1999 to 2012 is picked because that is the period of time examined in Webster’s study. The data is Missouri is available here and for the US here. The US data was net of the numbers for Missouri.

Screen Shot 2014-02-20 at  Thursday, February 20, 2.07 PM

More detailed discussion on how to do empirical tests: You can’t do a study like this on one state over time. There are 17 states with “universal background checks” (see table below for laws up through beginning of 2007). You can’t just pick one state. Let me give you an example. You flip a coin 20 times — ten heads and ten tails. If you specifically picked just five heads from the sample, could you conclude that the coin was biased? Presumably not. There is research on these universal background checks across all the states. Indeed, the third edition of More Guns, Less Crime provided one study on this, and, unlike the Webster study, it shows no benefit in terms of murder rates from these laws. The question the media should ask is: why pick one state when there are so many states with this law? Not only isn’t it the right way to do research, it is pretty obvious to anyone who has looked at the national data that Webster picked that one state to report because it was the one that gave him the result that he wanted. Do studies that collect data on every state take a little more time? Sure, but it isn’t that much more difficult and without it you can’t really determine anything.

Note that the results from More Guns, Less Crime show a slight 2 percent increase in murder rates from “universal background checks,” but the result was not statistically significant.

It is problematic to look at one state for another reason. It is essentially impossible to disentangle all the changes that are occurring on when you are looking at only one experiment. Take another example that has been hotly debated: the Supreme Court’s 1963 Miranda decision requiring that defendants have to be read their rights. Some claim that decision increased crime rates during the 1960s because it made it harder to convict criminals, but there is no way we can really tell. For example, during that term there were three other Supreme Court decisions just on crime issues. All one knows is that the crime rate when up after the decision, but you can’t even tell if it was the Miranda decision or one of the other cases that was responsible, let alone all the other changes that were occurring at that time. The point of using national data, as was used in More Guns, Less Crime, is that by using lots of different states that change their laws in lots of different years you can begin to disentangle these different effects. Unfortunately, you can’t do that with the Miranda ruling because all states were changed in the same way at the same time. With the current research only looking at Missouri, you are producing biased estimates because you are not able to properly disentangle all the changes that were occurring in Missouri.

Other questions are why the paper only examines murder rates and not any other type of violent crime or why only murder rates from 1999 and later are examined. Again, the answer is clear: none of the other violent crime rates, including robbery, showed the change that Webster desired. As to the years studied, one would think that since Missouri’s law went into effect in 1981 researchers would study the impact on crime both when the law was adopted and when it was repealed. Yet, again, the answer for why this wasn’t done is obvious.

For those interested, More Guns, Less Crime also has information on licensing. If you would want to play the game of pointing to one state on the other side of the issue, see Massachusetts for an example (see also here).

Screen Shot 2014-02-18 at  Tuesday, February 18, 1.18 PM
(Click on screen shot to make it bigger.)

johnrlott

3 Comments

  1. Topher

    The author misses one important point the study, they ran a regression to control for other factors that could change the handgun homicide rate. There’s more science here than you’re giving credit. Sure, questions should be asked here but not total dismissal based on hating the results.

  2. johnrlott

    Dear Topher:

    Please note that your point does respond at all to what was posted above. Our post here didn’t discuss the weaknesses in what factors that they tried to account for or didn’t account for. For example, if you disagree, please explain how the variables that they controlled for protects against the concerns over sample selected. Why not look at when the law was adopted in 1981? Shouldn’t it have had the opposite effect then? Why not look at any other states that adopted similar laws? To pick one state for just one of that state’s changes and then to only look at the simple before and after average isn’t serious research.

    • Topher

      You’re also mixing apples and oranges, the study uses gun homicide rate, yet I see in your graphs and texts you’re using the total murder rate. This type of mix-up causes confusion, hurts debate. When you do this I wonder who is analyzing these reports. Someone needs to get the basic facts correct is all. I don’t think the study is a home run for background checks, but I certainly didn’t get that from reading this analysis. Can you at least understand how even if the hand gun murder rate went down, we COULD have a correlation to prove that the repeal of the law caused an increase in that rate? This is what advance statistics allows for, it’s science.

Trackbacks/Pingbacks

  1. Gun control - Page 232 - [...] Cherry picking data is awesome! Also, debunked: crimeresearch.org/2014/02/what-does-missouri-show-about-the-benefits-from-universal-... [...]
  2. Background checks save lives? - Page 2 - [...] 17 states have mandated universal background checks. The rest of the data is discussed here: What does Missouri show…
  3. Evidence That Gun Control Works? | Konfeksiyon Tekstil - [...] Gun rights advocate John Lott pushes back: [...]
  4. Addressing the Johns Hopkins Gun Study | Modern Rifleman - [...] Lott on study: http://crimeresearch.org/2014/02/what-does-missouri-show-about-the-benefits-from-uni... [...]
  5. Lux Libertas – Light and Liberty » Media cherry picks Missouri gun data to make misleading case for more control - [...] it is true that the murder rate in Missouri rose 17 percent relative to the rest of the U.S.…
  6. The Return of Linkorama « Mike's Meandering Mind - [...] This on the other hand, does count as mathematical malpractice. The gun control advocates are hyping a Missouri study…
  7. Missouri Handgun Water Cooler Talk | crimeiny - […] on. This then makes for an apple to oranges comparison already.  No doubt they were influenced by this piece…
  8. Episode 40 | Gun Blog Variety Cast - […] What does Missouri show about the benefits from universal background checks?: Nothing really – http://crimeresearch.org/2014/02/what-does-missouri-show-about-the-benefits-from-uni… […]
  9. Radical Democrats Push Restrictive Federal Handgun Licensing | Patriot Powered News - […] this cabal uses intentionally skewed “data” from the Michael Bloomberg-funded Johns Hopkins Center for Gun Policy and Research to…
  10. NPR and more 'expertise' (stupidity) on guns and violence. - […] lives in Missouri. John Lott put the cherry picking findings of this study in perspective. What does Missouri show…
  11. How doth he lie to us? Let us count the ways… – Gun Nuts Media - […] to make straw purchases at gun stores in various states? The same Bloomberg who has funded specious “studies” to argue…
  12. Grasping at Straws in the North Woods - […] of a permit-to-purchase handgun law was followed by increased murder rates in the state. When the Crime Prevention Research…
  13. Grasping at Straws in the North Woods | Untruth - […] of a permit-to-purchase handgun law was followed by increased murder rates in the state. When the Crime Prevention Research…

Archives