The end of last week a study in the American Journal of Public Health claimed that there were more police feloniously killed in states that had more guns. The study got extensive news coverage at the TV networks such as NBC News, newspapers such as the Chicago Sun-Times, and international coverage such the UK Guardian. Yet, it took just a couple of minutes to read the paper and realize that the empirical work was done in a very non-standard way. There is a big benefit to using so-called panel data, where you can more accurately account for differences in crime rates across states or over time. This method is called “fixed effects.” Strangely the authors, David Swedler, Molly M. Simmons, Francesca Dominici, and David Hemenway, only control for the differences across states and not over time.
A couple of simple examples show why other studies on crime take into account these factors
Take the differences across places. Many people point out that the UK has both a lower gun ownership rate and a lower homicide rate than the US. Some use this to claim that gun control causes crime rates to fall. But the homicide rate actually went up by 50 percent in the eight years after the 1997 handgun ban went into effect. The homicide rate was still lower than that in the US, but there were lots of reasons it was lower to begin with, not the handgun ban.
The same point applies over time. Suppose a state passes a gun control law at the same time that crime rates are falling nationally. It would be a mistake to attribute the overall drop in national crime rates to the law that got passed. To account for that concern, researchers normally see whether the drop in crime rate for the state that had the change is greater or less than the overall national change.
Unfortunately, the American Journal of Public Health study doesn’t account for this last concern, and their results are extremely sensitive to this error. Here is a simple version of their regression explaining the number of police officers feloniously killed over the years from 1996 to 2013, but without accounting for the time effects that we just discussed. Before going through the results, the media coverage of this study is incorrect in claiming that more guns are associated with more police killings. What the journal article actually measures is not the gun ownership, but the percentage of suicides committed with guns (fsdsuicides). This regression looks at the total number of police feloniously murdered with their measure of “gun ownership” and the number of police officers. In this regression it appears that a one percentage point increase in the percent of suicides committed with guns increase, there is a significant 1.8 percent increase in the total number of police killed.
Just accounting for the changes in crime rates over time, completely reverses the claims made in the American Journal of Public Health. It now appears that a one percentage point increase in the percent of suicides committed with guns increase, there is a significant 2.25 percent decrease in the total number of police killed.
Including the other variables that they have used in their estimates along with the year fixed effects produce an even more statistically significant drop in police felonious killings. In that case, a one percentage point increase in the percent of suicides committed with guns increase, there is a significant 3.7 percent decrease in the total number of police killed.