Evaluating new research “Firearm Ownership and Violent Crime in the U.S.: An Ecological Study” By Monuteaux, Lee, Hemenway, Mannix, Fleegler

8 Jul , 2015  

AJPM logo

Live Science describes a new study in the American Journal of Preventive Medicine this way:

Monuteaux and his colleagues wanted to test whether increased gun ownership had any effect on gun homicides, overall homicides and violent gun crimes. They chose firearm robbery and assault, because those crimes are likely to be reported and recorded in the Federal Bureau of Investigation (FBI) Uniform Crime Report.

Along with that FBI data, the researchers gathered gun ownership rates from surveys in the CDC’s Behavioral Risk Factor Surveillance System, an ongoing, nationally representative survey in which participants answered questions about gun ownership in 2001, 2002 and 2004. Using those years and controlling for a slate of demographic factors, from median household income, population density, to age, race and more, the researchers compared crime rates and gun ownership levels state by state.

They found no evidence that states with more households with guns led to timid criminals. In fact, firearm assaults were 6.8 times more common in states with the most guns versus states with the least. Firearm robbery increased with every increase in gun ownership except in the very highest quintile of gun-owning states (the difference in that cluster was not statistically significant). Firearm homicide was 2.8 times more common in states with the most guns versus states with the least. . . .

But their paper (available here) isn’t testing what they claim.  It isn’t testing whether increased gun ownership causes crime rates to increase.  The study is far too simplistic and doesn’t include even the basic control variables that are typically included in other crime studies.

Here is a simple example.  Many people point to the fact that the UK has both a lower homicide and gun ownership rate than the United States.  The claim is often made then that the reason that they have a lower homicide rate is because they have fewer guns.  However, this ignores the fact that the UK homicide rate actually went up after their 1997 handgun ban or after their other very strict earlier gun control regulations.  The UK homicide rate still remained low relative to the US, but it was higher than it otherwise would have been.

The point here is simple: there are lots of reasons why the UK homicide rate was lower than that in the US before they even had gun control.  The question that needs to be asked is how the UK homicide rate changed relative to that in the US after its gun control regulations went into effect.  To do this, you have to control for the fact that the UK had a much lower homicide rate to begin with.  Statistically you do that by having what are called geographic “fixed effects” (dummy variables that pick up the average difference in each jurisdiction that you are examining).  Any test would also do the same thing by year so as to account for any national trends in crime rates.  So, for example, crime might have been falling nationally, but was it falling relatively more in those states that were getting more gun ownership.

The controls that are being used in this paper can’t begin to account for the differences in crime rates.  The regression estimates reported in Table 2 don’t tell what percent of the variation in crime rates are being explained by the variables used in these regressions, but I am willing to bet that it is less than 10 percent.

Yet, this paper in the American Journal of Preventive Medicine doesn’t account for either of these factors.  It is essentially making a purely cross-sectional comparison across states.  On account of that, if they had included Washington, DC in their estimates (with its high crime rates and low gun ownership), it would have dramatically altered their results.

It is easy to see how the results are reversed by just including these state fixed effects.  The first estimate below corresponds to the first estimate reported in Monuteaux, Lee, Hemenway, Mannix, Fleegler’s paper.  The first estimate uses regional fixed effects.  The second includes state fixed effects.  The gunBRFSS variable is their survey measure of gun ownership by state.  In order to get at the nonlinear concern that they raise, I do want academics normally do and have both a linear and a squared version of that variable.  You can see that including the state fixed effects causes their result to go from positive and insignificant to negative and significant.  It seems clear that they broke the survey measurement into arbitrary fifths to help get the result that they wanted.

We used the negative binomial approach used by these authors in these estimates, but there is no truncation issues here and the data fits a weighted least squares estimate.  However, just for the sake of argument we will use the approach that they want used.  (Click on results below to enlarge them.)

See how including fixed state effects reverses their results


While these “fixed effects” will pick up the average differences across places, there are other differences that won’t be accounted for.  One example is they don’t account for differences in any type of law enforcement (e.g., arrest or conviction rates, death penalty, per capita number of police, percent of the population in prison).  The above results include the arrest rates for aggravated assaults, but removing the arrest reduces the statistical significance for both estimates.

The authors ignore previous refereed published research on this in More Guns, Less Crime (University of Chicago Press, 2010) (see discussion here).  In those estimates state level fixed effects are used to pick up the average difference in crime rates.  Possibly citing this research would have forced the authors to explain why they got such difference results.

Finally, there is also the issue of whether people in high crime areas are more likely to get guns for protection.  The question is whether increased gun ownership is caused by higher crime rates or the reverse.

Here are the results with weighted least squares.

Here is what happens if you use weighted least squares and a linear version of the percent of the population with guns and no state fixed effects.

7 Responses

  1. Chris Bennett says:

    Hi there

    I think the Czech Republic might be a decent start to look at the chicken and egg scenario in your last paragraph.

    I have heard that the Czech Republic saw a big increase in the ownership of firearms as people feared crime surges after independence. When it did not materialize as expected, the ownership rates dropped.

  2. gunnut says:

    The gun ownership in Cz. Republic didn’t drop. You can see it here:


    • Dcow777 says:

      You’re right, I doubt few people GAVE UP their guns when crime didn’t balloon upwards but I’m sure that sales to new owners dropped. But nobody can really control for people WITH guns who lie to pollsters and say they don’t own guns.
      Let’s be truthful some stranger contacts you and wants to know if you’ve got a valuable possession that is often stolen and if someone gains entrance to your residence could steal it and injury or kill you with it if you are home? People lie to them about it.
      I know gun owners that won’t Join the NRA because all it takes is the government walking up to their headquarters with a warrant for a “fraud” investigation or the best reason of all that no one dares oppose that someone was “surfing child porn” and the government seizes all the computers for the “case”. Also all the membership info is in those computers too.
      So actual numbers of gun ownership and actual carry rates aren’t actually known. People won’t tell the truth for these studies.

  3. izzyb says:

    You mention the rate of homicide (murder) in the UK and I think that’s a very good example because you can review their crime data going back to the late 1800s online. (Here: https://www.gov.uk/government/statistics/historical-crime-data)

    It is very telling. The rate of murder (Homicide/infanticide) in the UK in 1900 was 0.821 per 100k, in 1950 it was 0.68 per 100k, but in 2000 it was 1.45 per 100k.

    Why hasn’t gun control in UK knocked the murder rate at least below the historical mean rate?

    1900 – 312 murders / 38,000,000 people
    1950 – 346 murders / 51,000,000 people
    2000 – 850 murders / 58,459,000 people

  4. […] See also: Evaluating new research “Firearm Ownership and Violent Crime in the U.S.: An Ecological Study” … […]

  5. We made a short summary of Manuel Eisner research on homicides in 700 (!) years in Europe.

    Laws and harsh punishment were not the reasons why homicides decreased: Violence is correlated with low autonomy, unstable self-esteem, a high dependence on recognition by others, and limited competence in coping with conflict,

    That is one of the reasons why violence concentrate in 5% of all your streets in the US. Violence exists on hot spots. Even when Ceasfire did not work in Chicago. The idea was great: tackle the social network in hot spots, not with harsh punishment but with the idea of a better future for their kids.

    Manuel Eisner is a Professor of Comparative and Developmental Criminology and Deputy Director of the University of Cambridge Institute of Criminology. He studied the history of crime from the thirteenth century until the end of the twentieth. This is a very short summary about his extraordinary work, which is published by the Department of Sociology at the University of Calgary:

  6. […] used in these regressions, but I am willing to bet that it is less than 10 percent." More at Evaluating new research "Firearm Ownership and Violent Crime*in the U.S.: An Ecological Study&q… __________________ What part of "shall not be infringed" do you not […]

Leave a Reply

Your email address will not be published. Required fields are marked *