Problems with the Washington Post’s and Huffington Post’s “more guns, more crime” claims

Nov 16, 2014 | Featured

Washington Post Banner

The media over the last couple of days has been pointing to a revised paper by Abhay Aneja, John J. Donohue III, and Alexandria Zhang that has been floating around in various forms since at least 2009.  Their earlier paper claimed that right-to-carry laws and no impact on murder, rape and robber.  While much the earlier versions of the paper remained unchanged, the authors add a few more years of data for some of their estimates.  Some of the changed estimates are done only using later years and not the whole sample.  On Friday, Christopher Ingraham in the Washington Post felt this unpublished “new research debunks a central thesis of the gun rights movement” of “More Guns, Less Crime” (see also discussions in the Huffington Post, the Raw Story, and other newspapers around the country that picked up the claim).

There are many errors in Ingraham’s article.  For example, “Stanford law professor John Donohue and his colleagues have added another full decade to the analysis.”  Yet, the third edition of “More Guns, Less Crime” has data from 1977 to 2005.  Moody, Marvell, Zimmerman, and Alemante have a new paper earlier this year that looked at data from 1977 to 2006.  Gius (2014) looked at data up through 2009.  Zimmerman (2014) looks at crime data up through 2010.  And note that none of those papers agree with Aneja, Donohue, and Zhang’s conclusion.   Previously even in the Washington Post, Emily Badger’s misleading column also discussed an earlier version of Donohue’s paper with data through 2006 (7/29).

1) The problem with using the 1999 to 2010 period of time is discussed extensively in this short paper available here (download recommended).  The abstract reads as follows: “Unfortunately, many who have examined the impact of so-called “shall-issue” or “right-to-carry” laws assume that the adoption of such laws causes a large, immediate increase in the number of permits. But that is often not the case, for states differ widely as to how easily permits can be obtained. This problem is particularly problematic for studies that have looked at the period after 2000 because the last states to adopt right-to-carry laws had relatively restrictive laws in terms of fees and training requirements.  These restrictions reduce the number of people who obtain concealed handgun permits.  In fact, the share of the adult population with permits increased less during the 1999-2010 period in the states that adopted right-to-carry laws during that period than the states that they are being compared against.”  The point of looking at the change in the number of permits and not just the change in the laws has been made since the 2000 edition of “More Guns, Less Crime.”

2) Take a big claim in the Aneja, Donohue, and Zhang paper:

If we estimate both the dummy and spline models using our preferred specification without state trends for each of these two time periods (overall or after 1999), then we have 4 estimates of the impact of RTC laws for each of seven crime categories (Tables 8a and 11a). In each of the seven crime categories, at least one of these four estimates suggests that RTC laws increase crime at the .10 level of significance, with murder, rape, and larceny estimates reaching significance at the .05 level. These crime increases are substantial, with the dummy variable model for the complete period (Table 8a) suggesting that RTC laws increased every crime category by at least 8 percent, except murder (in that model, murder rose 3 percent but it is not statistically significant). For the post- 1999 regressions, spline estimate (Table 11a) suggests that RTC laws increased the rate of murder by 1.5 percentage points each year (significant at the .05 level). In none of those 28 regressions was there any statistically significant estimate suggesting that RTC laws decreased crime.

Thus, the evidence that RTC laws increase crime is strongest if one accepts the dummy variable model with our preferred specification on state data (the Table 8a and 11a results) and accepts the Wolfers (2006) critique that one should avoid controlling for state trends.

The two sets of estimates can be seen here.

Screen Shot 2014-11-16 at Sunday, November 16, 12.41 PM

In Table 8a, the Aneja, Donohue, and Zhang now argue that a simply “dummy” variable that measures the before-and-after crime rate is their ideal specification.  In Table 11a, they point to a comparison of before-and-after trends.

1) Ironically, Donohue has had a number of papers arguing that the simple “dummy” variable method is misleading.  For example, see the discussion in Ayres and Donohue, Stanford Law Review, 2003.

Screen Shot 2014-11-16 at Sunday, November 16, 1.55 PM

Here is a discussion of the general problem of relying on just before-and-after averages as measured by a simple “dummy” variable (second edition of More Guns, Less Crime in 2000 (see also the same point on pp. 1219-1220 in Ayres and Donohue (2003))).

Screen Shot 2014-11-19 at Wednesday, November 19, 12.04 AM

3)  Despite these flawed estimates that are biased towards finding the results that Aneja, Donohue, and Zhang want, the large majority of their own results don’t show any harm from concealed carry laws.  ADZ claim aggravated assaults provided by far the strongest and consistent support for their claim, but then note even then just 11 of their 28 estimates show an increase in crime and these are the primarily the results just discussed in points (1) and (2).  From ADZ’s abstract:

Screen Shot 2014-11-16 at Sunday, November 16, 2.49 PM

4) Aneja, Donohue, and Zhang falsely claim over and over again that previous research failed to account for crack cocaine. Hypocritically, they then fail to actually use this data and don’t even note that Lott used the very data they recommend for all the years that it is available in the third edition of “More Guns, Less Crime.”   ADZ argue that estimates that don’t account for cocaine “are marred by omitted variable bias,” but then, unlike John Lott’s work, none of ADZ’s regression estimates actually account for cocaine.

Screen Shot 2014-11-16 at Sunday, November 16, 2.07 PM

Now here are the results graphically from “More Guns, Less Crime” (third edition, 2010) that used the data that ADZ say should be used for the years that they say it should be used.

Screen Shot 2014-11-16 at Sunday, November 16, 2.08 PM

Regarding committed variable bias, it is also interesting to note, that Donohue in all of his papers, including ADZ, refuses to account for any other types of gun control laws.  The reason is pretty obvious as he realizes that accounting for any other gun control law in the types of estimates that he runs would not provide support for them.  By contrast, Lott’s “More Guns, Less Crime” accounts for 13 different types of gun control laws to test the sensitivity of the right-to-carry laws.  At least implicitly, these authors are arguing that none of these other gun control variables are important in explaining crime rates.  But Donohue has strongly come out publicly in favor of other gun control laws.  There is no problem with him supporting other gun control laws, but if he really believes that they are useful, not including them implies that his estimates here are misspecified.

5) These media also don’t put this discussion in any type of context.  Not even including refereed research by Moody, Marvell, Zimmerman, and Alemante  (2014), Moody, Lott, and Marvell (2013), and Gius (2014) who claim that right-to-carry laws reduce violent crime and  Zimmerman (2014) who claims no effect but gets the interpretation of his results backwards (see here and it is my understanding that Zimmerman agrees with this), the vast majority of peer-reviewed research finds that right-to-carry laws reduce violent crime.

6) The earlier versions of this paper had serious data and specification errors in them.  A 2012 paper by Carlisle E. Moody, John R. Lott, Jr., Thomas B. Marvell, and Paul R. Zimmerman found the following errors:

• The observations for county 2060 in Alaska are repeated 73 times for 1996.

• The first full year of the shall-issue law for Kansas is coded as 1996 when in fact the law was not passed until 2006.

• The first full year of the shall-issue law for Florida is coded as 1989 when in fact the law was passed in 1987.
• The first full year of the shall-issue law for South Dakota is coded as 1987, however the law was passed in 1985.

• includes 100-140 more counties each year that should have been removed from the set of 3,140 counties because of errors in their reporting.

. . . the errors in ADZ’s data set biased the results towards what they wanted to show. . . .

See also this by Moody, Lott, and Marvell on their errors and their acknowledgement of errors in their paper.




  1. Problems With The Washington & Huffington Post’s 'More Guns, More Crime' Claims - […] DC --( Problems with the Washington Post’s and Huffington Post’s “more guns, more crime” […]
  2. Problems With The Washington & Huffington Post’s ‘More Guns, More Crime’ Claims - TOTAL Firearm Techniques Inc. - […] DC --( Problems with the Washington Post’s and Huffington Post’s “more guns, more crime” […]
  3. Dueling Studies | Shall Not Be Questioned - […] decline in the crack cocaine epidemic. In other words, if you cherry pick your data, Lott is wrong. John…
  4. WLS 067 - Curvy | We Like Shooting Podcast - […] New study saying CCW increases crime is hilariously wrong:… […]
  5. SayUncle » That new bogus gun study - […] John Lott explains why it’s bogus. […]
  6. That new bogus gun research | TiaMart Blog - […] John Lott explains why it’s bogus. […]
  7. Shooting Down the Gun Lobby’s Favorite “Academic”: A Lott of Lies | Armed With Reason - […] Donohue’s work appeared in the Washington Post on November 14th, Lott, in response, sought to discredit the study to…
  8. WOW! The Most Absurd Anti-Gun Tweets, Debunked - - […] site, Crime Prevention Research Center, also responded to the so-called study from Donohue, pointing out […]
  9. SayUncle » About that “study” - […] John Lott tells you why the more guns = more crime study is bunk […]
  10. Quote of the Day: Matt M | Weer'd World - […] Of course this comment, like many others, will go unanswered by the authors, because their concern for public safety…
  11. Response to Evan DeFilippis and Devin Hughes’ claims at “ArmedwithReaon” about my analysis | TiaMart Blog - […] Donohue’s newest piece that DeFilippis and Hughes you possibly can see discussions here, here, and here. I’ll make one ultimate…
  12. Debunking 'Under The Gun' - Documentary or Propaganda? - Page 3 - […] claims? John Lott runs a crime research organization where he publishes a lot of information. Problems with the Washington…