Responding to John Donohue’s responses to our evaluation of his new study

Jul 12, 2017 | Featured

John Donohue responded to our comments on our website and in an op-ed at Fox News.  Below we go through and respond to the points that he raised.  Amusingly, the author of the original piece in Michael Bloomberg’s “The Trace” emailed Donohue as soon as my comments came out.  While The Trace pretends to be a regular news outlet, Mr. Yablon never contacted me about Donohue’s claim to have debunked my research.

Mr. Yablon then posted Donohue’s responses to our critique of the John Donohue, Abhay Aneja, and Kyle Weber’s paper.

What Donohue leaves out is that the sentence after the end of the sentence he quotes says: “However, as the shall-issue law impact is being identified from only eight state changes in the data, it is difficult to give any strong causal interpretation to these estimates.”  More importantly, the whole point of Zimmerman’s paper is to use an instrument and to show that without that instrument you can get biased results.  In Table 4 in Zimmerman’s paper, there is no statistically significant result for the right-to-carry law variable.  As has been pointed out before by John Lott, there is a good reason to believe that estimates that look at only later years and do not account for differences in right-to-carry laws are biased towards not finding a benefit.

Our question to John Donohue is why didn’t you include the next sentence in your quote?  Nor does Donohue reference a later paper by Zimmerman and some co-authors that found: “the evidence shows that RTC laws are socially beneficial.

Donohue also very selectively quotes the Durlauf, Navarro, and Rivers.  The end of the first paragraph in their conclusion is pretty clear: “Overall, we conclude that the evidence that shall-issue right-to-carry laws generate either an increase or decrease in crime on average seems weak.

In our previous discussions of Donohue’s claims, we provided systematic data sources from state police bureaus in eight states.  We also previously noted the two news stories that Donohue provided.  Here he provides two more cases from the last couple of weeks.  This is a very strange debating technique in that no one is claiming that the 16 million permit holders in the US never commit crimes.  The point is that they very rarely commit any crime, and very rarely commit firearm related offenses.

Donohue’s response is simply not honest here.  There is a great deal of arbitrariness in terms how the synthetic approach is set up.  For example, is the window that they should have used to compare states 3 years, 5 years, 8 years, 10 years, 15 years or some other length?  By saying that you will only create a synthetic using states that have not adopted a right-to-carry law for at least 10 years after the target state has adopted the law, you are throwing out a lot of comparison states and a lot of information.  There is no theory that says what the right window is.  The choice of control variables has a big effect on what states will be used to create the synthetic state.  Donohue has a reputation for only reporting a very tiny percent of all the regressions that he runs.  If you are willing to run thousands of regressions, you can end up finding some set of states that are used to create the synthetic states that will give you any result that you want.

Donohue ignores the concerns that have been raised for over a decade about the changing type of states that adopt right-to-carry laws.  Later adopting states have much more restrictive regulations in obtaining a concealed handgun permit.  For example, Illinois, the last state to adopt a right-to-carry law, requires that you have to get 16 hours of training and the fee is $150.  The earliest states had no training requirements and very low fees.  If you include these later years (even worse if you look at years from 2000 to 2014), you are comparing restrictive right-to-carry states that changed their laws during the sample period to earlier states that have extremely liberal right-to-carry laws.   Lott has shown that this problem can actually reverse how you interpret the results from these regressions.




  1. Sajmon

    “There is a great deal of arbitrariness in terms how the synthetic approach is set up.”

    I think we all remember Webster-Vernick’s study about Conneticut and gun homicides. The quality of that research was abysmal. Their whole claim that Connecticut’s PTP law reduced gun homicides was not based on any fall in Connecticut’s rate but on an increase in gun homicide rate in Rhode Island. So the real phenomenon to be explained here was not why Connecticut’s gun homicides went down, but why Rhode Island’s went up unusually for part of the period.

    The “synthetic Connecticut” to which they compare the real Connecticut was derived from a “weighted combination of states that exhibits homicide trends most similar to Connecticut’s prior to the law’s implementation.” It ended up being 72 percent Rhode Island, with a mix of Maryland and California tossed in. Curiously, the synthetic-Connecticut used to compare the non-gun homicides was completely different than the mostly-Rhode Island one used for gun homicides; it was mostly New Hampshire.

    As I said – abysmal.

  2. Blake

    What beyond the overall national statistics need be evaluated? As RTC swept the nation, violent crime including murder plummeted by half. Why even be dragged into the weeds of cherry picked regressions and synthesized states.

    Just keep showing the plot of murder rate versus RTC permit holders, big and bold, and in bright vibrant colors!

    The deceivers are always going to deceive. The people looking for the truth appreciate simple basic facts.

  3. Jim McNamara

    Gun control researchers have a proclivity toward reaching preordained, flawed conclusions. Bias is a precusrosr that fuels their personal agenda. Their result-driven “research” can best be characterized as a contemptuous disregard of the scientific method in order to achieve an objective … the vilification of gun ownership. Evidence, facts, and statistics are immaterial to persons who are emotionally and passionately committed to fallacious beliefs. Information is discarded, contradictions ignored, and reality twisted and distorted in order to sustain untenable convictions.


  1. What John Lott Gets Right | Best Worldwide News - […] is criticized by a GVP researcher, he responds at length on his website and either reproduces or references the…
  2. What John Lott Gets Right | News - Zedid - […] is criticized by a GVP researcher, he responds at length on his website and either reproduces or references the…