John Donohue responded to our comments on our website and in an op-ed at Fox News. Below we go through and respond to the points that he raised. Amusingly, the author of the original piece in Michael Bloomberg’s “The Trace” emailed Donohue as soon as my comments came out. While The Trace pretends to be a regular news outlet, Mr. Yablon never contacted me about Donohue’s claim to have debunked my research.
Mr. Yablon then posted Donohue’s responses to our critique of the John Donohue, Abhay Aneja, and Kyle Weber’s paper.
What Donohue leaves out is that the sentence after the end of the sentence he quotes says: “However, as the shall-issue law impact is being identified from only eight state changes in the data, it is difficult to give any strong causal interpretation to these estimates.” More importantly, the whole point of Zimmerman’s paper is to use an instrument and to show that without that instrument you can get biased results. In Table 4 in Zimmerman’s paper, there is no statistically significant result for the right-to-carry law variable. As has been pointed out before by John Lott, there is a good reason to believe that estimates that look at only later years and do not account for differences in right-to-carry laws are biased towards not finding a benefit.
Our question to John Donohue is why didn’t you include the next sentence in your quote? Nor does Donohue reference a later paper by Zimmerman and some co-authors that found: “the evidence shows that RTC laws are socially beneficial.”
Donohue also very selectively quotes the Durlauf, Navarro, and Rivers. The end of the first paragraph in their conclusion is pretty clear: “Overall, we conclude that the evidence that shall-issue right-to-carry laws generate either an increase or decrease in crime on average seems weak.”
In our previous discussions of Donohue’s claims, we provided systematic data sources from state police bureaus in eight states. We also previously noted the two news stories that Donohue provided. Here he provides two more cases from the last couple of weeks. This is a very strange debating technique in that no one is claiming that the 16 million permit holders in the US never commit crimes. The point is that they very rarely commit any crime, and very rarely commit firearm related offenses.
Donohue’s response is simply not honest here. There is a great deal of arbitrariness in terms how the synthetic approach is set up. For example, is the window that they should have used to compare states 3 years, 5 years, 8 years, 10 years, 15 years or some other length? By saying that you will only create a synthetic using states that have not adopted a right-to-carry law for at least 10 years after the target state has adopted the law, you are throwing out a lot of comparison states and a lot of information. There is no theory that says what the right window is. The choice of control variables has a big effect on what states will be used to create the synthetic state. Donohue has a reputation for only reporting a very tiny percent of all the regressions that he runs. If you are willing to run thousands of regressions, you can end up finding some set of states that are used to create the synthetic states that will give you any result that you want.
Donohue ignores the concerns that have been raised for over a decade about the changing type of states that adopt right-to-carry laws. Later adopting states have much more restrictive regulations in obtaining a concealed handgun permit. For example, Illinois, the last state to adopt a right-to-carry law, requires that you have to get 16 hours of training and the fee is $150. The earliest states had no training requirements and very low fees. If you include these later years (even worse if you look at years from 2000 to 2014), you are comparing restrictive right-to-carry states that changed their laws during the sample period to earlier states that have extremely liberal right-to-carry laws. Lott has shown that this problem can actually reverse how you interpret the results from these regressions.