John Lott was interviewed by America’s First Freedom about the problems with some recent so-called “public health” research. The first question and answer is available here:
America’s 1st Freedom: What were the biggest things that the authors of the study on police deaths in the American Journal of Public Health did wrong?
John Lott: The main problem with the study was that they didn’t use traditional control variables that people who do this type of research employ on so-called panel data (where you look at a lot of different places over time). This study never explains why they aren’t doing the experiment the way that it would normally be done, nor do they talk about whether their unusual way of conducting the study makes a difference. Nor does the study even mention previous research that has been done on this very question that comes to the exact opposite conclusion. You would think that if a researcher is going to do a study in a very different way from others, he would provide some explanation for that.
One central problem can be explained this way. Suppose a state passes a gun-control law at the same time that crime rates are falling nationally. It would be a mistake to attribute the overall drop in national crime rates to the law that got passed. To account for that concern, researchers normally see whether the drop in the crime rate for the state that had the change is greater or less than the overall national change. This study doesn’t account for that basic control.
The rest of the interview is available here.