Problems with Public Health Research: Michael Siegel, Craig Ross, and Charles King, “The Relationship Between Gun Ownership and Firearm Homicide Rates in the United States, 1981-2010,” American Journal of Public Health

Dec 8, 2013 | Featured

In a recent issue of the American Journal of Public Health, three researchers examine the relationship between “gun ownership” as measured by the percent of suicides committed with firearms.  There is a significant literature indicating that this measure actually is picking up the relationship between demographic variables and firearm homicides as how people commit suicide varies appreciably with demographic characteristics (gender, age, race, etc.).  The paper is available here and here.

Conclusions. We observed a robust correlation between higher levels of gun ownership and higher firearm homicide rates. Although we could not determine causation, we found that states with higher rates of gun ownership had disproportionately large numbers of deaths from firearm-related homicides. (Am J Public Health. 2013;103:2098–2105. doi:10.2105/AJPH.2013.301409)

The letter I submitted to the journal was rejected, but it is pretty surprising that a count data approach was not used with actual count data and that the regressions didn’t use the most basic controls for panel data: for example, state fixed effects to pick up the average differences across states.

Dear Editor:

Siegel et al. conclude gun ownership is positively related to firearm homicide rates in the US, but they use inappropriate statistical tests and their results are extremely sensitive to the test used.1

Negative binomial regressions use count data, not the rate data these authors use. In addition, overdispersion (the variance greater than the mean) doesn’t imply the distribution is negative binomial in form and it isn’t in this case. Economists and criminologists frequently deal with skewness in homicide rates by running the negative binomial regressions on true count data (not on the rates) or by taking the natural log of the rate.2-6 Performing either procedure dramatically alters their results. The natural log of the rate is normally distributed.

Siegel et al.’s regressions fail to take advantage of the panel nature of their data set. While fixed year effects are accounted for, because of failure to obtain convergence, fixed state effects are virtually never included.

Redoing the negative binomial regression using count data on age-adjusted number of firearm homicides, the variables reported in Tables 2 and 3, and year and state fixed effects, I found the percent of suicides committed with guns (FS/S) significantly positively related to firearm homicides, though the effect is half what they found (a one standard deviation change in FS/S explains just 7.8% of one standard deviation in firearm homicides).

However, replacing firearm homicides with nonfirearm homicides implies an even greater statistically significant negative relationship with FS/S (p=0.002). I found no relationship between total homicides and FS/S. Replacing FS/S with the Behavioral Risk Factor Surveillance System (BRFSS) survey data on gun ownership didn’t produce convergence.

Weighted least squares regressions with the natural logs of the rates raise questions about their measure of gun ownership. I found a one percentage point increase FS/S produced a 1.2% increase in firearm homicides, but the point estimate using the BRFSS survey data implied the same change produced a 1.2% decrease.

The correlation between FS/S and the BRFSS survey data is 0.80. But running the BRFSS data on FS/S after accounting for fixed year and state effects shows an insignificant negative relationship. FS/S is clearly not related to gun ownership when basic fixed effects are accounted for. Instead, firearm suicides appear to be measuring demographic and other variables related to homicides, not gun ownership.

* President, Crime Prevention Research Center.

1 Michael Siegel, Craig S. Ross, and Charles King III, “The Relationship Between Gun Ownership and Firearm Homicide Rates in the United States, 1981–2010,” Am J Public Health 2013; 103(11): 2098-2105.

2 Lott JR. More Guns, Less Crime: Understanding Crime and Gun Control Laws. Chicago, IL: University of Chicago Press; third edition, 2010.

3 Wellford CF, Pepper JV, and Petrie CV. Eds. Firearms and Violence: A critical Review. Washington, DC: National Academies of Science; 2005.

4 Helland E and Tabarrok A. “Using Placebo Laws to Test ‘More Guns, Less Crime’,” ADVANCES ECON. ANALYSIS & POL’Y 2004; 4 (1).

5 Plassmann F and Tideman TN. “Does the Right to Carry Concealed Handguns Deter Countable Crimes? Only a Count Analysis Can Say,” J.L. & ECON. 2001; 44: 771–798.

6 Olsen DE and Maltz MD. “Right-to Carry Concealed Weapons Laws and Homicide in Large U.S. Counties: The Effect on Weapons Types, Victim Characteristics, and Victim- Offender Relationships,” J.L. & ECON. 2001; 44: 747-770.

Here is some additional information.

The natural log of the firearm homicide rate is actually fairly normal.  The closer the data is distributed along the 45 degree line, the closer the distribution is to being normally distributed.

Screen Shot 2013-12-08 at  Sunday, December 8, 1.22 PM

numbernfh = number of non-firearm homicides
numberfh = number of firearm homicides
numberhomicides = number of homicides
gunpercentfss = percentage of suicides committed with firearms
pctblack = percent of the population that is black
gini = gini coefficient to measure inequality
violent = violent crime rate
property = property crime rate
incarceration = incarceration rate
I would have used a different set of control variables, but in a letter limited to 400 words, I had to limit myself to what Siegel, Ross, and King did.  If you believe the specification and results below, they indicate that the more inequality, the lower the number of non-firearm homicides and total homicides.
. xi:xtnbreg numbernfh gunpercentfss pctblack gini violent property incarceration i.year i.state, irr
i.year            _Iyear_1980-2010    (naturally coded; _Iyear_1980 omitted)
i.state           _Istate_1-50        (_Istate_1 for state==Alabama omitted)
note: you are responsible for interpretation of non-count dep. variable
note: _Iyear_2010 dropped due to collinearity
 
Random-effects negative binomial regression     Number of obs      =      1487
Group variable (i): population                  Number of groups   =      1487
 
Random effects u_i ~ Beta                       Obs per group: min =         1
                                                               avg =       1.0
                                                               max =         1
 
                                                Wald chi2(84)      =  42606.65
Log likelihood  = -5992.4647                    Prob > chi2        =    0.0000
 
——————————————————————————
   numbernfh |        IRR   Std. Err.      z    P>|z|     [95% Conf. Interval]
————-+—————————————————————-
gunpercent~s |   .9943015   .0018161    -3.13   0.002     .9907484    .9978674
    pctblack |   1.019502   .0085629     2.30   0.021     1.002857    1.036424
        gini |   .9283215   .0102024    -6.77   0.000      .908539    .9485348
     violent |   1.053231   .0082509     6.62   0.000     1.037183    1.069527
    property |   1.001043   .0013194     0.79   0.429     .9984609    1.003633
incarcerat~n |   .9992555   .0000975    -7.64   0.000     .9990645    .9994465
. . .
. xi:xtnbreg numberfh gunpercentfss pctblack gini violent property incarceration i.year i.state, irr 
i.year            _Iyear_1980-2010    (naturally coded; _Iyear_1980 omitted)
i.state           _Istate_1-50        (_Istate_1 for state==Alabama omitted)
note: _Iyear_2010 dropped due to collinearity
 
Random-effects negative binomial regression     Number of obs      =      1487
Group variable (i): population                  Number of groups   =      1487
 
Random effects u_i ~ Beta                       Obs per group: min =         1
                                                               avg =       1.0
                                                               max =         1
 
                                                Wald chi2(84)      =  63182.72
Log likelihood  = -6628.3832                    Prob > chi2        =    0.0000
 
——————————————————————————
    numberfh |        IRR   Std. Err.      z    P>|z|     [95% Conf. Interval]
————-+—————————————————————-
gunpercent~s |   1.003719   .0017223     2.16   0.031     1.000349      1.0071
    pctblack |   1.075222   .0088181     8.84   0.000     1.058077    1.092645
        gini |   .9930376   .0102524    -0.68   0.499     .9731453    1.013337
     violent |   1.083108   .0082536    10.48   0.000     1.067052    1.099406
    property |   .9996777   .0012666    -0.25   0.799     .9971984    1.002163
incarcerat~n |   .9992185   .0000929    -8.41   0.000     .9990364    .9994005
. . .
. xi:xtnbreg numberhomicides gunpercentfss pctblack gini violent property incarceration i.year i.state, irr 
i.year            _Iyear_1980-2010    (naturally coded; _Iyear_1980 omitted)
i.state           _Istate_1-50        (_Istate_1 for state==Alabama omitted)
note: _Iyear_2010 dropped due to collinearity
 
Random-effects negative binomial regression     Number of obs      =      1487
Group variable (i): population                  Number of groups   =      1487
 
Random effects u_i ~ Beta                       Obs per group: min =         1
                                                               avg =       1.0
                                                               max =         1
 
                                                Wald chi2(84)      =  87246.61
Log likelihood  =  -7055.289                    Prob > chi2        =    0.0000
 
——————————————————————————
numberhomi~s |        IRR   Std. Err.      z    P>|z|     [95% Conf. Interval]
————-+—————————————————————-
gunpercent~s |   1.000437   .0013795     0.32   0.751      .997737    1.003144
    pctblack |   1.052453   .0069153     7.78   0.000     1.038986    1.066094
        gini |   .9700521   .0081124    -3.64   0.000     .9542817    .9860832
     violent |   1.068927   .0065915    10.81   0.000     1.056086    1.081925
    property |   1.000625   .0010272     0.61   0.543     .9986134     1.00264
incarcerat~n |   .9993033    .000076    -9.16   0.000     .9991543    .9994522
. . .

johnrlott

1 Comment

  1. Fallon

    The Controllers found a way to slip their tongue under the tent, PTSD diagnoses, and the nose is following with a snoot full of “We must prevent suicides!”

    Hassan denied 2A rights to as many returning veterans as he possibly could, admittedly, by diagnosing PTSD as liberally as possible. The civilian equivalent is being developed presently through rigged studies such as the one you’ve answered – – – only to be ‘palmed’ by the gate-keepers.

    The dezinformatsiya campaign is building.

    Hope you can find broadcast platforms to expose the truth, blow their “filters” to Hell!

Trackbacks/Pingbacks

  1. Evaluating Mother Jones' "10 Pro-Gun Myths, Shot Down" - Crime Prevention Research Center - [...] Myth #2: Guns don’t kill people—people kill people. A discussion of international data is available here. Chapter 5 of…
  2. More Guns, More Murder - US Message Board - Political Discussion Forum - [...] along the 45 degree line, the closer the distribution is to being normally distributed. Problems with Public Health Research:…
  3. Baron Guns - […] A good start for the problems with the study belowhttp://crimeresearch.org/2013/12/problems-with-public-health-research-michael-siegel… […]

Archives

$166.5 Billion, A Very Conservative Estimate of Criminal Victimization Costs from Criminal Illegal Aliens Released into the U.S., Assuming They Commit Just One Similar Crime in the U.S. to the Most Serious Crime They Committed in Their Home Country

$166.5 Billion, A Very Conservative Estimate of Criminal Victimization Costs from Criminal Illegal Aliens Released into the U.S., Assuming They Commit Just One Similar Crime in the U.S. to the Most Serious Crime They Committed in Their Home Country

We will shortly be releasing a Real Clear Investigations analysis based on estimates developed by the U.S. Department of Justice’s National Institute of Justice (NIJ) suggests that crime by illegal aliens who entered the U.S. by July 21, 2024, cost the country some...