‘Gun rights’ advocates quickly remind us that guns don’t kill people but people do. In other words, there are many law abiding and responsible gun owners who pose no problem at all. The problems involving guns are really problems of criminals and perhaps to a lesser extent, the mentally ill. A recent article published in the American Journal of Medicine tries to bring the ‘voice of reason’ into this debate.
The article, ‘Gun Ownership and Firearm-related Deaths’ is by Sripal Bangalore and Franz H. Messerli. Although it isn’t a very long article, it’s still too long for me to go over every detail of it. But here is a quick summary.
The article uses data on 27 ‘developed’ countries which aren’t currently involved in civil war. These data include different types of firearm related deaths due to various causes (self- imposed intentional shotgun death, self-imposed unintentional shot gun death, etc.), crime rate, and mental illness burden in a given country. The main statistical questions relevant to the gun control debate addressed in the article were these:
1) Is there an association between gun ownership and firearm-related deaths?
2) Is there an association between mental illness and firearm-related deaths?
3) Is there an association between gun ownership and crime?
The main findings of the study were as follows:
1) Gun ownership is positively correlated with firearm-related deaths.
2) Gun ownership isn’t correlated with crime rate.
3) Mental illness is correlated with firearm-related deaths.
The message for the gun control debate that Bangalore and Messerli take from these findings is that the notion, often heard from gun rights advocates, that countries with higher gun ownership are safer than those with low gun ownership is debunked.
I applaud the attempt on the part of Bangalore and Messerli to bring data to bear on an often contentious debate. But someone familiar with statistics and econometrics, that branch of statistics used primarily in economics, might wonder how much light is shed by Bangalore and Messerli’s work. This is because to a statistically or econometrically minded observer their study suffers from several problems.
One is the issue of focusing only on the 27 developed countries. As far as I could tell, they provided no reason for doing this. But whatever the reason, this raises doubts about their findings.
According to the World Atlas, there are between 189 and 196 countries in the world. The lack of precision here is due to political questions having to do with the status of places like Taiwan. But 189 and 196 are quite a bit larger than 27. Were the 152 to 158 countries excluded from their analysis all experiencing civil war? In any case if data on the countries left out of Bangalore’s and Messerli’s analysis were added to it, their findings might change drastically. So any conclusions drawn from their analysis arguably only applies to developed countries not experiencing civil war instead of countries in general.
Bangalore’s and Messerli’s choice of 27 countries and their use of p<.05 to denote statistical significance raises another question about their analysis. The p<.05 business is related to a branch of statistics called inferential statistics. Leaving out many of the details, the agenda of inference is to generalize from a subset of cases to a bigger set that contains that subset. The subset is called a sample and the bigger set is called a population. When these generalizations can be defended, using methods I won’t go into, we say there’s a statistically significant finding. But one tradition of statistical theory, called design based inference, tells us that generalizing findings from a sample to a population can only be defended if that sample was chosen in a certain way. Assuming Bangalore and Messerli are working out of the design based tradition, are the 27 countries meant to be a sample of the 189 or so countries? And if so, the question of how they were chosen comes up again. Because if this sample wasn’t chosen randomly, arguably we just don’t know what to make of their findings regarding statistical significance.
Now design based inference isn’t the only inferential tradition in statistics. There is model based inference as well. The idea here is not that one is using a sample to generalize to a population but to generalize to a model that is believed to have generated the data. But going this route requires one to explicitly state the model one has in mind and make a case for why that model and not some other one. Perhaps this is what Bangalore and Messerli have in mind. But I couldn’t find anywhere in the paper where they explicitly tell us this.
Another set of problems has to do with the possible directions of causality in Bangalore’s and Messerli’s models. They are very careful to refer to their findings as correlations instead of causal relationships. Here is an example:
Although correlation is not synonymous with causation, it seems conceivable that abundant gun availability facilitates firearm related deaths. Conversely, high crime rates may instigate widespread anxiety and fear, thereby motivating people to arm themselves and give rise to increased gun ownership, which, in turn, increases availability (p. 875).
They seem to have learned well the lesson that correlation doesn’t imply causality. But what people often forget is that neither does correlation imply lack of causality. The real point is that correlation by itself tells you nothing about whether variation in one factor is a cause of variation in another. So let’s, for the sake of discussion, follow Bangalore and Messerli and be open to the possibility of a causal interpretation of their statistical models. Looked at in this way, there arises another problem in their multivariate findings on page 875 of their paper.
Here Bangalore and Messerli report findings from a regression model with firearm-related deaths the dependent variable and gun ownership and mental illness the independent variables. The logic seems to be that gun ownership and mental illness might cause firearm-related deaths. But as they allude to in the quote above, firearm-related deaths as a result of high crime may also cause high levels of gun ownership. This is because people may try to protect themselves from a high level of crime related gun deaths by buying guns.
There are statistical models, with names such as two-stage least squares and nonrecursive path models, that can be used to represent such bidirectional relationships. Bangalore and Messerli don’t take advantage of these models and by not doing so their estimates of the effects of mental illness and gun ownership on firearm-related deaths might be biased. In layperson’s terms, their estimates of the causal effects (or associations if you prefer) of gun ownership and mental illness on firearm-related deaths might be way off base. It may be true that countries with higher gun ownership aren’t any safer than those with low gun ownership. But we should be careful about concluding this on the basis of Bangalore and Messerli’s study.
