Do guns or people kill people?

Author: Michael A. Lewis

In mid-September, the US witnessed yet another mass shooting. This time a gunman made his way onto a military base in the Washington D.C. area and killed about a dozen people. When these events occur in the US there is almost a ritualized response. ‘Gun control’ advocates quickly tell us that such cases are an indication of how dangerous guns are, especially in the hands of the mentally ill, and that we need much more strict control over their availability.

‘Gun rights’ advocates quickly remind us that guns don’t kill people but people do. In other words, there are many law abiding and responsible gun owners who pose no problem at all. The problems involving guns are really problems of criminals and perhaps to a lesser extent, the mentally ill. A recent article published in the American Journal of Medicine tries to bring the ‘voice of reason’ into this debate.

The article, ‘Gun Ownership and Firearm-related Deaths’ is by Sripal Bangalore and Franz H. Messerli. Although it isn’t a very long article, it’s still too long for me to go over every detail of it. But here is a quick summary.

The article uses data on 27 ‘developed’ countries which aren’t currently involved in civil war. These data include different types of firearm related deaths due to various causes (self- imposed intentional shotgun death, self-imposed unintentional shot gun death, etc.), crime rate, and mental illness burden in a given country. The main statistical questions relevant to the gun control debate addressed in the article were these:

1) Is there an association between gun ownership and firearm-related deaths?

2) Is there an association between mental illness and firearm-related deaths?

3) Is there an association between gun ownership and crime?

The main findings of the study were as follows:

1) Gun ownership is positively correlated with firearm-related deaths

2) Gun ownership isn’t correlated with crime rate

3) Mental illness is correlated with firearm-related deaths.

The message for the gun control debate that Bangalore and Messerli take from these findings is that the notion, often heard from gun rights advocates, that countries with higher gun ownership are safer than those with low gun ownership is debunked.          

I applaud the attempt on the part of Bangalore and Messerli to bring data to bear on an often contentious debate. But someone familiar with statistics and econometrics, that branch of statistics used primarily in economics, might wonder how much light is shed by Bangalore and Messerli’s work. This is because to a statistically or econometrically minded observer their study suffers from several problems.

One is the issue of focusing only on the 27 developed countries. As far as I could tell, they provided no reason for doing this. But whatever the reason, this raises doubts about their findings.

According to the World Atlas, there are between 189 and 196 countries in the world. The lack of precision here is due to political questions having to do with the status of places like Taiwan. But 189 and 196 are quite a bit larger than 27. Were the 152 to 158 countries excluded from their analysis all experiencing civil war? In any case if data on the countries left out of Bangalore’s and Messerli’s analysis were added to it, their findings might change drastically. So any conclusions drawn from their analysis arguably only applies to developed countries not experiencing civil war instead of countries in general.

Bangalore’s and Messerli’s choice of 27 countries and their use of p<.05 to denote statistical significance raises another question about their analysis. The p<.05 business is related to a branch of statistics called inferential statistics. Leaving out many of the details, the agenda of inference is to generalize from a subset of cases to a bigger set that contains that subset. The subset is called a sample and the bigger set is called a population. When these generalizations can be defended, using methods I won’t go into, we say there’s a statistically significant finding. But one tradition of statistical theory, called design based inference, tells us that generalizing findings from a sample to a population can only be defended if that sample was chosen in a certain way. Assuming Bangalore and Messerli are working out of the design based tradition, are the 27 countries meant to be a sample of the 189 or so countries? And if so, the question of how they were chosen comes up again. Because if this sample wasn’t chosen randomly, arguably we just don’t know what to make of their findings regarding statistical significance.   

Now design based inference isn’t the only inferential tradition in statistics. There is model based inference as well. The idea here is not that one is using a sample to generalize to a population but to generalize to a model that is believed to have generated the data. But going this route requires one to explicitly state the model one has in mind and make a case for why that model and not some other one. Perhaps this is what Bangalore and Messerli have in mind. But I couldn’t find anywhere in the paper where they explicitly tell us this.

Another set of problems has to do with the possible directions of causality in Bangalore’s and Messerli’s models. They are very careful to refer to their findings as correlations instead of causal relationships. Here is an example:

Although correlation is not synonymous with causation, it seems conceivable that abundant gun availability facilitates firearm related deaths. Conversely, high crime rates may instigate widespread anxiety and fear, thereby motivating people to arm themselves and give rise to increased gun ownership, which, in turn, increases availability (p. 875).

They seem to have learned well the lesson that correlation doesn’t imply causality. But what people often forget is that neither does correlation imply lack of causality. The real point is that correlation by itself tells you nothing about whether variation in one factor is a cause of variation in another. So let’s, for the sake of discussion, follow Bangalore and Messerli and be open to the possibility of a causal interpretation of their statistical models. Looked at in this way, there arises another problem in their multivariate findings on page 875 of their paper.

Here Bangalore and Messerli report findings from a regression model with firearm-related deaths the dependent variable and gun ownership and mental illness the independent variables. The logic seems to be that gun ownership and mental illness might cause firearm-related deaths. But as they allude to in the quote above, firearm-related deaths as a result of high crime may also cause high levels of gun ownership. This is because people may try to protect themselves from a high level of crime related gun deaths by buying guns.

There are statistical models, with names such as two-stage least squares and nonrecursive path models, that can be used to represent such bidirectional relationships. Bangalore and Messerli don’t take advantage of these models and by not doing so their estimates of the effects of mental illness and gun ownership on firearm-related deaths might be biased. In layperson’s terms, their estimates of the causal effects (or associations if you prefer) of gun ownership and mental illness on firearm-related deaths might be way off base. It may be true that countries with higher gun ownership aren’t any safer than those with low gun ownership. But we should be careful about concluding this on the basis of Bangalore and Messerli’s study.

Bookmark and Share

Comment on this article

Submit your comment
  1. Image of unique ID

Comments

James Cavenaugh

This would be a nice letter to the editor of The Journal of American Medicine.

reply to this comment

Michael M.

Quote:

Quote: WB 


I take your point about cross national comparisons among similar countries. This is why I said that arguably their findings only apply to "developed" countries not involved in a civil war. The problem is their conclusion that the notion that countries with high gun ownership are safer than those with low gun ownership is debunked. If anything they've only debunked it for developed countries not countries in general. It may very well be that the relationship between gun ownership and gun deaths is quite different in developing nations. In short, their conclusion about what they've debunked seems too broad. 

One way to look at the question of whether their finding holds for developing nations as well would have been to include such nations in the study and create a binary predictor variable for a nation's development status. They could then have looked at the gun ownership/gun death relationship adjusting for development status.   

"One is the issue of focusing only on the 27 developed countries."

This isn't a problem at all. It is common to limit cross-national comparisons to countries that have similar levels of economic and political development (such as advanced democracies). In this way, all sorts of factors (such as corruption, state weakness, underemployment, and the like) don't complicate the results.

The fact that, say, Canada has far fewer fire-arm deaths than Nigeria is likely unrelated to the number of guns in each respective country. The level of state development and rule of law are likely the crucial factors here. But such differences don't exist between advanced democracies. They all enjoy similarly high levels of prosperity and political stability, thus allowing researchers to examine and isolate the effect of gun quantity.

However, the issue of endogeneity that you note is a problem.

 

Enter your comment

 

I agree with the reasons for the country sampling, otherwise other variables become too important. And all we are interested in, in reality, is variability across developed world (btw, the idea of a "developed country involved in a civil war" is somewhat humorous.) However, even in this context, the notion of "country" is artificial. As is obvious, USA is hugely diverse, culturally and economically, compared, say, to Canada or Austria. In this sense, the US is not a "country". You have to control for this variability, which is not impossible.

reply to this comment

Michael L.

Quote: WB 


I take your point about cross national comparisons among similar countries. This is why I said that arguably their findings only apply to "developed" countries not involved in a civil war. The problem is their conclusion that the notion that countries with high gun ownership are safer than those with low gun ownership is debunked. If anything they've only debunked it for developed countries not countries in general. It may very well be that the relationship between gun ownership and gun deaths is quite different in developing nations. In short, their conclusion about what they've debunked seems too broad. 

One way to look at the question of whether their finding holds for developing nations as well would have been to include such nations in the study and create a binary predictor variable for a nation's development status. They could then have looked at the gun ownership/gun death relationship adjusting for development status.   

"One is the issue of focusing only on the 27 developed countries."

This isn't a problem at all. It is common to limit cross-national comparisons to countries that have similar levels of economic and political development (such as advanced democracies). In this way, all sorts of factors (such as corruption, state weakness, underemployment, and the like) don't complicate the results.

The fact that, say, Canada has far fewer fire-arm deaths than Nigeria is likely unrelated to the number of guns in each respective country. The level of state development and rule of law are likely the crucial factors here. But such differences don't exist between advanced democracies. They all enjoy similarly high levels of prosperity and political stability, thus allowing researchers to examine and isolate the effect of gun quantity.

However, the issue of endogeneity that you note is a problem.

 

Enter your comment

reply to this comment

WB

"One is the issue of focusing only on the 27 developed countries."

This isn't a problem at all. It is common to limit cross-national comparisons to countries that have similar levels of economic and political development (such as advanced democracies). In this way, all sorts of factors (such as corruption, state weakness, underemployment, and the like) don't complicate the results.

The fact that, say, Canada has far fewer fire-arm deaths than Nigeria is likely unrelated to the number of guns in each respective country. The level of state development and rule of law are likely the crucial factors here. But such differences don't exist between advanced democracies. They all enjoy similarly high levels of prosperity and political stability, thus allowing researchers to examine and isolate the effect of gun quantity.

However, the issue of endogeneity that you note is a problem.


reply to this comment

Bill Zimmerly

> "The message for the gun control > debate that Bangalore and > Messerli take from these > findings is that the notion, > often heard from gun rights > advocates, that countries > with higher gun ownership > are safer than those > with low gun ownership > is debunked." I am not familiar with the data that they used, but if they left out data where armed citizens were successful in defending themselves (as I was) against criminals, then the conclusion labeled as "debunked" might not be so. I'm always having to point out to people who don't take the importance of the second amendment seriously that the difference can be seen in the headlines: UK: "73 year old man found dead after a violent home invasion" http://www.bbc.co.uk/news/uk-england-london-20280157 -instead of- USA: "Woman, 77, shoots at would-be burglar" http://www.wsbtv.com/news/news/local/woman-77-shoots-would-be-burglar/nTN58/ People really -NEED- to use their brain and read between the lines! These two articles show a living 77 year old woman who successfully defended herself and her progeny -WHEREAS- the 73 year old British man could not. Likely because he was unarmed and didn't even have a fighting chance against the home invaders. It would seem from that small sample (and the even larger sample of people who are like me) that GUNS SAVE a lot more innocent lives than are lost because of accidents, suicide, or mental illness.

reply to this comment

D. O. Kasdan

I will agree with Bangalore & Messerli that, "it seems conceivable that abundant gun availability facilitates firearm related deaths."  This is an important assertion that helps and hinders. The face validity is fine; even if the rate of deaths grows more slowly than the rate of gun abundance (in other words, a positive correlation less than 1), you will still have marginal more gun deaths per unit of abundance, even if it follows an economic curve.
With a cursory review of the article myself (prompted by Lewis's analysis above), I found that there are plenty of alternative explanations that Bangalore and Messerli could have investigated to provide a fuller context for their findings. More disclosure about their sample selection and a bit more discussion of the generalizability (i.e. the issues of their model/design inference that Lewis describes nicely) would be well served. There are a lot of theoretical perspectives to start from with this issue, such as political, sociological, or in Bangalore and Messerli's case, the medical fields approach to firearm-related deaths. Their decision to include crime rates and mental illness in the study may have confounded the legitimacy of their discipline's approach.

And finally, I find it ironic that we can purport to predict complex social problems without even getting an accurate count of sovereign nations on the Earth!

reply to this comment

Skip to Main Site Navigation / Login

Site Search Form

Site Search