Sunday, October 11, 2009

Why skydivers would be better off without parachutes

From Reason Magazine, debunking another media reported "study" of why guns are no good:
In Philadelphia, researchers at the University of Pennsylvania find, possessing a gun is strongly associated with getting shot. Since "guns did not protect those who possessed them," they conclude, "people should rethink their possession of guns." This is like noting that possessing a parachute is strongly associated with being injured while jumping from a plane, then concluding that skydivers would be better off unemcumbered by safety equipment designed to slow their descent. "Can this study possibly be as stupid as it sounds?" asks Stewart Baker at Skating on Stilts. Having shelled out $30 for the privilege of reading the entire article, which appears in the November American Journal of Public Health, I can confirm that the answer is yes. ...

Read the rest here. If guns make those who carry them less safe, shouldn't we immediately disarm all our cops? After all, the guns cops carry make them less safe, if we simple folk are to believe the study's conclusion.

And UCLA law professor Eugene Volokh eviscerates the study's methodology and conclusions here:
... Conspicuously missing from the press release and the news story were two critical limitations that were admitted in the original study. These qualifiers mean that the press release headline, as well as all the other statements and implications of causation, were quite mistaken. Perhaps defensive possession and carrying of guns helps protect the possessor and carrier, and perhaps it doesn’t. But the study sheds virtually no light on the subject.

1. To begin with, there’s the obvious causation/correlation problem. Maybe, as the authors speculate, carrying a gun increases your chances of being shot with a gun (as suggested by the framing of the issue as “whether guns are protective or perilous”), or at least fails to decrease them (”guns did not protect”). Or maybe a third source — perhaps some people’s being the targets of death threats, or being in a dangerous legal line of work, or being gang members or drug dealers — causes both higher gun carrying among those people and higher risk of being shot.

By way of analogy, we don’t suggest that pacemakers cause heart attacks, or don’t protect against heart attacks, just because we find a correlation between the presence of pacemaker and the incidence of heart attacks. Obviously, people might get pacemakers precisely because they’re at risk of heart attacks. Well, people might get guns precisely because they’re at risk of attack. (Stewart Baker makes a similar point.) [emphasis added]

One can try to control for this in some measure — but while the study controls for some relevant attributes (race, sex, age, neighborhood, having a “high-risk occupation,” and having at least one arrest on one’s record), it leaves a vast range of factors uncontrolled. You’d think that gang members are more likely than others to carry guns and to get shot, even controlling for the presence of an arrest record. (Lots of law-abiding people carry guns, but I expect that more gang members do.) But the study doesn’t control for that, or for many other things. ...

For the stats and science geeks, Prof. Volokh's comments on the study's fatal flaws are well worth a read. His conclusion:
... And all this is in addition to the possible confounding factors discussed in item 1 above. If there were no such confounders, then perhaps even a low odds ratio might be telling, or perhaps even a statistically insignificant odds ratio above 1 might in some measure undermine the “guns as protective” theory. But these two problems put together — the possibility that the result stems from the existence of a high-risk group whose members are especially likely both to carry guns and be the targets of attack, and the possibility of even slight misreporting dramatically affecting the results — make the study highly uninformative.

So it’s possible that gun possession was “perilous,” in the sense of increasing the risk of the possessor’s being injured. It’s possible that it “did not protect those who possessed guns,” in the sense that it didn’t reduce the risk of the possessor’s being injured. But it’s also possible that it was “protective,” in that it reduced the risk of the possessor’s being injured, but this result is swamped by the other phenomena I point to. The study doesn’t give us much extra information about which theory is correct. And yet it is publicized, and it’s reported, as if it did robustly show the causal relationship.

The good professor is being kind, methinks. Hopefully this shoddy study, conducted by an associate professor of epidemiology, isn't what passes for excellence in the Public Health departments of our universities.

No comments: