Reply #51: First off, sorry for coming back to this so late

[Euromutt]

Long story. Anyway.

Moreover, public health researchers and social scientists cite each other's papers, and even co-author with each other.

Really? Can you cite, let's say, ten papers co-authored by a medical/public health researcher and a non-public health researcher, all of which are by different people? (I might add that I count David Hemenway as a public health researcher, even if he is formally an economist, on the basis that he's employed by the Harvard School of Public Health.)

Yes, surely an epidemiologist has used a "pathogen" as an analogy to a gun -- similarly, surely a physicist working in finance has compared a stock price to a "particle" -- a coding theorist working with DNA might speak of the "alphabet" of nucleotides.

Using analogies is fine, as you as you draw the appropriate parallels. Merely possessing a firearm isn't analogous to smoking, it's analogous to possessing a pack (or more aptly, a forty-year supply) of cigarettes, and possession of tobacco alone doesn't increase your risk of lung cancer, emphysema, etc. Of course, there are very few conceivable reasons to possess a pack of cigarettes other than smoking its contents, whereas there are other reasons to possess a firearm than to commit criminal offenses. Similarly, simultaneously possessing a motor vehicle and some alcoholic beverages at the same time isn't the same as consuming alcoholic beverages and then operating the vehicle while under their influence. And that's where the pathogen analogy as used by all too many public health researchers breaks down: in conflating possession with use. When it comes to pathogens, carcinogens, etc. that's usually not an issue because exposure to such things is risky by definition; "pathogen" literally means something that causes ("gives birth to") suffering (πάθος).

The NRA types loves to hate Kellerman, and every gun advocate worth his salt is equipped with pages of anti-Kellerman talking points.

This can be explained by two points: first, Kellermann's conclusions are those most frequently cited by anti-gun activists, and second, his work epitomizes how the public health/medical research works toward a predetermined conclusion, particularly the disconnect between the study's actual findings and the stated conclusion.

Many of the criticisms (e.g. that the study wasn't controlled for drug use or criminal record) are simply fabrications.

"Simply fabrications" is overstating the case; there were certainly aspects in Kellermann's various studies where claims that such things as drug use and criminal record had been controlled for--or at least, adequately controlled for--rang hollow, particularly in his 1993 study "Gun Ownership as a Risk Factor for Homicide in the Home," in which he did not share his raw data, making it rather hard for anyone to assess the veracity of that claim.

Sure, it would be nice to do a larger prospective study -- maybe someday we'll have that. Until then, we have Kellerman, along with other case-control studies that have corroborated Kellerman's findings.

And the only thing all that research has established is correlation, not that "more guns" is the cause of "more death." It's analogous to finding that keeping insulin in the home is strongly associated with one or more members of the household having diabetes and concluding that, therefore, insulin causes diabetes. "Maybe someday we'll have" a larger prospective study? Given that that segment of the public health research community that takes an interest in this topic has been at it for 30-odd years without producing one, I doubt we ever will. After all, as I've pointed out before, associations found in retrospective studies "often find associations that later turn out not to hold up under study using prospective studies or randomized trials or, alternatively, turn out to be much weaker than the retrospective study showed" (see here), and why risk invalidating all that prior research?

What's more, research done by pro-gun criminologists is very frequently of this same variety -- retrospective analysis of demographics/surveys/police databases/etc. Funny that you don't seem to go after the pro-gun research on the same broad methodological grounds.

Very simply, I'm judging the public health research on firearms by the standards to which other medical/public health research is held. When someone like Branas claims in his press release that his methodology "is the same approach that epidemiologists have historically used to establish links between such things as smoking and lung cancer," it's fair to point out that even Richard Doll's findings were only provisionally accepted until they were confirmed by cohort studies. And Doll's work showed a far stronger association between smoking and lung cancer than Branas' team found between carrying a gun and being shot; we're talking about finding that ~90% of lung cancer patients were smokers, as opposed to Branas' finding that ~6% of shooting victims were carrying a firearm on or about their person.

But if you want to play this game, it is worth pointing out, as someone else here has, that criminology is academic backwater, compared to either epidemiology or economics.

If by "academic backwater," you mean criminology is less practiced at blowing its own trumpet to get news media attention, then I would have to agree. However, that has no bearing on the scientific validity of the research produced.

In particular, the latter two are generally more skilled with statistics and data analysis.

I'd like to see some evidence for that claim, because as far as I'm aware, that's not true at all. I would also argue that there's a difference between prowess in statistics on the one hand and effective data analysis on the other. John Lott's work has been described as "statistical one-upmanship":

He has more data and a more complex analysis than anyone else studying the topic. He demands that anyone who wants to challenge his arguments become immersed in a very complex statistical debate, based on computations so difficult that they cannot be done with ordinary desktop computers. He challenges anyone who disagrees with him to download his data set and redo his calculations, but most social scientists do not think it worth their while to replicate studies using methods that have repeatedly failed.

I should point out that I read More Guns, Less Crime and didn't find it in the least persuasive, in part precisely because Lott had to crunch such a huge volume of data to produce his conclusion. The more complex the calculations needed to make your point, the more dubious the validity of your point gets; that's just Occam's Razor in action.

Note that Lott is, strictly speaking, an economist, not a criminologist.

Kleck's infamous and influential survey on defensive gun uses serves as a good example of where a criminologist could have used some epidemiological training. A phone survey found that around 1% of people reported using a gun defensively in the last year. Of course, as any epidemiologist would know, in this situation, the results are extremely sensitive to false positives: unless you keep the false positive rate to below 1% (essentially impossible here) the study is basically meaningless. It took Hemenway to point this out, yet even to this day Kleck's implausible numbers for DGUs are often cited by the pro-gun side.

Excuse me while I express a derisive snort. Kleck & Gertz issued a persuasive rebuttal to Hemenway's critique. It might also be noted that Hemenway, with his frequent collaborator Deborah Azrael, conducted two surveys on DGUs in 1994 and 1996 (respectively, "Use of Guns in Self-Defense: Results of a National Telephone Survey" and "An Armed Society is a Polite Society? Survey Results," both Harvard University Working Papers and both published in 1996) with very similar methodology to Kleck & Gertz's; that is, prior to the publication of Hemenway's critique in 1997. Call me overly suspicious if you will, but I think there's something a bit... off about somebody claiming that a particular research method is inherently ill-suited to the task only after he's used it twice himself, and had it produce findings that didn't suit his agenda (in the order of 900,000 DGUs annually).

Much the same applies to Cook & Ludwig's study, sponsored by the NIJ and published in 1997. After trying to replicate Kleck & Gertz's experiment, and producing results that, in Cook & Ludwig's own words, were "in the same ballpark that propounded by Kleck and Gertz," C&L then spent the remainder of the paper coming up with ad hoc explanations why their findings were invalid, in broad terms dismissing the methodology as unsuited to the task. But if the method were so obviously unsuited, why declare its unsuitability only after using it? Tellingly, moreover, in the same paper, C&L express total confidence in the veracity of the responses concerning the percentage of respondents reporting one or more firearms being kept in the household. Hence my use of the term "ad hoc": Cook & Ludwig--and Hemenway, for that matter--only express doubts as to the suitability of the survey method only when it produces undesirable results, only as it pertains to those undesirable results, and only after the fact, not before. It's also worth noting that in the intervening 15 years, neither party has produced research on DGUs using a method they did think was suitable.

If you want "play this game" of making comparisons to clear-cut examples of denialism, I can point out that Cook, Ludwig and Hemenway's behavior concerning DGUs is akin to that of creationists who can't actually produce any empirical evidence that supports their claims, and contend themselves with claiming to have poked holes in evolutionary theory, and then committing a false dichotomy that, since the people they oppose are supposedly wrong, they themselves must therefore be right.

Another false claim of yours is that public health researchers don't know about or choose to ignore research outside their circle. From the surveys I've seen from both "sides", it is the mainstream public health/economics types who give more comprehensive citation lists, which will typically include criminological research, including the pro-gun papers.

Excuse me while I stifle a derisive snort. While it's, in the strictest sense, true that public health researchers do cite criminological research, it is frequently only to dismiss its findings, or represent them incorrectly in an exercise of what Don Kates has termed "gun-aversive dyslexia." It's notable, for example, that Kleck's initial work on DGUs, published in 1988, received no mention at all in the public health literature for three years, until Philip Cook's article "The Technology of Personal Violence" was published in 1991, in which Cook took issue with Kleck's preliminary findings; only at this point did public health researchers even acknowledge the existence of Kleck's work, and then only to claim it had been discredited.

Interestingly, one of the researchers to refuse to even acknowledge Kleck's work as late as 1993 was Garen Wintemute ("Policies to Prevent Firearms Injuries," Health Affairs, 1993), the very person whose research gave rise to this thread.