Asymmetrical Information - Megan McArdle

01.28.13

Department of Awful Statistics: Are Mass Shootings Really On the Rise?

They're not, but the alarmist numbers are more likely to get media attention.

In mid-December, the day after Sandy Hook, Mother Jones assembled a list of mass shootings.  It seemed to show that mass shootings were an accelerating epidemic.  

Since 1982, there have been at least 62 mass shootings* across the country, with the killings unfolding in 30 states from Massachusetts to Hawaii. Twenty-five of these mass shootings have occurred since 2006, and seven of them took place in 2012. We've mapped them below, including details on the shooters' identities, the types of weapons they used, and the number of victims they injured and killed.

Naturally, a list like this attracted a great deal of attention.  The blogs I read, my Facebook and twitter feeds, and the Christmas parties I attended were crammed with people asking why mass shootings had accelerated so dramatically.  

James Allen Fox, a criminology professor who studies mass murder, was on the news in several places saying that this wasn't true.  But the Mother Jones figures got more play, presumably because in the wake of Sandy Hook, they felt true.  Now Fox has reviews the Mother Jones figures, and shows how they went wrong. 

After much debate over parameters, Mother Jones settled on several criteria for inclusion in its mass shootings database, specifically:

The killings were carried out by a lone shooter. (Except in the case of the Columbine massacre and the Westside Middle School killings, both of which involved two shooters.)

The shootings happened during a single incident and in a public place. (Public, except in the case of a party in Crandon, Wisconsin, and another in Seattle.) Crimes primarily related to armed robbery or gang activity are not included.

The shooter took the lives of at least four people. An FBI crime classification report identifies an individual as a mass murderer-as opposed to a spree killer or a serial killer-if he kills four or more people in a single incident (not including himself), and typically in a single location.

If the shooter died or was hurt from injuries sustained during the incident, he is included in the total victim count. (But we have excluded cases in which there were three fatalities and the shooter also died, per the previous criterion.)

We included six so-called "spree killings"-prominent cases that fit closely with our above criteria for mass murder, but in which the killings occurred in multiple locations over a short period of time.

Not only is Mother Jones’s decision to disqualify cases based on certain criteria hard to defend, the criteria themselves were not necessarily applied consistently. Mother Jones included the 1993 Chuck E. Cheese robbery/massacre of four people committed by a former employee, but excluded the Brown’s Chicken robbery/massacre of seven victims that occurred the very same year, presumably because two perpetrators were involved in the latter incident or perhaps because these gunmen had no prior connection to the restaurant.

Mother Jones also eliminated massacres involving family members, even though they too can involve large body counts, such as the massacre of 14 relatives and two others by R. Gene Simmons of Russellville, Ark. in 1987. Other massive shootings, like the execution-style slaughter of 13 in a Seattle club in 1983, were ignored because of their relation to gang activity or some criminal enterprise. Particularly mystifying is the decision not to include cases involving multiple perpetrators yet to waive this condition for two school shootings.

Notwithstanding the questionable motive-based selectivity built into the Mother Jones analysis, it seems odd to ignore shootings with large death tolls just because there was more than one shooter or because the shooter was related to his or her victims. These incidents are no less devastating to the families and communities impacted by the crimes.

In other words, the Mother Jones team built a data set with restrictions specified for no obvious reason.  Then they ignored those criteria if they threatened to exclude a case that obviously "should" belong, like Columbine.  The result is not a database of mass shootings; it's a database of mass shootings that Mother Jones wanted to include in their database.  It tells you nothing about anything, except maybe the prior beliefs of Mother Jones staffers.

And its results are radically at odds with what Fox says is the full database of mass shootings in recent decades.  

While there are clusters, which may be due to copycatting, there's no obvious pattern.  They're neither rising nor falling, and the highest fatality shooting took place while the assault-weapons ban was in place.  

Unfortunately, as is often the case, the bad numbers drove out the good.  "Mass shootings rising rapidly!" is the kind of headline that grabs the reader and invites people to email it to half their friends.  "Mass shootings neither up nor down, as far as we can tell!" is not nearly so compelling.  And of course in this case, Mother Jones' jerry-rigged dataset fit neatly with a political project that had just gained new urgency.  At this late date, it's unlikely that the truth will every catch up.