A few weeks ago in this space, we wrote about the bias toward sensationalist science reporting: lazy journalists report the press release and present the results in traditional scientist-as-hero mode, while the more thoughtful members of the press see through the hype and want no part of it. The result is that readers are exposed only to the puffery but only rarely to the skepticism.
Here’s an example. A couple weeks ago, one of us received the following email from a reporter for the news section of the journal Nature:
“The paper is about paedophiles being more likely to have minor facial abnormalities, suggesting that paedophilia is a neurodevelopment disorder that starts in the womb. We're a bit concerned that the stats look weak though—small sample size, no comparison to healthy controls, large SD, etc.
“If you have time, could you take a quick look and let me know if the statistics seem to be strong enough to back up their conclusions? The paper is here:”
We spent a few minutes looking at the article in question and replied: “Yes, I agree, I don’t find this convincing, also it’s hard to know what to do with this. It doesn’t seem newsworthy to me. That said, I’m not an expert on this topic.” And the reporter followed our advice and did not cover the article.
So, good news, but you can see the bias. The study was covered in various high-profile news outlets including the London Times and Daily Telegraph, with the latter report being entirely uncritical (for example, using the words “researchers discovered” and “found,” rather than “claim”). The critical article that would have appeared in the highly respected magazine Nature did not appear because the editors there made the good judgment that the story is not newsworthy. Thus, the reports that did appear give a misleading impression.
Even more troubling, the authors of the study made even stronger statements (again reported uncritically in the newspaper article) that would not be supported by the research even if its statistics were flawless. From the Telegraph: “The findings suggest previous theories that offenders molest children as a result of being sexually abused themselves are almost certainly wrong, the report authors said.”
“Almost certainly wrong,” huh? Nope. The research article, from the Archives of Sexual Behavior, is a study of “a clinical sample of men referred for assessment following a sexual assault, or another illegal or clinically significant sexual behavior,” but with no measures of any abuse that these men themselves had received as children. A correlation between pedophilia and innate attributes such as left-handedness (which, again, is itself questionable given the weak data in the paper) does not in any way preclude the theory that offenders molest children as a result of being sexually abused themselves.
We have no idea what is happening with child molesters, but the researchers’ uncritically-reported “almost certainly wrong” statement represents the crudest sort of determinism, the idea that if factor X (in this case, measurements such as “hair whorls,” “asymmetrical ears,” and “an excessive gap between first and second toe”) is correlated with an outcome of interest, that factor Y (in this case, possible sexual abuse as children) cannot have any effect.
We see this sort of thinking all the time, picking out one factor and ignoring everything else, what political scientist Daniel Drezner has memorably called, “piss-poor monocausal social science”.
Biological determinism (which commonly takes the form of genetic determinism) seems to us to be associated with all sorts of confusion. For a silly example, there’s this bit from singer Sam Smith:
“From what I can remember, they believe that you can be homosexual, but you just can’t practice it, which is ridiculous. I would just say, ‘I am proof that it's genetic. It has to be, because it wasn’t a choice.’ And that’s it.”
The fallacy, of course, is to think that everything is a choice or is genetic. Just google *identical twins one straight one gay* if you want to learn more. We followed these links ourselves and found that genetic essentialism on this topic is not restricted the pro-gay side. From the other direction is the homophobic “nooneisborngay.com” that announces, “How is it possible that identical twins with identical DNA have different sexual attractions? Simple. No one is born gay ... if people are born homosexual, then all identical twin pairs should have identical sexual attractions, 100% of the time...” Nope nope nope. A lot can happen between conception and birth!
And you see similar misconceptions about inborn political attitudes. But that’s a subject for a future column.
Andrew Gelman and Kaiser Fung are statisticians who deal with uncertainty every working day. In Statbusters they critically evaluate data-based claims in the news, and they usually find that the real story is more interesting than the hype.