Searching for Fraud

I just came across this rather chilling interview Ed Yong did with scientific investigator Uri Simonsohn back in July.  Simonsohn has bee looking for evidence of fraud in social science research--and finding a really disturbing amount of it. "I don’t know how systemic the crime is." Simohnson told Yong.  "What’s systemic is the lack of defences. Social psychology — and science in general — doesn’t have sufficient mechanisms for preventing fraud. I doubt that fabrication is any worse in psychology than in other fields. But I’m worried by how easy it was for me to come across these people."  

The same is, of course, true of journalism.  It's essentially impossible to prevent journalistic fraud--if I tell you that I saw someone doing something, there's rarely a videotape of the event.  If I'm doing an interview on the cell phone, there isn't a recording, either.  And so our readers have to trust us.  

The overwhelming majority of journalists strive mightily to justify this trust.  And then . . . some don't.  But like social science research, we don't have much of a mechanism to catch them.  What was so astonishing about the Jayson Blair case was the number of people who he claimed to have interviewed who knew they'd never spoken to him; they just didn't bother to call the Times and point this out.  As I recall, when asked why, one of them said he figured what was printed in the papers was all lies anyway.  

There were apparently complaints about Jack Kelley of USA Today for years before anything was done--and of course, most people didn't even know about the complaints.  With others, there were no complaints, only signs.  Stephen Glass was the editor of the Penn student newspaper when I was there, renowned for stories like this one, about spending the day with homeless people.  Only later, after his persistent fabrications were exposed by Adam Penenberg, did I think, "Hmmm, that story sure was impossible to check."  And as far as I know, there were no suspicions at all about Jonah Lehrer until Michael Moynihan tried to track down a Bob Dylan quote from his new book.  

This kind of fabrication became a central plot point in the last season of The Wire--and because David Simon was a journalist, rumor has it that he was basing the story on a real fabulist he knew about.  And yet, I thought that this was the least realistic season.  It would be relatively simple if editors allowed fabulists to continue their creations in order to win prizes and subscribers.  Then you could just replace the bad editors.  But in fact, all the editors I've ever met hate fabulists and care deeply about getting things right.  

So why, then, does it take so long to catch fabulists?  In part because as I said, it's really hard: there will never be a 100% perfect trail against which to check stories.  But there's another reason: it is the worst accusation you can make against someone in journalism.  It is the career death penalty.  You can make a bad mistake--get a story badly wrong, get taken in by a source, even plagiarize--and still work again, especially if you have a record suggesting that this is an isolated incident.  But you cannot make stuff up and continue working as a journalist.  There is good reason that this is so; it is how we ensure that it happens very rarely.  But as I wrote a couple of weeks ago about child molesting, the very severity of the accusation means it is invoked only when there is virtual certainty that it is true.  

If science and journalism want to protect themselves, they need to do more spot checks.  Random spot checks, so that getting checked carries no taint.  There need to be institutions devoted to simply trying to replicating what the reporters and scientists did.  

The problem is that there's no institutional incentive.  Journalism is already going broke; they can't afford staffers who just re-report stories.  Maybe science could, but what's the incentive?  Journals aren't interested in null results or "yup, we replicated that other study"; they want new studies with new findings.   And the career incentives for scientists are to seek publication, not add their voice to a growing chorus.