Asymmetrical Information - Megan McArdle

10.19.12

More Fact Checker Fail

It's time to recognize that fact-checkers often aren't equipped to know what "the facts" are.

I've been hard on "fact checkers" this campaign season.  And deservedly so!  The entire genre of hastily researching a topic that you don't normally cover and then using the results of your research to confirm or deny the things that politicians say is properly called "blogging", not "fact checking".  Fact checkers from the LA Times to Candy Crowley have made a number of erroneous "factual claims"--and yes, indeed, conservatives, the errors do all seem to run in the same direction.  

Here's another from Politifact, wherein they rate an Obama claim "mostly true" while repeating an Obama claim that is entirely false.  Jonathan Adler ably sums up:  

Another example of fact checkers having trouble with facts can be found in Politifact’s commentary on whether it was fair for President Obama to criticize Mitt Romney for failing to say whether he supported the Lilly Ledbetter Fair Pay Act. In making its assessment Politifact totally bungled its description of the Supreme Court’s decision in Ledbetter vs. Goodyear Tire & Rubber Co., and in the process perpetuated a false claim about the decision oft repeated in political debate (including by Lilly Ledbetter herself).

Politifact wrote:

In 2007, the Supreme Court had ruled in Ledbetter vs. Goodyear Tire & Rubber Co. that the 180-day statute of limitations started from the day an employer made the decision to discriminate — making it impossible for employees who learned of such discrimination later to get relief, such as back pay.

The problem is the last part of this sentence is false. In Ledbetter the Supreme Court did not hold that employees who learned of alleged discrimination more than 180 days after the alleged offenses were precluded from suing their employers. That question was not before the Court, and the majority opinion expressly noted (in footnote 10) that it was not answering this question. Ledbetter’s claim was not premised on a recent discovery of past discrimination. In fact, during the course of the litigation Ledbetter acknowledged that she first learned of the alleged discrimination more than 180 days before she filed suit. Indeed, as Hans Bader notes, Ledbetter admitted in a deposition that she learned of the pay disparities in 1992, but did not file suit until 1998.  

I'm also not clear how the Obama administration's true-but-arguably-misleading statement that a spokesmen asked by a reporter about the Lily Ledbetter Act said "I'll get back to you" and didn't (happened, but the Romney campaign has elsewhere stated they would not change the law), gets a "mostly true", while the Romney campaign's true-but-arguably-misleading statement that oil production was down on federal lands (true, but the most recent year wasn't necessarily representative) gets a "half true".  How exactly are they measuring the weight of these "facts"?  

In fairness, the false claim about the lawsuit has been widely repeated, including by Lily Ledbetter, so I'm sure that the President, and the fact checkers, came by their misunderstanding honestly.  But this is exactly why I object to calling this stuff "fact checking" . . . and especially why I object to other journalists who say "Fact checkers have said that this is [true, or not true]" without doing any of their own checking.  The fact checkers are not subject matter experts; they're spending a couple of hours doing research on the internet.  They aren't qualified to parse the various claims on hundreds of different topics, and they couldn't possibly be.    

And looking at the big goofs of the last month or so--welfare reform, whether IPAB rations care, this one--it's hard not to suspect that they are interrogating conservative claims just a wee bit harder than liberal ones.  A year ago, I wrote about how bias actually works (note to conservatives--not a giant conspiracy).  Instead, it's a function of what questions you ask: 

Before I was a journalist, I used to wonder why journalists were suppressing obvious important facts; after I became a journalist, I realized that it's often incredibly hard to know that there's a fact you're missing.  I seem to recall a scathing editorial about mortgage financing in maybe 2005, in which the outraged writer pointed out that banks were booking the asset value of the loans as if they were going to realize 100% of the payments . . . even though some of those loans would go bad!  Fraud!  Malfeasance!

. . . er, accounting.  Anyone who knows anything about accounting knows that what the writer was saying is absolutely true . . . and that if you look on the other side of the balance statement, you find that they have booked a corresponding allowance for bad debts.  You can argue that in 2005, those allowances weren't big enough--hell, I think at this point, it's not an argument, but a fact.  Nonetheless, the opinion column was ludicrously wrong.

And yet, understandably wrong.  I'm sure that the writer had been fed the story by a consumer activist (I can probably name the group).  He looked at the financial statements, and the numbers checked out.  He'd never heard of such a thing as an allowance for bad debts, so how was he to know it was there?

Bias matters not because liberals deliberately slant their stories, but because they are much more likely to interrogate the facts that contradict their ideological beliefs, than the ones that support them.  When they come across an uncomfortable fact, they'll go out of their way to figure out why it isn't really true.  When they come across a fact that confirms what they believe, they'll be more likely to accept it at face value.

I should make clear, since that quote is a clip from a longer piece, that I do not believe that liberals are somehow less likely than conservatives to check their priors.  I'm writing about liberals in this context because the overwhelming majority of my colleagues in the media vote Democratic--as I myself have been known to do, from time to time.  A conservative dominated media wouldn't be any better about policing its own assumptions.

I should also make clear that I actually think that what fact checkers do is really valuable.  But fact checkers need to be aware that they often simply don't know what questions to ask--and that they should therefore avoid declaring that their opinion is a fact.