Midterm Election Results: Internet Predicted Outcomes

George Gallup described polling simply as the science of reflecting public opinion, and like any science, it’s only as good as the technology driving it. The first “modern” election polls, launched between the World Wars by Literary Digest, leveraged a new innovation, air mail, but failed famously in 1936, as the disproportionately wealthy audience forecast an Alf Landon landslide over FDR. Telephone surveys proved better, but also favored the rich at first (when phones were a luxury) and the old more recently (since younger voters tend to be cell-only, unreachable by pollsters). Exit polls, demographic modeling, prediction markets—all have been deployed to varied results.

Which brings us to the Internet, the most extensive and immediate opinion generator ever invented. Can’t that be harnessed to become a more accurate predictor of opinion?

For the past two months, that’s what The Daily Beast has been trying to figure out, through our Election Oracle. This wasn’t an Internet poll in the traditional sense—if it were, Justin Bieber would be the new speaker of the House—but rather taking the collective opinion expressed as people blogged, commented, and tweeted, and distilling that into a number.

First, we found a partner who could help us execute this concept: WiseWindow, a leading company in the field of business intelligence, which tells everyone from Microsoft and Sony to Stanford and Paramount what people online think about their brands and products.

Then, we set out to create a methodology. Specifically, the Oracle began perpetually scanning 40,000 websites, message boards, and forums, as well as Twitter and other public social media feeds (private networks like Facebook remained off-limits, and every public comment was tabulated anonymously), took the millions of political comments we found each day and sorted them by candidate. The Oracle specifically studied the exact text of each comment, and figured out if was positive, negative, neutral, or mixed. This plus/minus factor, using a 10-day moving average, was then mixed with traditional polling to create a prediction. Like the esteemed Nate Silver at FiveThirtyEight.com or the InTrade prediction market, our predictions were expressed as a likelihood of winning (i.e. 60 percent means you have a three-in-five chance of winning the election, not that you’ll get 60 percent of the vote.)

It was often tricky. Is someone commenting on Harold Johnson, the Republican running in North Carolina’s 8th district? Or the beach volleyball star named Harold Johnson? Or the Vietnam-era general? Or the fantasy game designer? And if someone comments about how a candidate strongly supports health-care reform, is that an insult or a compliment? How do we factor in the natural ability of incumbents to attract more attention online, by virtue of their office? And how do we adjust when comments come from outside the candidate’s district? Over time, we—and the Oracle—got smarter about such issues.

So how did the Election Oracle perform? Extraordinarily well, actually.

From there, we subjected ourselves to a lot of trial-and-error. During the primary season, we furtively back-tested races to figure out what worked and what didn’t. And that process continued once we went live almost two months ago. Methodologies and weightings were regularly tweaked, and our editors got directly involved if anything on the Oracle looked completely off. (Humans still run the machines here.)

So how did the Election Oracle perform? Extraordinarily well, actually. Arguably, as good as any prediction outfit out there when if comes to statewide races.

Senate: Of 37 Senate races, the Oracle only muffed one—Harry Reid’s upset against Sharron Angle in Nevada, which virtually every other poll and prediction misfired on, as well. The Oracle correctly predicted every other nail-biting win, from Mark Kirk in Illinois to Patty Murray in Washington to Joe Manchin in West Virginia.

The Oracle was even prescient about the only two races it felt were too close to call. Colorado resulted in a virtual tie: Sen. Michael Bennet won re-election by roughly 15,000 votes out of 1.4 million cast, and they’ll still be counting the votes in Alaska for a few more weeks to see if Lisa Murkowski’s write-in campaign will triumph.

How did others pollsters do? Nate Silver, who used a baseball statistics background to revolutionize the analysis of tradition polls at FiveThirtyEight, performed similarly—wrong on Angle, pretty much right or close on the rest. But his polling-based data told him, for instance, that Joe Sestak had only a 2.6 percent chance of beating Pat Toomey in Pennsylvania. The tight results showed that to be far too low. The Intrade prediction market performed similarly—wrong on Reid, less right than we were on many of the others. And while Charlie Cook wisely termed Reid-Angle a “toss-up,” he also did that with six other races, including some that weren’t, including the Barbara Boxer-Carly Fiorina face-off in California.

Governors Races: Another 37 races, some 30 of which we deemed competitive enough to track, and again, we only muffed one—we didn’t see Democrat Chris Quinn’s win in Illinois. Again, we had that as a 30 percent likelihood, compared with Nate Silver’s 18 percent shot. (In fairness to Silver, we often proved too cautious, such as giving New York Republican Carl Paladino a 20 percent chance, compared with FiveThirtyEight’s confident 100 percent flameout.)

And again, our 50-50 win predictions were validated, issued only in races that, once the voters had their say, bore out as true virtual ties: Connecticut, Oregon, Florida, and Vermont.

The House: We got 10 races wrong, and there are two ways to look at that. The glass half-full outlook holds that with 435 House seats up for grabs, making the wrong call on 10 comes to an amazing batting average (97.7 percent).

In the interest of making the Oracle better, we’re inclined to be more self-critical. The vast majority of those 435 seats, even in this volatile election, were safe. Indeed, we only actively tracked 77 competitive races, putting our prognostication percentage (including credit for 50-50 predictions) at a bit over 87 percent, albeit 87 percent of the knottier campaigns.

Studying the data, the reason our House predictions trail those in the Senate and governors races is simple: less data. Smaller districts and stakes, obscure candidates and positions all conspired to give us far less information to analyze, and thus more errors in our predictions.

We hope that will be less of an issue moving forward. We now have tens of millions of pieces of data to back-test. For instance, the data showing Harry Reid’s surge were sitting on our servers, plain as day, but we had too many filters (since he’s both an incumbent and the Senate majority leader) to see it.

Trial-and-error. Revisions and refining. That’s how you perfect a research model. But we’ve seen enough to find optimism in collective Internet opinion as an election tool. And we look forward to honing that tool as the 2012 campaign heats up.

Randall Lane is editor at large at The Daily Beast. The former editor in chief of Trader Monthly, Dealmaker and P.O.V. magazines, and the former Washington bureau chief of Forbes, he is the author of The Zeroes: My Misadventures in the Decade Wall Street Went Insane.