The Necropolitan Sentinel

In the aftermath of Tuesday's elections, we're seeing a lot of analyses, and one item in particular jumped out — the disparity between the reported results and the exit polling.

Here's one such story from the Washington Post:

 

Exit poll numbers released to subscribers just before polls closed in the Wisconsin recall election Tuesday dangled the possibility that Milwaukee Mayor Tommy Barrett (D) could win.

The numbers seemed to pop off the screen — 50 percent apiece for Barrett and Republican Gov. Scott Walker, the subject of the recall effort. Walker had a clear lead in independent pre-election polls, so the tie score sent analysts scrambling and buoyed Democratic hopes when the numbers were widely reported elsewhere minutes later at the official poll close time.

Just a half hour later, the exit poll shifted to 52 to 48 percent, tilting in Walker’s favor. (The final margin appears to be seven percentage points.) A potential Gov. Barrett era had ended before it started, and a fresh round of bash-the-exit-poll commenced.

Then there's Michael Barrone adjusting the Romney-versus-Obama exit poll from the sample (I assume they asked the people if they voted for Walker or Barrett, as well as if they'd pick Obama over Romney):

The Wisconsin exit poll evidently reported the race for governor in the recall ballot as 50%-50%. With 92% of the vote in, the Milwaukee Journal Sentinel’s excellent website reports the score as 54%-46% Walker. Let’s say that’s the final results: only 13% of precincts from Milwaukee County and 3% of precincts from Madison’s Dane County—the Democrats’ two reservoirs of big majorities—remain uncounted. It has been emblazoned on mainstream media that the exit poll also showed Barack Obama leading Mitt Romney in the state 51%-45%. But if you think the exit poll was 4% too Democratic—and that’s in line with exit poll discrepancies with actual vote results over the last decade, as documented by the exit poll pioneer, the late Warren Mitofsky*—that result looks more like 49%-47% Romney. Or assume the remaining Milwaukee County precincts whittle Republican Governor Scott Walker’s margin over Democratic Milwaukee Mayor Tom Barrett to 53%-47%, which looks likely, the Obama-Romney numbers would look like 48%-48%.

I'm a bit skeptical of these kinds of adjustments, especially since I am not seeing any error bars on the poll results. Remember error bars aka confidence intervals? They're usually quoted as +/- 2% with 95% confidence.

I'm not going to turn this into a lesson on the Central Limit Theorem, but I am going to point to these confidence interval calculators.

The first calculator tells you how big a sample you need to draw, given a particular population size, confidence interval half-width, and confidence level. The vote totals I'm seeing right now are about 2.5 million, so I'll pick that population size. To get the result to within a percentage point, I'm going to pick 0.5(%) as my confidence interval half-width, and finally I want 99% confidence.

Result: 64,838 people to exit-poll.

Did they poll almost 65,000 people? I can't tell, because I don't see that info anywhere.

Oh wait! Here's a FoxNews exit poll:

 

Total Respondents

2095

Oh wait, that's just from the Presidential Primary in April. But let's assume that a similar number were polled for the Wisconsin race. That's where the second calculator comes in. I choose 2095 for the sample size, population of 2.5 million, 99% confidence interval, 50% result…. confidence interval is +/- 2.82%.

Now, the 53% of Walker is outside that +/- 2.82%. But barely. Heck, by telling my 50% for Walker, there's already an error bar — it could have been 50.49%, in which case 53% is within the confidence interval.

If they polled 65,000 people (I doubt it) and they got the 50-50 result, then I'm all on board with using Barrone's adjustments. However, with a smaller sample size — the adjustments Barrone is making would make me extend the error bars even farther.

The WaPo article linked earlier points out a different, larger issue — 15% of voters were absentee voters in this election, and such people weren't exit-polled, obviously. There is a strong indication that absentee voters favored Walker. These confidence intervals work only if you're randomly sampling from the entire population, and if you're missing 15% of the population that may have different characteristics….yup, the error bars get broadened.

Now, I would love to think that Obama and Romney are neck-and-neck in Wisconsin, but I'm in doubt that the fall electorate will be the same as that for the recall, for a variety of reasons. For one, I bet there's a lot of people in Milwaukee not particularly worked up over Walker, and so didn't bother to vote on Tuesday. Some hundreds of thousands more people voted in the 2008 presidential election than in the 2012 recall. I can think of lots of reasons why said people wouldn't have voted this week (for example, school is out for the summer for college students).

So let's not get cocky, Barrone.

Tweet