Over a week ago, at the height of the post-Democratic Convention Obama surge, I posed the following question during the nightly Daily Kos Elections Polling Wrap:
Is there an active effort to minimize the Obama convention bounce?That question was inspired by a number of media reports. First, there was the curious decision by Gallup to measure the Obama bounce simply by the change in Obama support, rather than the change in the margin between the two candidates. This had the causal effect of essentially halving the bounce, in Gallup's assessment. This was followed by ABC's decision to fixate on the rather small change in the margin of their likely voter screen (which moved three points in Obama's direction), while totally ignoring the far larger change in the margin of their less restrictive pool of registered voters (which had moved a total of seven points in the direction of the president).
Then, just two days later, CNN upped the ante when they declared Obama leads of five points in the states of Florida and Virginia to be a "tie." Such a characterization is not only absurd on its face, but it also is counter to how the Associated Press advises polls to be reported. Indeed, the words "tie" or "statistical dead heat" are only supposed to be used when the race is actually ... well ... tied.
Ironically, this week continued the trend of using (or, more appropriately, misusing) data to paint a picture of a toss-up where the data seems to suggest otherwise. This week's effort, ironically, was perpetrated by none other than the aforementioned Associated Press, who received an enormous amount of attention for their poll of the presidential race.
In what was a fairly good polling week for Barack Obama, it was the AP that set off Drudge Sirens midweek, and warmed the hearts of Republicans everywhere. They did so by declaring the race between President Obama and Mitt Romney to be merely a one-point race.
That poll result has found its way into countless efforts at analysis of the current state of play of this election. Journalist after journalist has cited the AP poll as a solid piece of evidence that this race is, indeed, far from over.
(Continue reading below the fold.)
Here is a representative example, courtesy of the Fix's Aaron Blake:
As we wrote Tuesday, Gallup polling shows that the bump Obama got from the Democratic convention two weeks ago has subsided. And another new poll, released Wednesday by the Associated Press and pollster GfK, shows basically the same picture, with 47 percent of likely voters supporting Obama and 46 percent backing Romney — a tie ballgame nationally.A closer look at the AP poll, however, shows that it is highly unlikely that Obama/Romney is really a one-point race. Indeed, only a small handful of responsible reports of the AP poll offered up a key missing detail: namely, a yawning gap between registered and likely voters that almost certainly was the result of an aberrant likely voter screen.
And, lest this seem to be simply a matter of opinion, there is ample evidence to buttress that point:
- Responsible analysis of polls should be wary any time there is a chasm between the result of a poll among registered voters, and results among the smaller pool of "likely voters." Some gap, to be sure, is to be expected. Routinely, for example, Nate Silver has posited that the likely voter screen should move polls about 2-3 points to the good for Republicans. A small gap like that would seem, on the surface, to be reasonable. After all, if you assume that the voters who would be the quickest to stand down are younger voters and ethnic minorities, a likely voter screen would probably shed those voters. And those are two voting subgroups that tend to favor Democrats.
In the case of this AP poll, however, that gap was cartoonishly large. While the likely voter result was Obama 47, Romney 46, the gap among the larger pool of registered voters was Obama 50, Romney 40. That is a nine-point movement towards the GOP when the likely voter screen is employed. A nine-point gap should invite a lot of scrutiny. The only way that comes even in the ballpark of plausibility is if there is a demonstrated, and sizable, enthusiasm gap between the parties. The problem with that: The data this week has actually confirmed precisely the opposite. A Pew survey released this week actually found Democrats to be more engaged than Republicans at this rather late point in the cycle. With that in mind, it starts to feel less and less reasonable to assume that the voters that are excluded from a "likely voter" screen would tilt so decisively to the left.
- Now, for the sake of argument, let's accept the presumption that those "unlikely" voters are disproportionately Democratic. As I said earlier, given the groups that tend to be less habitual voters, that might even be plausible.
Even factoring that in, a close look at the AP poll shows an even bigger flaw in its likely voter screen, and one that threatens to discount the poll entirely.
Let's look at the numbers: AP's pollster (GfK) polled 1512 adults for their presidential poll. Of those adults, they found 1282 of them to be registered voters. From that pool of registered voters, they identified 807 likely voters.
Among those adults, Obama had an enormous lead (52-37!). However, a poll of adults is fairly meaningless, given that a certain ratio of them aren't registered to vote. Sure, some of them may register between now and November, but that is far from certain, and a reasonable case can be made to discount their vote preferences. Likewise, it has become common practice to discount the registered voter topline results, as well. Why? Well, of course, we also know that not all registered voters will actually turn out on Election Day. This, as anyone reading this likely already knows, is why pollsters turn to likely voter screens.
But some quick math underscores the absurdly restrictive tendencies of AP's likely voter screen. Out of a pool of 1,282 registered voters, they only counted 807 likely voters. That would seem to assume that only 63 percent of the registered electorate will actually vote for the presidency. A Census Bureau study in the wake of the 2008 election found that the turnout has been far higher than that, even in the lightly attended 1996 election of Clinton-Dole. The range, according to the Census Bureau, has been between 82-89 percent. In the name of full disclosure, other estimates of turnout by registered voters are considerably lower. It is worth noting, though, that even the lowest turnouts were still north of 63 percent.
Therefore, in order for AP's turnout model to be true, even by the most pessimistic estimates of past turnout, voter turnout would still have to be lower than 1996, when less than half of the eligible electorate turned out. That would seem to be unrealistic, to say the least.
- What has been most criminal in the media coverage (and this is not wholly AP's fault) is that everyone is apparently assuming that the registered voter topline results simply did not exist. Virtually all of these breathless reports of a one-point race failed to note the far wider gap that existed with the less restrictive registered voter screen.
This is based on the long-held media assumption that likely voter screens inherently are more accurate than simply polling registered voters. However, a limited research effort I did on that subject for a 2010 Sunday Kos piece noted that, in 2006/08, that was not actually the case. Indeed, the RV screen actually got closer to the final result. In the coming week or two, my to-do list will include a full research sample of 2008 presidential polling, seeing if the LV or RV screen was more accurate in the Obama versus McCain race (look for that in an early October edition of Sunday Kos).
Cynical Democrats and fans of the president have long complained that the press seems to have a vested interest in portraying the Obama-Romney election as far closer than it "really is," in order to maintain a sense of tension that would (of course) drive viewership and readership.
When you see the AP poll being covered in the way that it was, and when you see the AP (via their pollster, GfK) employ a likely voter screen as restrictive as the one they used, and when you see media outlets cherry-picking close polls while discounting blowouts, it is hard to paint those cynics as silly conspiracy theorists. To be sure, the polling landscape (in part because of the sheer volume of numbers we are now seeing) is more varied than a lot of Democrats would care to admit. But the overwhelming preponderance of data looks better for the Obama team than it does for the Romney team. And it is more than a little bemusing to see how few members of the traditional media are willing to say that out loud.