Whenever someone doesn't like the results of a robo-pollster, the methodology is immediately called into question.
“National Journal refuses to print Rasmussen polls, yet Democrat leaders will trumpet bogus polls like this one in a futile effort to create a false sense of momentum for the hand-picked candidate of New York Sen. Chuck Schumer and the DSCC,” McConnell spokesman Justin Brasell said, referring to the chairman of the Democratic Senatorial Campaign Committee.
The reality is that Robo-pollsters like SurveyUSA and Rasmussen have had the best track record the past few election cycles. In 2004, it was the robos who dominated the results:
Before the election, we publicly doubted and privately derided Rasmussen and SurveyUSA, which used recorded voices to read their poll questions. We rolled our eyes when they touted the virtues of uniformity and when they complained that live interviewers "may not know how to read or speak the English language," could "chew gum," or might "just make up the answers to questions." It sounded to us like a rationalization for cutting costs.
Look who's laughing now. Rasmussen and SurveyUSA beat most of their human competitors in the battleground states, often by large margins.
Then there was 2006:
There were some interesting trends: Phone polls tended to be better than online surveys, and companies that used recorded voices rather than live humans in their surveys were standouts. Nearly everyone had some big misses, though, such as predicting that races would be too close to call when in fact they were won by healthy margins. Also, I found that being loyal to a particular polling outfit may not be wise. Taking an average of the five most recent polls for a given state, regardless of the author -- a measure compiled by Pollster.com -- yielded a higher accuracy rate than most individual pollsters.
On to the results: In the Senate races, the average error on the margin of victory was tightly bunched for all the phone polls. Rasmussen (25 races) and Mason-Dixon (15) each were off by an average of fewer than four points on the margin. Zogby's phone polls (10) and SurveyUSA (18) each missed by slightly more than four points. Just four of the 68 phone polls missed by 10 points or more, with the widest miss at 18 points.
Let's look at Poblano's scorecard for 2008:
- Selzer & Co
- SurveyUSA (Automated)
- Rasmussen (Automated)
- U of New Hampshire
- Chicago Tribune/Market Shares
- Field Poll
- Mason-Dixon
- Research 2000
- Quinnipiac
- Detroit News/Mitchell
- Ohio Poll/U of Cincinnati
- Public Policy Polling (automated)
- Strategic Vision
- Insider Advantage (automated)
- Franklin Pierce
- ABC/Washington Post
- Zogby
- EPIC-MRA
- Farleigh Dickinson
- Suffolk
- CNN/Opinion Research
- LA Times/Bloomberg
- ARG
- Fox/Opinion Dynamics
- Gallup
- Public Opinion Strategies
- Star Tribune
- Marist
- Franklin & Marshall/Keystone
- CBS/NY Times
- Zogby Interactive (Internet)
- Columbus Dispatch (mail-in poll)
What's remarkable is that the robo-pollsters, since they can poll at lower costs, run far more polls than the human-using pollsters (introducing more chances of blown calls). Yet it's funny that of the three robo pollsters, two of them are in the top three. The third, PPP, had a terrible Pennsylvania which threw off its averages, but has otherwise kicked ass in recent contests. In fact, PPP was the best pollster in Oregon, North Carolina, and Indiana.
It's easy to piss on the robo-pollsters since they're the new kids on the block. I did it to my detriment in 2002. But I quickly learned my lesson. Others ... they're apparently just more set in their ways or need cheap spin to explain away terrible results.
It's particularly ironic to see the NRSC trashing Rasmussen since the polling outfit has been a particular favorite of conservatives over the past few years -- the outfit has consistently given Bush (and McCain) better marks than other pollsters. Yet now that it's displaying the GOP weakness in Senate races this fall, they've turned on the outfit.
p.s. You can go check out every result to every poll by every pollster over at SUSA's site. They track it all -- the good, the bad, the ugly, the kind of transparency almost never available by traditional pollsters. And there's no doubt that while no poll has a perfect record, the robo-pollsters have done far better.
Update: Insider Advantage is also a robo-poller. Weird little factoid -- Indiana bans robo-polling (follow the link) and the state AG plans on going after robo-pollers who polled the primary. Seems extreme.