Skip to main content

Whenever someone doesn't like the results of a robo-pollster, the methodology is immediately called into question.

“National Journal refuses to print Rasmussen polls, yet Democrat leaders will trumpet bogus polls like this one in a futile effort to create a false sense of momentum for the hand-picked candidate of New York Sen. Chuck Schumer and the DSCC,” McConnell spokesman Justin Brasell said, referring to the chairman of the Democratic Senatorial Campaign Committee.

The reality is that Robo-pollsters like SurveyUSA and Rasmussen have had the best track record the past few election cycles. In 2004, it was the robos who dominated the results:

Before the election, we publicly doubted and privately derided Rasmussen and SurveyUSA, which used recorded voices to read their poll questions. We rolled our eyes when they touted the virtues of uniformity and when they complained that live interviewers "may not know how to read or speak the English language," could "chew gum," or might "just make up the answers to questions." It sounded to us like a rationalization for cutting costs.

Look who's laughing now. Rasmussen and SurveyUSA beat most of their human competitors in the battleground states, often by large margins.

Then there was 2006:

There were some interesting trends: Phone polls tended to be better than online surveys, and companies that used recorded voices rather than live humans in their surveys were standouts. Nearly everyone had some big misses, though, such as predicting that races would be too close to call when in fact they were won by healthy margins. Also, I found that being loyal to a particular polling outfit may not be wise. Taking an average of the five most recent polls for a given state, regardless of the author -- a measure compiled by -- yielded a higher accuracy rate than most individual pollsters.

On to the results: In the Senate races, the average error on the margin of victory was tightly bunched for all the phone polls. Rasmussen (25 races) and Mason-Dixon (15) each were off by an average of fewer than four points on the margin. Zogby's phone polls (10) and SurveyUSA (18) each missed by slightly more than four points. Just four of the 68 phone polls missed by 10 points or more, with the widest miss at 18 points.

Let's look at Poblano's scorecard for 2008:

  1. Selzer & Co
  2. SurveyUSA (Automated)
  3. Rasmussen (Automated)
  4. U of New Hampshire
  5. Chicago Tribune/Market Shares
  6. Field Poll
  7. Mason-Dixon
  8. Research 2000
  9. Quinnipiac
  10. Detroit News/Mitchell
  11. Ohio Poll/U of Cincinnati
  12. Public Policy Polling (automated)
  13. Strategic Vision
  14. Insider Advantage (automated)
  15. Franklin Pierce
  16. ABC/Washington Post
  17. Zogby
  18. EPIC-MRA
  19. Farleigh Dickinson
  20. Suffolk
  21. CNN/Opinion Research
  22. LA Times/Bloomberg
  23. ARG
  24. Fox/Opinion Dynamics
  25. Gallup
  26. Public Opinion Strategies
  27. Star Tribune
  28. Marist
  29. Franklin & Marshall/Keystone
  30. CBS/NY Times
  31. Zogby Interactive (Internet)
  32. Columbus Dispatch (mail-in poll)

What's remarkable is that the robo-pollsters, since they can poll at lower costs, run far more polls than the human-using pollsters (introducing more chances of blown calls). Yet it's funny that of the three robo pollsters, two of them are in the top three. The third, PPP, had a terrible Pennsylvania which threw off its averages, but has otherwise kicked ass in recent contests. In fact, PPP was the best pollster in Oregon, North Carolina, and Indiana.

It's easy to piss on the robo-pollsters since they're the new kids on the block. I did it to my detriment in 2002. But I quickly learned my lesson. Others ... they're apparently just more set in their ways or need cheap spin to explain away terrible results.

It's particularly ironic to see the NRSC trashing Rasmussen since the polling outfit has been a particular favorite of conservatives over the past few years -- the outfit has consistently given Bush (and McCain) better marks than other pollsters. Yet now that it's displaying the GOP weakness in Senate races this fall, they've turned on the outfit.

p.s. You can go check out every result to every poll by every pollster over at SUSA's site. They track it all -- the good, the bad, the ugly, the kind of transparency almost never available by traditional pollsters. And there's no doubt that while no poll has a perfect record, the robo-pollsters have done far better.

Update: Insider Advantage is also a robo-poller. Weird little factoid -- Indiana bans robo-polling (follow the link) and the state AG plans on going after robo-pollers who polled the primary. Seems extreme.

Originally posted to Daily Kos on Wed May 28, 2008 at 02:20 PM PDT.

Your Email has been sent.
You must add at least one tag to this diary before publishing it.

Add keywords that describe this diary. Separate multiple keywords with commas.
Tagging tips - Search For Tags - Browse For Tags


More Tagging tips:

A tag is a way to search for this diary. If someone is searching for "Barack Obama," is this a diary they'd be trying to find?

Use a person's full name, without any title. Senator Obama may become President Obama, and Michelle Obama might run for office.

If your diary covers an election or elected official, use election tags, which are generally the state abbreviation followed by the office. CA-01 is the first district House seat. CA-Sen covers both senate races. NY-GOV covers the New York governor's race.

Tags do not compound: that is, "education reform" is a completely different tag from "education". A tag like "reform" alone is probably not meaningful.

Consider if one or more of these tags fits your diary: Civil Rights, Community, Congress, Culture, Economy, Education, Elections, Energy, Environment, Health Care, International, Labor, Law, Media, Meta, National Security, Science, Transportation, or White House. If your diary is specific to a state, consider adding the state (California, Texas, etc). Keep in mind, though, that there are many wonderful and important diaries that don't fit in any of these tags. Don't worry if yours doesn't.

You can add a private note to this diary when hotlisting it:
Are you sure you want to remove this diary from your hotlist?
Are you sure you want to remove your recommendation? You can only recommend a diary once, so you will not be able to re-recommend it afterwards.
Rescue this diary, and add a note:
Are you sure you want to remove this diary from Rescue?
Choose where to republish this diary. The diary will be added to the queue for that group. Publish it from the queue to make it appear.

You must be a member of a group to use this feature.

Add a quick update to your diary without changing the diary itself:
Are you sure you want to remove this diary?
(The diary will be removed from the site and returned to your drafts for further editing.)
(The diary will be removed.)
Are you sure you want to save these changes to the published diary?

Comment Preferences

  •  You know, if we laid all the pollsters end to end (5+ / 0-)

    it would be a great idea.

    An agnostic not because I don't know if there's a God, but because I don't care.

    by filmgeek83 on Wed May 28, 2008 at 02:23:13 PM PDT

    •  who polls the people who rig the elections? (0+ / 0-)

      thats a poll i wanna see!

      "The right of voting for representatives is the primary right by which other rights are protected. To take away this right is to reduce a man to slavery, for slavery consists in being subject to the will of another, and he that has not a vote in the election of representatives is in this case. "

      Thomas Paine, Dissertation on the
      First Principles of Government

      •  ANother reason for Robo pollsters (3+ / 0-)
        Recommended by:
        Terri, dogstar30, zackamac

        Robo pollsters also have a unique advantage in this race, because it eliminates the race and/or gender biases that can occur based upon the race and gender of the pollster.  

        When I was in college, I took a class in which we helped write and test a poll about race relations for Zogby.  We discovered a study that showed that people can identify someone's race based upon their voice with 85% accuracy (Naturally, I would assume that gender identification is even higher).  We also found that the answers (as well as whether a person agreed to be polled) were skewed based upon the race of the caller.  For example, the African-American woman in our class received several hostile responses accusing her of "stirring up trouble," while none of the white students received any such responses.  

        This is obviously significant in the 2008 polls, because both whites and, perhaps especially, African-Americans may be less inclined to tell an African-American pollster that they are not supporting Obama.  Likewise, I would bet that older women may feel less inclined to tell another woman that they will not be voting for Hillary Clinton.  

        Automated polls would seem to eliminate the bias problem because people are merely responding to computers.  I have suspected for a long time now that this is one of the reasons that the automated polls have been more accurate than the exit polls in several states throughout the process.

        •  Good points. (0+ / 0-)
          I was thinking along the same lines, but you fleshed out the idea much better. Also, the robo-call pollsters, because the method is so much cheaper, may be able to use a far larger sample population for their polls, which would tend to lead to a more accurate result.

          "I'm asking you to believe. Not just in my ability to bring about real change in Washinton...I'm asking you to believe in yours." - Barack Obama

          by Wordie on Wed May 28, 2008 at 05:14:18 PM PDT

          [ Parent ]

  •  Has everyone been polled but me (6+ / 0-)

    I've had the same land line for 9 years.

  •  What's Selzer & Co? (0+ / 0-)

    never heard of 'em but they are #1 on here.

    Adopt a Shelter Dog!
    "No one worked harder to re-elect George Bush in 2004 than John McCain"

    by psycho liberal on Wed May 28, 2008 at 02:24:32 PM PDT

  •  People are probably more honest to robots (9+ / 0-)

    that would be my guess. When talking to a real human many people are intimidated or conceal opinions that embarass them.

  •  Wow,,,Gallup's that bad? (4+ / 0-)

    That's the outfit you see used on the teevee ALL THR E TIME.

    It's a neighborly day in this beautywood. Relentless!

    by ablington on Wed May 28, 2008 at 02:26:19 PM PDT

  •  Innovation... (1+ / 0-)
    Recommended by:
    lgmcp often frowned upon by the old guard -- it's the nature of progressive ideology vs. conservative, new polling outfits vs. old ones, and blogs vs. traditional media.

    But it often proves to be superior, too.

    •  absolutely opposed to automated calling. (0+ / 0-)

      It takes from 7 - 10 seconds to hang up successfully on a call that won't hang up on you.  If you don't wait for the full length of time needed, the unwanted call is still there.  This can go on for a good minute or longer.

      As such, those calls have the potential to interfere with emergency use of the telephone.  

      The only way robocalling will ever be acceptable is if

      a) telcos provide "called party control" that lets you hang up for 1 - 2 seconds to successfully get a new dial tone,

      b) a national standard is provided and enforced by FCC to enable called persons to press the # button to cause the calling party's automated system to disconnect immediately, and

      c) means are provided by the telcos at no added charge, to enable people such as night-shift workers (who sleep during daytime hours) to have automated call screening that requires callers to dial a specified digit in order to get through.  ("This is 510-555-2368, we don't take calls from telemarketers, if you are a family member, friend, or co-worker, and your call is urgen,t press 2 to get through, otherwise please leave a message...")

      Personally what I would like to do is use an EMP device on the manufacturers of those robocalling systems, to melt every piece of computerised equipment in their offices into puddles of goo.  However that would probably be illegal.

  •  Telephone based polling.,,, (1+ / 0-)
    Recommended by:
    psycho liberal

    doesn't it use just landlines( no cell-phones)? I'm just thinking that this form of polling ignores the type of constituency that the "leading" Democratic nominee for president most appeals to. I "never forget the Poles"(polls), but I nthink that perhaps the polls are "out of touch". The times they are a changin'.

    CHRISTIAN, n. One who believes that the New Testament is a divinely inspired book admirably suited to the spiritual needs of his neighbor. A. Bierce

    by irate on Wed May 28, 2008 at 02:26:46 PM PDT

    •  Polling Firms correct for Cellular Usage (2+ / 0-)
      Recommended by:
      irate, spookthesunset

      Trust me, successful pollsters correct for this.  

      Learn about Centrist Economics, learn about Robert Rubin's Hamilton Project.

      by PatriciaVa on Wed May 28, 2008 at 02:28:51 PM PDT

      [ Parent ]

      •  You forgot the :-).,,,,, (0+ / 0-)

        Now really, just how would they do that?

        CHRISTIAN, n. One who believes that the New Testament is a divinely inspired book admirably suited to the spiritual needs of his neighbor. A. Bierce

        by irate on Wed May 28, 2008 at 02:34:04 PM PDT

        [ Parent ]

        •  Math and Statistics (0+ / 0-)

          Just thinking off my head:

          1. You'd need to know the ratio of land-line/cell-phone usage in your polling area.  
          1. You'd need to take other statistics like the age/demographic breakdown of cell phone users
          1. You'd need to have a major in statistics
          1. Mystery statistics I'm forgetting

          Combine 1-4 in a large pot and bring to a boil.

          Or something like that.... but it is doable, trust me.

    •  it's because people who... (0+ / 0-)

      keep only a cellphone and have no landline at home are STUPID and looking to get darwinized if they ever have an emergency and can't spend 5- 10 minutes waiting to reach the right dispatcher and repeat their address two or three times.  

      Telephone systems engineer speaking here, 25 years' experience.  

      And no cellphone.  

      •  Not stupid, young... (0+ / 0-)

        ... and not yet forced to face the reality that bad things don't just happen to "other people".

        There is a definite difference, but based on your experience, it appears you are about my age, and have forgotten what it was like to be 25 years old -- either that or you were an exception (much more likely to be your recollection than the actual reality - I say this in generic terms, with no intention to claim specific insight into your circumstances.)

        "Obstacles are what you see when you take your eyes off the goal." - Anon

        by Deighved H Stern MD on Wed May 28, 2008 at 04:18:04 PM PDT

        [ Parent ]

        •  when I was 25.... (0+ / 0-)

          I had a landline with an answering machine.  I got my messages and returned my calls.  

          Nothing difficult about that.

          In fact I was an early-adopter of answering machines: in college, I was the first person to have one.  At first it slightly freaked-out my friends.  But as I explained it to them, better to leave a message than to have to keep calling until I got in.  

          My attitude toward technology is highly selective.  I use it as I choose for purposes I choose, and I don't let fads or similar nonsense dictate my choices.  I build a good bit of it for myself, including my refrigerator/freezer system (saves 1,000 KWH per year), graywater recycling system (cuts my water consumption by about 30% per month), and so on.

          I had a cellphone for a few years.  Hated like hell being subject to constant interrupts and everyone's nervous whims.  Ditched it during the dotcom crash when money was tighter than Jerry Falwell's sphincters.  Clients were thrilled: no more interruptions of expensive field time.  I was thrilled: no more ceaseless demands for trivialities.   And the brief one-liner explanation is, "I'm getting all the surveillance I need from my tax dollars, why should I pay for more?"

          I have ancient dial phones at my house, behind a high-end PBX that also provides an emergency bypass so that people who know the code can ring my bedroom and get through when I'm sleeping.  Yes, on a red phone.  

          BTW, the ancient dial phones sound better than cellular, right back to about 1925.  Before then, they sound worse than cellular.  If you're going to have antiquated sound quality, may as well have the real item to go along with.

  •  Less "prejudice" Bias with RoboCalls (1+ / 0-)
    Recommended by:

    Under certain circumstances, some may have reservations about admitting certain "biases" to a fellow human, but not to a RoboOperator.

    Learn about Centrist Economics, learn about Robert Rubin's Hamilton Project.

    by PatriciaVa on Wed May 28, 2008 at 02:27:20 PM PDT

  •  Rasmussen (0+ / 0-)

    is a Republican pollster owned by conservative Republicans...
    Let's not bolster its reputation so much in the short term that it hurts us in the long run.

  •  Best Bang For The Buck (1+ / 0-)
    Recommended by:

    I'm sticking with Kos' butt this cycle.

  •  I've stopped reading the polls... (0+ / 0-)

    As far as I can tell, they're like statistics - and everybody knows that 34% of statistics are just made up anyway.  15% of people know that.  ;-)

    "I believe in compulsory cannibalism. If people were forced to eat what they kill, there would be no more wars." - Abbie Hoffman

    by Jensequitur on Wed May 28, 2008 at 02:33:11 PM PDT

  •  It's called shoot the messenger (0+ / 0-)
  •  With lower costs (0+ / 0-)

    What's remarkable is that the robo-pollsters, since they can poll at lower costs, run far more polls than the human-using pollsters (introducing more chances of blown calls

    I would infer that their sample sizes tend to be larger?  Hence the superior accuracy?  

    "The extinction of the human race will come from its inability to EMOTIONALLY comprehend the exponential function." -- Edward Teller

    by lgmcp on Wed May 28, 2008 at 02:35:22 PM PDT

  •  Automated polling is good (1+ / 0-)
    Recommended by:

    Among other things, it eliminates inadervent bias because an interviewer might (among other things) forget to rotate names or prounce one name differently than the other.  

    "Polls are like crack, political activists know they're bad for them but they read them anyways."-Unknown

    by skywaker9 on Wed May 28, 2008 at 02:37:26 PM PDT

  •  about the word "democrat" (0+ / 0-)

    "democrat" is a noun, yet the fright wing continues to use it as an adjective (apparently in an attempt to denigrate democrats and avoid the use of the correct word "democratic" because it sounds too good to be associated with our party).  The incorrect usage should be pointed out every time it occurs.

  •  In Kentucky, SurveyUSA has done very well ... (0+ / 0-)

    ... but anyone who wants to believe the latest Rasmussen poll on Kentucky's Senate race can join the Clintons and invest in an ocean front property in Arkansas.

  •  thanks Markos (2+ / 0-)
    Recommended by:
    chrississippi, Spathiphyllum

    this sort of thing is part of what makes DailyKos great.

    Head to Heading Left, BlogTalkRadio's progressive radio site!

    by thereisnospoon on Wed May 28, 2008 at 02:41:47 PM PDT

  •  Obama just got another superdelegate from Oregon (0+ / 0-)

    Lower that count!

  •  It's sad to see Gallup so far down the list (1+ / 0-)
    Recommended by:

    In the late 80's early 90's I worked for a firm that did contract polling for Gallup, Mason-Dixon, Newsweek, and various other marketing-related research polls. The problem was that monitoring every call was impossible which lead to a few idiots thinking they could fake answers on blown calls (where people would hang up in mid-survey).  Eventually they would get caught, but the damage was already done.  

    The interesting thing to me is that I used to be able to talk most people who were getting frustrated with the survey length into staying on to complete it (which got me promoted to "coder" and "closer" btw). You can't do that with robo-polling. I would have thought more people would be comfortable quitting a computerized survey than a survey conducted by a real person.

    It makes me wonder if hiring practices have gone downhill - maybe they were relying on temp services too much (back then, the company I worked for didn't go with temp services for hiring). To get the job, I had to read several survey questions to the hiring manager and write down answers to open questions to see if I matched what was said (and to see if I could read and write clearly in English). It was a long interview process - over an hour. I can see how attractive robo-polling would be in that respect.

    So this old-school "phone jockey" is VERY surprised that the robo-polling is doing so well.

    -6.88 -6.31

    "They're all crazy. They're all crazy except you and me. Sometimes I have me doubts about you." -- Garrett Fort

    by Spathiphyllum on Wed May 28, 2008 at 02:52:31 PM PDT

  •  Rasmussen blew 2000 big time (1+ / 0-)
    Recommended by:

    and to their credit they admitted a flaw and have improved to where they have earned the right to be called one of the most accurate pollsters. Our primary inclused so many unusual social dynamics that it was very difficult to account for some thing like black trunout in Southern states as well as the fact that undecideds were very likely to vote with the "stereo type" of their demographic.

    After Obama's eighth straight victory, Penn told reporters: "Winning Democratic primaries is not a qualification or a sign of who can win the general election.

    by nevadadem on Wed May 28, 2008 at 02:57:29 PM PDT

  •  Robo-pollsters pronounce names correctly (0+ / 0-)

    and I think that is one of the reasons that they get better results.  In my life I have been polled several times by "live" pollsters who can't pronounce the name of the governor, congressman, etc., with the questions read by some disinterested person who never bothered to learn how to read out loud.  No wonder people hang up on live pollsters and no wonder they get strange results.

  •  kos, there are so many interesting points here. (0+ / 0-)

    Not the least of which is GOP folks using Rasmussen until it didn't look good.  That is funny on several points.  If robo-polls are cheaper, then I'm sure that is why the GOP used them - always work with OPM, other people's money - never use much of your own.  

    Also, do you think that there is some sort of payolla going on it terms of who in the media quotes which poll?  Why do so many use Gallup when it is #25 on the list?  I have seen Marist used quite often at #28.  Is the MSM being paid to quote these people, like advertising, or are they getting the most pessimistic numbers to present a specific viewpoint?

    I am sure that most people are wary of pollsters.  They probably think it is a veiled campaign call.  When a machine is asking a direct question then they are less likely to think someone on the other end of the phone is going to try to change their mind.

    "Write your life in ink..."

    by Caringthinkingperson on Wed May 28, 2008 at 03:00:01 PM PDT

  •  Rasmussen actually runs a bit Republican (0+ / 0-)

    Because of the large samples they can run more cheaply than live pollers, the robo-callers do tend to be more precise, achieve a "tighter shot group" if you will, around the target of the actual result.  But in 2004 and 2006 Senate races, Rasmussen was actually somewhat less accurate than the better quality live-pollers, in that its "shot group", while tight, was off-center of the true target, about 2-3 points more Republican than reality.  

    You don't have to infer any skullduggery to explain this tendency, though this tendency is undoubtedly why the right tended to like Rasmussen.  The pollsters can't sample the actual voting population before the population has voted.  So they have to give their best guess at who, among the folks they can sample (folks who can be reached by phone, and then agree to answer questions), actually will vote.  The filter of who they can sample, and who they guess will actually vote, plus a "secret sauce" adjustment for the vagaries of these filters, all impose inevitable inaccuracy, quite aside from the imprecision involved in sampling even if they could sample from a population that does not vary from the target population.  Rasmussen simply runs 2-3 points Republican at the end of all these filters and adjustments.  This is not that much less than Zogby's Democratic bias, though Zogby has tended to wild swings of imprecision.  Or rather, Rasmussen ran that way in 2004 and 2006.  Peoples' tendency to talk to pollsters may have changed, who will actually vote may have changed -- we won't know until 11/4/08.

    But until we do have a chance to compare predictions with actual results, and in the absence of any information that Rasmussen has changed its "secret sauce", for now I think we have to assume that Rasmussen still runs a tad Republican.  Given its recent results, that would make the bad news for the other side even worser.  No wonder their pundits have turned on Rasmussen with a vengeance.

    The presidency must be destroyed.

    by gtomkins on Wed May 28, 2008 at 03:05:20 PM PDT

  •  I don't trust the 2004 "results" from several (0+ / 0-)

    states, for obvious reasons.  If you can't be sure of the official tally, you can't compare polls.

    Fortunately it appears that Obama won't have to depend on any one state as a must-have, and anyway Ohio has a state government and SoS that actually are on top of electoral shenanigans, and not involved in perpetrating them.  Pity that Florida can't seem to get its act together and run an election right.  But I don't think that Ohio, New Mexico, and probably Florida in 2004 can  be used for comparing the accuracy of polls.  The results are tainted. There ARE still liberals hailing from the Deep South.

    by PolitiCalypso on Wed May 28, 2008 at 03:54:54 PM PDT

  •  talking to a machine (0+ / 0-)

    is there any evidence that suggests people who hold, oh, say, racist opinions are more likely to tell the truth to a machine than to a live person?

    "I didn't have good intelligence!"

    by el fuego on Wed May 28, 2008 at 05:21:41 PM PDT

  •  That's scary then (0+ / 0-)

    Because Rasmussen in their most recent poll has Clinton taking five states that Obama loses: Arkansas, Florida, Kentucky, Nevada, and Ohio -- a total of 66 electoral votes. Obama only takes two that Clinton loses:  Colorado and Iowa -- 12 votes. (Although Obama is very competitive in Ohio; he's virtually tied with McCain.  But Hillary beats McCain by 17 points.)

    I was hoping maybe Rasmussen was not so accurate!

  •  Insider Advantage Also Uses Automated Polls (0+ / 0-)

    They were unhappy because Indiana had a state law against robocalling of any kind, and they felt this limited their ability to produce the most accurate results possible. What isn't clear from the link below is whether they always do robopolling except for their Indiana polls, but it is clear they do at least some automated calling.


Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site