The long-awaited report by Gallup on how they botched their polling in the Presidential election cycle of 2012 came out yesterday: Gallup's 2012 Report
There's a lot of detail in the report, but, overall, there really isn't much new in it that dedicated followers of polling didn't already know including here on Kos including posts such as: Daily Kos/Greg Dworkin and my own diaries during last fall's cyle.
My take after the squiggle, but, here are the four basic 'Explanations' per Gallup:
1.
Gallup’s likely voter estimating. Gallup’s likely voter procedures in 2012 shifted the race four points in Romney’s favor; one point more in Romney’s favor than the average shift among other polls for which registered voter and likely voter information is available.
2.
Gallup’s regional controls on interviews. Gallup attempts to ensure geographical representation of the U.S. population in its national samples by requiring a minimum number of completed interviews by census region, and through weighting by region. However, interviewing factors can influence the percentage of completed interviews obtained in geographic units within each region. As a result, completed interviews by time zone within a region can be disproportionate, even if a region is appropriately represented. At times during the pre-election polling period, this led Gallup to underrepresent the Eastern time zone within the Midwestern and Southern regions, and the Pacific time zone within the Western region.
3.
The way in which Gallup asked about and weighted race and ethnicity. In its pre-election polling, Gallup collected race and ethnicity information from respondents, using a series of forced-choice yes/no questions for each of several race and ethnicity categories, which resulted in a disproportionate number of respondents reporting they were multiracial and American Indian/Alaska Native.
4.
The use of a listed landline instead of an RDD list-assisted landline sample frame. Gallup’s experimental research in the spring of 2013 showed that on an unweighted basis, the listed landline sample in use in 2012 consisted of older and more Republican respondents than the RDD list-assisted landline sample. The listed landline and cellphone sample frames also underrepresented dual cellphone and landline users whose landline was unlisted, which required significantly more weighting of respondents interviewed on cellphones.
My simple take on Gallup's mea culpa is: We told you so. Everybody from the Daily Kos to The Huffington Post to Nate Silver warned Gallup about their issues for months heading into the election. In fact, this is the THIRD Presidential Election cycle in a row where Gallup has been off by 2.4% to 5% of the final vote. And, don't forget that this year's error of 5% was because Gallup took a break from polling during Sandy. Their pre-Sandy polls had Romney up by as much as 7%. If Gallup hadn't been able to "Copy Other Pollsters' Homework" during that hiatus, Gallup could have had a ELEVEN PERCENT debacle on their hands (as I argued in my Diary at the time: Will Gallup Have the Nerve?
As to the Four official Gallup Explanations:
1. Gallup fell for the 'enthusiasm gap' scenario that had overtaken much of the media for the year leading up to the election. I think it was based in part on the foaming at the mouth Tea Party movement that was so vociferous in their hatred for the President that any progressive-leaning voter that was interviewed by Gallup looked mild in comparison. That, combined with Gallup's likely voter screen which they admit "are more heavily weighted toward past voting behavior than other firms’ questions," created this disaster. Young and minority voters tend to have a more checkered voting history than older right-wingers (not to mention, by definition, First-Time voters who have no history). It's a fact, and it's why sites like Kos are so vital in turning out the vote in NON Presidential-year elections. Bottom Line: Gallup has to do a far better job in letting in more younger and minority voters in their Likely Voter screen - If someone says they are going to vote, assume they will all other things being equal. Remember, that Gallup's REGISTERED Voter polls have consistently out-performed their LIKELY Voter polls for years.
2. The regional and time zone problem. This seems like a technical area with very little real impact on their overall results. I will leave this one to Kos readers with more intimate knowledge of polling science to deduce. Bottom Line: Little impact on Gallup's final results, IMHOP.
3. Race and Ethnicity. This has been a bugaboo for years with Gallup. Their samples tend to be whiter and older than the electorate as a whole. If the wording of the report is true and their samples truly showed a "disproportionate number of respondents reporting they were multiracial and American Indian/Alaska Native", that should have stopped the higher ups at Gallup dead in their tracks. According to the latest census, the Native American population of the U.S. is 0.8%. Anything signficantly higher should have been a gigantic red flag. By implication, it seems that Gallup, again, had a signficant under-count of Hispanics and Blacks in their samples. Bottom Line: See #1. The Gallup voter screen strikes again. Until Gallup comes up with samples that resemble 2012/3 and not 1980, they will continue to lag other pollsters in accuracy (Romney actually got a similar percentage of the White vote as Reagan did in whalloping Carter: Romney wins White Vote like Reagan in '80).
4. Cell phones, cell phones, cell phones. Again, everybody told Gallup: "We Told You So". As in #3, Gallup seemed to have skipped the last decade as far as the demographics of this nation. In an era where estimates as high as 1/3 of the nation is cellphone only, Gallup was really only using LISTED phone numbers, primarily HOME phone numbers!? How many folks' Cell Phones are listed? Bottom Line: In a mobile society, Gallup has to find ways of reaching a broader cross-section of society as it exists TODAY.
I eagerly look forward to Rasmussen's similar mea culpa - fat chance, eh?