Skip to main content

View Diary: Do states have 'house effects' when it comes to polling? (68 comments)

Comment Preferences

  •  Selection bias (1+ / 0-)
    Recommended by:
    MichaelNY

    Rather than think of this as a "house effect" for different states, it's better to think of the problem this way -- that there is a selection bias involved in predictive polling that the pollsters learn to correct for adequately by assuming the bias will stay roughly the same from election to election, at least with the same electorate.  The various state electorates differ in composition from the national electorate that most pollsters have more experience at compensating for, therefore they will tend to get sytematic errors in states where that different composition also causes differences in selection bias.

    Some pollsters are only getting as low as a 2% of voters who can be reached, and then agree to answer their questions.  Obviously, most of that 98% non-participation is random.  There can't be a huge differential bias whereby the people who don't vs people who do participate are systematically likely to favor one party over the other, or the pollsters would have to apply large corrections to get predictive results.  But 98:2 is such a huge lever arm, that even tiny differential biases can exert an appreciable effect.

    Different states can have relativley small variances from the national standard, and that would be enough to make them systematically different in results.  I'm not sure if pollsters are applying different correction factors to their results from different states, but where there is enough historical data to support the needed calculations, they should.

    Some people criticize Nate Silver for being too cautious in his modeling, because he allows for a relatively large variance from expected polling accuracy.  The folks at the Princeton Election Consortium, for example, give out much tighter predictions, and you can get pretty tight confidence intervals with the huge sample sizes you get by aggregating, because they don't assume that the polls, in their aggregate, could fail systematically to follow their historical fidelity to the actual results.  But, just as the differences in differential selection bias between states can't be corrected by assuming every state's voters act like the national standard, so they can't corrected as the electorate changes characterisitics as time goes forward.  You can only correct for regional differences by observing the regional differences, and you can only correct for the temporal differences by observing voting behavior in the new time period.  But such differences, of course, can only be observed after they have happened -- which won't quite do for predictive polling.  

    The states must be abolished.

    by gtomkins on Sun Dec 23, 2012 at 09:44:03 PM PST

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site