As a statistician, with a lot of experience in both quantitative (numeric) and qualitative (e.g., interview) research methods, I literally shuddered when I read Mr. Schoen's Wall Street Journal piece -- the "first systematic random sample of Occupy Wall Street opinion" (Polling the Occupy Wall Street Crowd)-- when I actually examined the details of the actual data.
There's already been ample critique concerning Mr. Schoen's portrayal of OWS and the degree to which they are "radical" (a word he used four time in his piece), but I want to add to that criticism, pointing out a couple of things that I haven't seen reported on this travesty of research.
I'm going to refrain from a detailed analysis of the questions themselves: There were quite a few double-barreled questions that any competent pollster with even a basic class in research methods or critical thinking knows to avoid. I have bigger fish to point out (being a vegetarian, I wouldn't be frying them...)
So, first, a sample size of 200 has a margin of error of roughly +/- 7%. Any pollster who has such a high M.O.E. should be saying up front that the results being reported should be viewed cautiously. But I'll give Mr. Schoen the benefit of the doubt and call the absence of that caveat laziness.
Where his splendidly displays an utter regard for truth is where he asks about Obama's support and how folks plan on voting in 2012.
From his article:
"An overwhelming majority of demonstrators supported Barack Obama in 2008. Now 51% disapprove of the president while 44% approve, and only 48% say they will vote to re-elect him in 2012, while at least a quarter won't vote."
He seems to get this data from the following questions:
6. Did you vote in the 2008 Presidential election?
56% Yes
44% No
7. (if yes) Did you vote for President Obama
74% Yes
26% No
So, pause here and consider: Out of the 56% that reported voting in 2008, 74% voted for Obama. That means, in total, 41.4% of the sample voted for Obama (74% of 56%, using simple multiplication). What Mr. Schoen seems to have conflated is the 74% of the respondents in question #7 as representing the entire sample, which they clearly do not. So, his above statement that the "overwhelming majority of demonstrators supported Barack Obama in 2008" is wrong. I would further contend, given his other representations, that he is deliberately lying: He runs a polling firm, and worked as a pollster under Clinton, for crying out loud. This is basic stuff.
Further, what is clear in the above passage is that Mr. Schoen is claiming that Obama is losing support among this sample of protestors. Once again, he is not only wrong, he has it backwards.
Let's examine these two questions:
8. Do you plan on voting in the upcoming 2012 Presidential election?
73% Yes
2% Maybe
25% No
10. Do you plan on voting to reelect President Obama in the upcoming 2012
Presidential Election?
48% Yes
25% No
27% Undecided
So, 73% of the sample are planning to vote in 2012. And 48% plan to vote for Barack Obama. Notice that, unlike with Question #7, there is no “(if yes)” qualifier with question #10. In other words, the 48% from question #10 appears to be the response of the total sample, and not a subset that responded in a certain way to the previous question (#8).
This suggests that more folks in the sample are planning to vote for Obama than did in 2008 (48% as compared to 41.4%). Let me repeat that: More folks, not fewer. His support (as measured by voting or intent to vote) has increased. And that's assuming that none of the undecided voters will vote for him.
Now, perhaps that is not too much of a surprise, given the younger demographic of the protesters and the possibility that many of them were not eligible to vote three years ago, but that is mere supposition.
What is clear is that the actual results are the exact opposite of what Mr. Schoen is implying in the above paragraph -- that Barack Obama has lost support among this sample.
I have had undergraduate college students interpret data better than this. I can reconcile it only one way: Mr. Schoen had his results first, and interpreted the data to fit those preconceived results. In other words, he lied. And did it blatantly and badly.