Lately, bloggers are focusing on creating and funding polls.
Some of the questions I have seen are fundamentally flawed.
In graduate school, I took several courses in survey research. This was during the time that interactive computing replaced those pesky cards. Later in my work as a librarian I worked with survey researchers. Those were the good old days when there were no caller ids, phone blocks or cell phones!
In retirement I have been working for a telephone bank for a survey research company(market and political). My colleagues and I cringe at the quality of many of these polling questions. Often, we have to reword them for confused respondents. And that ain't good.
from The Luntz Playbook. a pdf file.
Now I'm going to list some of the most fundamental principles of America. All of
These are very important, but which is the SINGLE MOST important principle?
(Combined First and Second Choices)
52% DEMOCRACY
40% JUSTICE
31% EQUALITY
29% OPPORTUNITY
22% SECURITY
21% FAIRNESS
4% DON'T KNOW/REFUSED
Having worked three years a telephoner for a national research company which does phone banks for a variety of research companies. The surveys vary widely in quality; from my experience here are some of the problems, directly impacting the response rate:
1. Too many questions; a survey which take more than 15 minutes, including demographics. Nearly all respondents ask how long is this survey? A vague or dishonest answer usually results in a break off, a frustrating waste of time for you and the respondent.
2.Leading questions. Leading questions will add a skew to your survey. A leading question forces the respondent to a positive answer: example:
Does the fact that the no-fly list contains more than 80,000, many of them false flags such as infants, make your waiting time in line:
` 1. Much longer
2. Somewhat longer
3.Somewhat less longer
4. Much less longer
- Vaguely worded questions. Clarity is important; define terms when necessary. Make as many questions as possible multiple choice with no neutral or 'middle' answer. All questions should have an odd number of answers, such as always, sometimes, never. Five choices of answer are most efficient.
- Some surveys try to measure consistency by rewording and repeating the same question. Respondents are not dolts; they will say "you already asked me that" and hang up.
- Choose as few demographics as you can; omit those which do not directly impact the topic of your survey.
The most common are: age, political affiliation, attendance in houses of worship*; gender, ethnicity.
*A question which has come up in nearly all political polls since the 2000 election is stupid and should be avoided: are you an evangelical or born again Christian?