A Polling Primer

During election seasons, we are inundated with public opinion polls of all shapes and sizes. Assuming that they are all biased, many of us have simply stopped paying attention. Yet social scientists conduct nonpartisan polls all the time – the trick is telling the difference between a trustworthy poll and its many imitators.

Here are a few pointers for spotting serious, non-partisan public opinion surveys:

1. Real surveys will have at least about 1,000 respondents. You can find the number of respondents to a poll by looking for N=#, usually at the top or off to the side of the results. The mathematics of statistics is fascinating – a survey with only about 1,000 respondents can quite accurately predict the opinions of the entire US population! But…
2. Those 1,000 respondents have to be a national random sample. That way, nearly everybody has a similar chance of being called. Just imagine if you asked 1,000 people in your neighborhood about their presidential preference. Their answers are unlikely to reflect the opinions of the general US population. Only a national random sample can do that.
3. Check the Margin of Error (MoE), which should be no higher than +/- 3%. This means that if we were to ask all 300,000,000 people in the US, the actual numbers we got would be 3% greater or lesser than the results obtained with the random sample of 1,000.
4. If the survey was conducted by a major media organization, a non-partisan interest group or think tank, or a university, you can probably trust it. Many people feel that surveys conducted by some news outlets are designed to reflect those organizations’ perceived biases, but that isn’t the case. Big news corporations like CNN, Fox News, The Washington Post, the New York Times, etc., don’t actually conduct their own surveys. They hire professional pollsters to do it for them. You may not like the name of the news source that paid for the poll, but you can trust that the poll itself was conducted without bias.

Imitation Polls to Avoid:

1. Pseudo-Polls: These are advertisements or viewer-feedback gimmicks that masquerade as real surveys. A good example is when a news website asks you to vote on an issue, then reports the results. Sometime news shows do this – their goal isn’t to conduct real social science, it’s to get you to visit their websites. These pseudo-polls don’t take national random samples, so even if they have tens of thousands of respondents, they don’t have the right kind of respondents, so you can’t trust their results.
2. Push Polls: Push polls are mock surveys intended not to study public opinion, but to set you up to agree with the biases of the pollsters. Even if they call a national random sample, they ask loaded questions that lead you to answers the way they want you to. Every political party does these – they make excellent advertising and sometimes less savvy news outlets pick them up as currents events. Here’s an example of a push poll question: “Knowing that if candidate X wins the election, the US deficit will skyrocket, the economy will collapse, and fire and brimstone will rain down from the sky, how likely are you to vote for candidate X?” Here’s another one, of the sort sometimes conducted for fundraising purposes: “Were you aware that X is the only candidate with a real plan to improve national security and strengthen America’s economy?” … “Are you interested in strengthening the economy and national security?” … “How likely are you to donate money to candidate X?”

So if you get a call from a pollster, what should you do?

Well, a reputable polling organization will have its callers introduce themselves and their employer right away: “Hi, my name is Rick, and I’m calling on behalf of State University’s Public Opinion Research Center, to conduct a non-partisan survey about the upcoming presidential election.” If you don’t get this information right away, ask for it! That way, you can evaluate the purpose of the poll and decide whether or not to participate.