Are Public Opinion Polls Really Accurate?

How many times have you looked at the results of a public opinion poll and wondered if the results were accurate? Who did they talk to? Is it really possible to measure the entire nation's opinion on something by asking less than one thousand people a question or two?

Believe it or not, when conducted properly, public opinion polling is generally quite accurate. Conducting good survey research, however, is no simple task. To be accurate, the questions on a survey must be asked of a group of people--what pollsters call a sample--that is representative of the larger population. The questions themselves must also be good indicators of the opinions or attitudes the pollster is trying to measure and the questions must also be asked consistently from one person to the next. Pollsters generally worry about two sources of error in survey research: sampling error and non-sampling error.

Sampling Error

The key to accurate measurement of public opinion is the ability of a researcher to select a sample of individuals that looks and acts like the larger population they come from in every important way. For example, a sample must have almost exactly the same proportions of men and women, blacks, whites and Hispanics, Democrats and Republicans and old and young people as the entire population. In practice, representative samples are best drawn by randomly selecting individuals from the population of interest. For example, if a pollster wants to know which candidate is likely to win an upcoming election, he or she randomly "samples" individuals from the population of all voters. (People who are not going to vote on election day, although eligible to vote, are not part of the actual voting population and should not be included in the sample.) Randomness is important because it removes any bias that might creep in by allowing the pollster to select people, unintentionally or not, on the basis of their race, sex, height, availability or any number of other criteria that would make the sample unlike the population from which it is drawn.

Simply identifying a random sample, however, does not alleviate all concerns about sampling error. Indeed, one of the more difficult aspects of sampling is actually getting the people selected to be in a sample to respond to the questions on a survey. If there is any degree of systematic refusal to answer questions, e.g. more women refuse to answer questions than men, the remaining individuals in the sample who cooperate with the pollster will not be represenative of the population (there will be too many men in the sample) and the results will be invalid.

The difficulties of sampling notwithstanding, a survey researcher can, in fact, confidently and accurately generalize about the opinions and attitudes of large groups of individuals by selecting relatively small random samples of individuals from those larger groups and completing interviews with them.

Non-Sampling Error

Once a random sample is selected and cooperation is secured from all members of the sample, a pollster must still worry about "non-sampling" error. Question wording, question order, the tone of voice used by interviewers, and even the way an interviewer is dressed can all influence individual responses to survey questions. To the extent possible, survey researchers must develop and ask questions that are valid and reliable indicators of what they intend to measure. For example, if a pollster is interested in attitudes about race, asking people if they are "racist" would not be likely to yield accurate results. However, asking individuals their opinions about the comparative abilities of whites and minorities, the need for race-based public policies and other similar questions would provide a more accurate indication of how people think and feel about race.

Public opinion polls must generally be read with caution. Any one opinion poll might be inaccurate for a variety of reasons. Moreover, even accurately measured public opinion is often based on limited or inaccurate information and people are prone to change their minds. Survey research, however, when conducted properly, provides an accurate portrait of the attitudes, beliefs, opinions and preferences of the people. Such information is not readily measurable in any other way, especially in between elections. Consequently, public opinion and opinion polling are, for better or worse, critical components of American political life that are here to stay.