Social survey ‘woes’

In a recent talk about “long-standing woes” of social surveys, I began by observing that so many newspaper editorials and columns, when they make use of a social survey, still include the qualification: that is, if surveys can be believed.

For us professional doers of scientific surveys, that qualification by some journalists—not by scientists and not by the general public, I hasten to say—who should know better, is so persistent that it has become a normal occupational hazard. But it is annoying, since it tries to question the truth of our work.

It was my friend Wolfgang Donsbach who pointed out to me the antipathy, in many countries, of journalists to scientific polling, which competes with them in reporting public sentiment. The chitchat in coffee shops, barber shops and taxis, so beloved to columnists, has been exposed as inferior to the polls. (Professor Donsbach [1949-2015] was founder of the Institute of Communication Sciences of Dresden University, editor of the International Journal of Public Opinion Research, and president of the World Association for Public Opinion Research.)

We pollsters can easily defend the truth of our work. To the question of, do Filipino survey respondents tell the truth, the answer is YES. In particular, we know from the election outcomes that Filipinos tell the truth about their voting intentions. What kind of society would we have if we Filipinos habitually tried to deceive each other, in our ordinary conversation?

Survey interviewers are trained to be very careful not to promise any consequence, either reward or punishment, from a respondent’s answer to any question. Respondent anonymity is absolutely guaranteed. Replies to the survey questions do not identify the respondents beyond simple demographic details.

To the question of, do we professional surveyors report the truth about what our survey respondents tell us, the answer is OF COURSE. Our quality controls on field work and data processing follow the principle of: One strike, you’re out.

A single case of dishonesty is ground for dismissal. If a single interview submitted by a field person has been falsified, then all her previous interviews (interviewers are typically 100-percent female) are also trashed. They must be replaced by honest ones done by another person, in order to complete the sample.

Data are precious. At SWS, falsification is treated like stealing money from a cash register. With all raw data permanently archived and ultimately open to scholars, any findings we have issued are verifiable by new, and independent, calculations.

The Filipino people appreciate surveys. Our confidence about the public’s regard is survey-based itself. Over 70 percent of Filipinos say the country would be better off if the leaders paid close attention to opinion polls. Over 80 percent say opinion polls are Good for the country, and only 5 percent say they are Bad. We borrowed these survey items from Gallup’s “polling on polls” in the United States. They show that Filipinos have the same positive feelings, as Americans, about opinion polls.

Unfortunately, Filipinos are also similar to Americans in doubting that a scientific sample of 1,000 respondents can represent the entire national population. But 1,000 is the gold standard for national opinion polling in all countries, regardless of their vastly different populations. This is an important lesson in elementary statistics; public ignorance of it is an indictment of the quality of statistics teaching.

Contact [email protected]. Done in response to an invitation from the National Academy of Science and Technology Philippines to talk on “Long-Standing Woes of Social Surveys,” at its 2019 Social Innovation Forum, Hotel Jen, Manila, 11/12/19.