Do surveys make sense?

“All good intellects have repeated… that there can be no real knowledge but that which is based on observed facts,” argued Auguste Comte, the 19th-century French philosopher who played a crucial role in the development of modern scientific methods, especially in the social sciences.

Absent the scientific method, which places facts above fiction, there will be no real political analyses but instead a collection of rumors, conjectures, and literary acrobatics. Scientific surveys, based on a meticulous combination of random yet representative samplings, are one of the wonders of modern social sciences.

In recent decades, we have seen how scientific surveys have managed to predict, sometimes with uncanny accuracy, what were initially highly uncertain electoral races. But given the increasingly unpredictable nature of our politics, and the climate of fear reigning across once vibrant democracies, are surveys still as reliable?

Let me first explain why we shouldn’t mindlessly ignore or self-servingly besmirch surveys conducted by reputable institutions such as Social Weather Stations (SWS) and Pulse Asia, which are led by some of the country’s top social scientists.

I came to know about the history of scientific surveys, and their subsequent introduction into the Philippines, through my college mentor, Professor Felipe Miranda. From what I gather, Professor Miranda, along with my fellow Inquirer columnist, Dr. Mahar Mangahas, were at the forefront of modernizing social sciences in the Philippines throughout the late 20th century, thanks to their earlier training at the University of Chicago. Their academic stint in the American institution coincided with the so-called “behavioral revolution” in social sciences, which treated human beings as broadly rational beings whose patterns of behavior can be subjected to statistical analysis.

As political science students, we in turn were expected to learn statistics and even calculus. A former professor, Olivia Caoili, once told us that conducting surveys are akin to the job of food critics: All you need to do is to randomly pick a representative portion of a dish in order to know if the whole thing is good to go. By the same token, a nationwide survey is not a census, but instead relies on demographically representative yet randomly selected respondents.

Nevertheless, there are three interrelated gaps in social sciences, including in surveys. First of all, it’s hard to predict human behavior. This is because, unlike atomic particles or climactic patterns, human beings are capable of reflection and, accordingly, radically changing their behavior at times.

Second, human “reflexivity” also means that participants in any social science endeavor may end up modifying their routine behavior when under observation. Hence, the so-called “Hawthorne effect” and “social desirability bias” problems in surveys, whereby respondents may conceal their true perceptions for social acceptability.

And this brings us to the third gap—the real problem of conducting surveys in a climate of fear. As I have mentioned in the past, including during panel talks with the heads of SWS and Pulse Asia: What is the incentive of respondents from a community ravaged by extrajudicial killings to be fully honest with their sentiments before some random stranger about those in power?

Thus, the importance of incorporating alternative modes of data gathering, which don’t rely on face-to-face surveys per se. Phone-call-based surveys, for example, have been used to assess public sentiments and electoral trends in authoritarian regimes with a considerable degree of accuracy.

A small South African software company, which relied on Artificial Intelligence and “Big Data” analysis of social media posts, proved more successful in predicting “Brexit” and Donald Trump’s victory in 2016 than most blue-chip survey agencies.

As Dr. Robin Garcia, a Filipino pioneer of digital surveys, recently told me, “While unscientific or bad online surveys have a risk of sampling bias, face-to-face surveys have a risk of non-sampling biases.” Thus, he believes that “If done scientifically with high-level statistical interventions, online surveys are the future of opinion research,” especially because they are “cheaper, faster, and can be more accurate than face-to-face surveys.”

Time will tell if digital surveys will eventually replace traditional ones. Personally, I think it’s best to combine them both and, accordingly, find a median of all reputable surveys to better understand the big picture trends.

[email protected]