(Over)interpreting opinion polls

Berkeley, California — As an opinion columnist who occasionally comments on our national politics, one of the things I often get asked is whether I believe opinion surveys, especially those on public officials’ approval ratings and voter preferences for candidates running for national office.

I usually begin my response by saying that I recognize opinion polling as an imperfect science, one that, despite its clear limits, nonetheless has theoretical and methodological foundations—as well as a track record both globally and locally. Sure, there have been spectacular failures like Donald Trump’s unexpected victory in the US: a prognostication made far more complicated and confounding by the existence of the electoral college. But in the main, surveys have tracked well with election results. Vice President Leni Robredo’s ascendancy in 2016, for instance, was faithfully chronicled by polling that showed her rise from (as per Social Weather Stations) 3 percent in September 2015 to 28 percent in April 2016, ultimately beating Ferdinand “Bongbong” Marcos Jr. by a hair.

I hasten to add that not all polling companies are equal: I would trust those with a track record that spans several electoral cycles, as well as those who are helmed by people with integrity and academic backgrounds, in contrast to those who are led or owned by people with a clear political agenda.

Notwithstanding my overall sense that surveys conducted by respectable firms are credible, however, I don’t think they are immune to criticism on methodological and theoretical grounds, and, more importantly, the way they are overinterpreted and fetishized in public discourse.

For instance, one valid critique during the Duterte years was whether there was some kind of “fear factor.” Pulse Asia president Ronald Holmes admitted as much in a Rappler interview with Lisandro Claudio back in October 2020, at a time when many were confounded with President Duterte’s ability to maintain high approval ratings: “You can’t rule out that there are some people whose prevailing sentiment is apprehension or fear … it is difficult to measure and difficult to rule out its possible influence.”

Even if fear were a factor with Mr. Duterte, however, it has less explanatory power in the case of Marcos Jr., who Mr. Duterte himself has dismissed as a “weak leader” and a “weak and spoiled child.” Of course, that Marcos Jr.’s alliance with Sara Duterte can be seen by the people as a tacit support from the regime, but the fear argument is less convincing outside of people’s opinion of Mr. Duterte himself.

Another critique is that the sampling may not be representative of the electorate. Relying on a few thousand respondents to account for the sentiment of millions, survey companies rest on various assumptions and projections, which, if erroneous, can lead to inaccurate results or a wider margin of error.

Beyond the above concerns (which survey companies should do well to allay by being proactive in addressing people’s questions on methodology), there are also objections based on the hypothesis that polling can lead to “mind conditioning” or a “bandwagon effect,” shaping (and not just reflecting) public opinion. The empirical data in other countries reveal a mixed picture, and the significance of such an effect (and other hypothetical effects like rooting for the underdog) remains to be more robustly studied in the Philippines. Regardless, such is the valence of this view that some countries around the world, like Singapore and Italy, enforce some kind of ban on publishing pre-election and exit polls.

What’s clear, however, is the fact that surveys tend to be overinterpreted by the people. Often conducted weeks before they are released, surveys are often perceived as a real-time indicator of people’s sentiment, even as various events may have led to shifts in the intervening time between the conduct of the survey and the publication of its results. The probabilistic nature of surveys, moreover, tend to be glossed over in favor of generalizations about their “truth-ness”—not to mention knee-jerk accusations of bias whenever the results don’t favor one’s preferred candidate. Bewildering as a 60-percent or even 54-percent support for a dictator’s son can be, the reality is that millions of Filipinos have been victims of disinformation for decades.

If there’s anything that opinion polls definitively tell us, however, it’s that people change their minds about politicians all the time, and that a few weeks can make a big difference. This should give us hope that we can still convince them to support our candidates of choice in the only survey that truly matters.

—————

[email protected]