The ‘satisfied’ in surveys

In my column last Thursday (“The ‘undecided’ in opinion surveys”), I made the point that many questions asked in surveys are on issues so complex that a conscientious respondent would not be able to truthfully answer them. The result quite often, I said, is the large number of “undecided” respondents—a phenomenon that may take forms other than the explicit “don’t know” or “neutral” response.

My impression is that opinion polls have devoted too much attention to fine-tuning their sampling and weighting techniques and too little to formulating the right questions.

We know that they are measuring “something”—something that seems to vary across groups and over time. But just what that is is difficult to say.

The most recent reports of both the Social Weather Stations and Pulse Asia Research, the country’s leaders in opinion polling, illustrate the point I am trying to make here. The SWS survey, conducted in early June, confirms the finding earlier reported by Pulse Asia—i.e., the remarkable improvement in Vice President Jejomar Binay’s public satisfaction (approval) ratings. Pulse Asia uses the term “approve” (aprobado), while SWS prefers the word “satisfied” (nasisiyahan). But they more or less refer to the same thing: the public’s assessment of a government official’s performance of his or her duties.

The Vice President’s latest public approval or satisfaction ratings have come under closer scrutiny this time than in the past mainly because they do not appear to jibe with the continuing decline in his presidential preference ratings for the same period. SWS shows an improvement of 11 percent in Binay’s June 2015 “net satisfaction” rating, compared to his rating in March. Pulse Asia registers the same trend: a 12-percent rise in Binay’s “approval” rating from the last quarter.

People are wondering how these numbers can be reconciled with Binay’s 2-percent drop in SWS’s presidential preference question, and the 7-percent decline in public support for his presidential bid in Pulse Asia’s survey for the same period. The quick answer that has been given is that these numbers reflect responses to two different questions. But then, one can look at the correlation between these two ratings over time. If I’m not mistaken, this is the first time the Vice President’s positive performance rating has registered a significant increase even when public support for his presidential bid has been declining. That is the puzzle.

All this brings me to the crucial issue: What exactly are the surveys getting at when they tell people to respond to the kind of questions they pose to them? This is what SWS asks its respondents: “Maaari po bang pakisabi ninyo kung kayo ay nasisiyahan o hindi sa pagganap ng tungkulin ni (name of official) bilang (position) ng Pilipinas? Kayo ba ay lubos na nasisiyahan, medyo nasisiyahan, hindi tiyak kung nasisiyahan o hindi, medyo hindi nasisiyahan, lubos na hindi nasisiyahan, o wala pa kayong narinig o nabasa kahit na kailan tungkol kay (name of official)?”

The Pulse Asia formulation is not much different: “Mayroon ako ritong mga pangalan ng ilang mga opisyal ng ating pamahalaan.

Pakisabi ninyo ang inyong opinyon tungkol sa pagganap nila ng kanilang tungkulin nitong huling tatlong buwan. (Show rating board) Kayo ba ay talagang aprobado, aprobado, maaaring aprobado at maaaring hindi aprobado, hindi aprobado, o talagang hindi aprobado kay (name) sa kanyang pagganap bilang (position) o wala pa kayong narinig, nabasa, o napanood na kahit na ano tungkol sa kanya kahit na kailan?”

Note the last phrase in both questions—a probe on the respondent’s awareness. Unfortunately, it is not awareness of the public official’s performance of his or her duties that is explicitly being probed here, but mere awareness of the official’s presence in the media. Let us imagine ourselves being asked the same question with regard to Vice President Binay’s performance in the last three months.

First of all, it makes little sense to rate the performance of a vice president as vice president, given that the function of this position is largely that of a spare tire. In VP Binay’s case, it would make more sense to assess him in his role as presidential adviser on overseas Filipino workers’ affairs or as chair of the Housing and Urban Development Coordinating Council (HUDCC). In this regard, it might be worth probing, for instance, how much the public knows of Binay’s duties as HUDCC chair. My guess is: not much.

How many would even be aware that the HUDCC chair presides over the boards of six government shelter agencies: Pag-Ibig or Home Development Mutual Fund, National Housing Authority, Housing and Land Use Regulatory Board, Home Guaranty Corporation, National Home Mortgage Finance Corporation, and Social Housing Finance Corporation? A fair question to ask might thus be: What new policy initiatives or achievements in housing have we seen during VP Binay’s 5-year stay as HUDCC chair? I wonder how many respondents have even the vaguest idea.

Without this elementary awareness, this rating is baseless insofar as it purports to be a measure of public approval or satisfaction with a government official’s performance. What, then, does this artifact of public opinion represent? In all probability, it represents a general emotional predisposition toward a person or an institution. Call it implicit trust or call it something else, but to portray it as though it were an objective indicator of the public’s approval or satisfaction with a government official’s performance of his or her functions is simply unwarranted.

* * *

[email protected]