Analysts vs surveys: Aquino post-‘Yolanda’ edition

When Reuters ran an analytical piece on the “backlash” against President Aquino on Nov. 15, a week after Supertyphoon “Yolanda” swept through central Philippines, I sent a message to one of my friends in the wire agency. “Did you compare raw satisfaction (74% in Mar 2013) with NET satisfaction (+49, Sept 2013)?”

Like many at that time, my assessment of the Aquino administration’s response had swung from the initial thumbs up the day after the storm (when we interpreted the lack of news from the disaster areas as good news) to an emphatic thumbs down several days later, when the question of leadership was very much in the air (and on the Inquirer front page). “Who’s in charge here?” Indeed.

But like many others, I wanted to be fair; I wanted to base my criticism on the right facts, rightly understood. Which is why I asked my friend in Reuters; perhaps I did not make myself clear, because the answer I got was anodyne: “if I remember june and sept 2013.”

I replied with two more messages. First, having just consulted the surveys in question, I corrected my friend’s mistake. “Actually, March (not ‘one point last year’) and Sept. But 74 in March is satisfaction, 49 in Sept is satisfied minus dissatisfied.” Second, I pointed to my real concern: “Your story, by comparing apples & oranges, can be accused of misleading use of surveys. Use NET  na  lang  in both,  bagsak  din  naman.” (The entire short conversation can be read on my Twitter timeline; scroll all the way down to Nov. 15.)

The wire story was circulated around the world. It carried a resonant but unsupportable head: “Analysis: Hero to zero? Philippine president feels typhoon backlash.” In visual terms, the equivalent of the familiar hero-to-zero phrase is black and white. That is, and to belabor the point, the headline is saying that President Aquino had gone “from white to black.” Stated in those terms, it would be difficult to find anecdotal, much less statistical, support for such a transformation. (Again, I wish to emphasize that I thought the President’s initial response was terrible.) The catchy metaphor in the head, however, was not my concern.

It was the glaring misunderstanding of survey results.

In the Reuters analysis, we read: “At one point last year, Aquino, the only son of democracy icon and former president Corazon Aquino, enjoyed a 74-percent approval rating.” In fact, the 74-percent approval rating was recorded by Social Weather Stations in March 2013, not “at one point” in 2012.

Three paragraphs later, we read: “Aquino has since been accused of failing to convincingly tackle a culture of political patronage. His popularity rating sank to 49 percent in September.” But in fact the figure of 49 percent corresponds to SWS’ net satisfaction rating; that is, 68 percent satisfied less 19 percent dissatisfied. The right comparison is between the 74 in March and the 68 in September. There is a drop, to be sure, but hardly the vertigo-inducing one suggested in the comparison between 74 and 49.

Why does this matter? First, wire journalists traditionally enjoy the highest reputation among industry members. Such an obvious instance of innumeracy (nothing more; I do not impute political motives of any sort) is disappointing in a Reuters piece.

Second, and more important: It forces us to look more skeptically at the usual political analysts. (It goes without saying that that attitude of skepticism should apply to this column too.)

Here, for example, is what one of my favorite analysts, Mon Casiple, said to Reuters last November: “This could be big. If nothing happens in the next week or so, and the rehabilitation goes haywire, he will have a big political problem.” (Casiple maintained this stance until late last month; his Yahoo! News column of Dec. 24 suggests that Mr. Aquino is on a “political suicide path.”)

And here is what political science professor Benito Lim said, referring to what he saw as a damaged President: “I think he will not be popular despite the fact that he is trying his best.”

Reuters summed up its round-up of interviews with political analysts in unmistakable terms. “Political analysts say Aquino’s ratings will likely suffer in the next opinion polls, especially in the typhoon-swept central Philippine provinces that have been bastions of support.” (I must note that I thought the same thing then.)

The SWS fourth-quarter survey results are in, however, and they make the Reuters analysts look like rank amateurs.

From 68 percent in September, the President’s satisfaction rating held statistically steady at 69 percent; his net satisfaction rating remained at plus 49 percent. In the Visayas, the hardest-hit disaster areas, his satisfaction rating rose from 68 to 70 percent (a movement “paralleled” by the nominal increase in the number of dissatisfied from 19 to 21), and his net satisfaction remained statistically steady, up from plus-48 to plus-50.

I regret that I was not able to make good on a plan last month to challenge critics of the two main survey organizations to accept a dare. At a time when conventional wisdom (as represented by the Reuters analysts) was certain that President Aquino would suffer a major fall in his approval or satisfaction ratings, after his administration’s calamitous initial response to the Yolanda catastrophe, I had wanted to make a small bet—not because I was convinced that Mr. Aquino’s survey ratings would hold steady (again, I shared the view of many that “of course” the President’s post-Yolanda numbers would drop), but because I wanted the survey critics to acknowledge the competence, the fundamental professionalism, of both Social Weather Stations and Pulse Asia.

Could SWS be wrong? There is that possibility, as demonstrated by their failed 2004 exit poll. But that’s a wager I would not bet on.

* * *

[email protected]/johnnery.wordpress.com