Survey reporting 101
One of the hazards of giving a multifaceted survey report is that the media get more leeway to select items of their liking. Offer the media a buffet, and they’ll only take what dishes they like, and not try to taste others.
“Perception of graft surges,” said the Inquirer’s headline (1/16/2014), with subhead “More top execs see a lot of corruption,” over a piece about the 2013 SWS Survey of Enterprises on Corruption that I presented at the Good Governance Summit, at the Philippine International Convention Center this week. Beside the piece was a table, “Corruption in the public sector,” giving percentages of respondents answering “a lot,” “some,” “a little,” or “none” in 2013 and 2012. The data behind the headline were the percentages for “a lot,” being 56 in 2013 and 43 in 2012.
A headline is supposed to state the heart of a report. The phrase “a lot” is being bandied about, as in Amando Doronila’s “Businessmen see ‘a lot’ of corruption in gov’t” (Inquirer, 1/17/2014). (I know that a news writer, in this case Ms Ana Roa, is usually not the same person doing its headline or tables. Incidentally, my contract with the Inquirer specifies that my column titles may not be rewritten; I required it based on experience of my titles being messed up by another paper.)
My presentation was accompanied by a media release, posted simultaneously on the SWS website, of five pages of text and 28 charts, titled “The 2013 SWS Survey of Enterprises on Corruption: the fight against public sector corruption has MIXED FINDINGS” (caps in original). In other words, although SWS summarized its findings as partly good and partly bad, the newspaper headline elected to focus on a bad part.
Not totally bad news. The SWS media release enumerated 15 separate findings, starting with: “1. Those seeing ‘a lot’ of corruption in the public sector rose to 56% in 2013, from the record-low 43% in 2012. Nevertheless, the 56% in 2013 is the second-lowest since 2000. [Chart 1].” The second sentence of this finding was ignored by the Inquirer story. The original Chart 1 also showed the percentage of “a lot” as 64 in both 2009 and 2008, and between 60 and 77 in the seven previous surveys in 2000-2007. The Inquirer front-page table, on the other hand, only cited two years’ worth of data.
Yes, 2013 was unfavorable versus 2012 in particular, but it was favorable versus nine earlier periods, as far back to 2000. This particular survey finding is a case of ‘two steps forward, one step back.’ A slightly longer time-perspective would drastically change the appearance of the very same item from which the headline came; I assume the table-maker realized that.
Some totally good news. Here, on the other hand, is finding No. 12, which is entirely good news: “12. Solicitations of bribes from enterprises are FALLING. A record-low 44% say they were solicited for any of seven listed bribes, [an] improvement from 50% in 2012 and 60% in 2009” (caps in original). This item comes from asking the enterprises whether they had been solicited for a bribe in the past year, regarding a specific transaction. It therefore reports experience, not merely perception.
The types of bribes are: 1. getting local government permits/licenses, 2. paying income taxes, 3. getting national government permits/licenses, 4. complying with import regulations, 5. supplying government with goods/services, 6. collecting receivables from government, and 7. availing of government incentives. The businessmen’s experiences of being asked for bribes of types 1, 2, 3 and 5 are record-lows in 2013, for the entire series of 11 surveys ever since 2000; experiences for types 4, 6 and 7 are all less in 2013 than in the last few years of the previous regime.
It’s not my work to put the message of “mixed findings” into a sexy headline. But why should it be impossible for truthfulness and sexiness to get along in journalism?
* * *
Another Inquirer report, “P-Noy: We govern based on facts, not on surveys” (by Michael Lim Ubac, 1/15/2014) started thus: “President Aquino yesterday blamed the sudden dip in his approval ratings to demographics, claiming many of the respondents in the latest Social Weather Stations (SWS) survey may have been the victims of Supertyphoon ‘Yolanda’ and other disasters this year.”
The premise of the opening sentence was wrong. There was no such dip. P-Noy’s net satisfaction rating was exactly the same, at +49, in the SWS surveys of December and September of 2013. Neither was there a real drop in the Pulse Asia series, contrary to Mr. Doronila’s appraisal.
So, didn’t Yolanda make a difference? To find out, just wait for a forthcoming special SWS report on how the supertyphoon victims fared, compared to nonvictims, including implications for P-Noy’s rating; I won’t scoop it now.
Further on, Mr. Ubac wrote: “He [P-Noy] had been told that many of the survey respondents were from the Visayas and Mindanao.” Whoever told that to the President implicitly made the unfair claim that SWS unprofessionally altered the regional weighting of its data. The SWS area-weighting is 14 percent for the National Capital Region, 44 percent for the Balance of Luzon, 19 percent for the Visayas, and 23 percent for Mindanao, as guided by official population statistics. Our one-time increase in the Visayan sample size (to 650, from the normal 300) was only done to sharpen the December 2013 survey findings in that area, and did not change the Visayan weight in the national total.
* * *
Contact SWS: or [email protected].
