The SWS panel mobile survey experiment

LAST THURSDAY (April 21, 10:48 a.m.), InterAksyon.com posted that Leni Robredo got a plurality of 33 percent calling her the “best” in the vice-presidential debate on April 17. Next were Chiz Escudero with 28 percent, and Alan Peter Cayetano with 27 percent. The base of these numbers was the 66 percent of voters who said they had watched the debate.

The data came from the Bilang Pilipino SWS Mobile Survey project, a pioneering smartphone-mode survey of a panel formed from a probability-based national sample, recruited directly in the field by SWS staff, for standard face-to-face interviews on March 8-11.

The panel consists of 1,200 validated voters. From March 14 to the present, two or three survey questions have been fielded to the panel every day, from Monday to Friday. This provides, by far, the fastest scientific reading of public opinion in the Philippines (see “The Bilang Pilipino SWS Mobile Survey,” Opinion, 4/2/16).

The questions about the VP debate were fielded to the Bilang Pilipino panel on April 18, last Monday. By 11 p.m., SWS had received 632 responses, or 53 percent of the original panel. A completed sample of this size is small for a national survey, yet is scientific. The numbers should be treated with an error margin of plus or minus 4 percent.

By Tuesday afternoon, SWS had done not only the basic proportions of attitudes on April 18, but also their relation to the individual respondents’ preference for vice president, when earlier surveyed on April 14. This led to the discovery that, after the debate, the loyalty of voters for Robredo as of April 14 was greater than the loyalty of voters for Escudero as of April 14. This is the kind of analysis that can only be done if one has a panel of respondents questioned at successive points in time.

Combining online polling with probability sampling. When SWS was invited by InterAksyon/TV5 to experiment with online polling, we saw it as a chance to test the ideas of online research pioneer Prof. Willem Saris. His speech, “The web survey, the data collection method of the future,” at the 2011 Conference of the World Association for Public Opinion Research (WAPOR) in Amsterdam, said:

“Most of today’s online surveys exclude a large part of the population that does not have internet access and make use of self selection via advertisements; so called Opt-in panels. These two aspects of this data collection method violate the basic principle of survey research, probability sampling, and hinder the process of statistical inference. Yet, there is an alternative way of using internet to collect data. This approach consists of drawing a probability sample of the population and asking the selected people to participate in an online survey. People who do not have internet access are provided with the necessary facilities to participate.” In the case of Bilang Pilipino, the said facilities were donated by partners Starmobile (for smartphones), Voyager Innovations (for the platform), and Smart Communications.

“Although this procedure requires an important initial investment, this investment is compensated by lower fieldwork costs. Moreover, as the proportion of people with Internet access increases, the costs of this initial investment decreases. This approach is in use in the Netherlands and the USA and new panels based on the same principles are now created in Germany and the UK.

“This form of online data collection is faster and at least as good as face to face research, as we will illustrate. Notwithstanding its panel character this approach is also more efficient and less time-consuming for cross-section research. Given the panel structure of the design, the background information and much other information is collected only once; this saves a lot of time compared with standard cross-sectional research. Therefore, I think that this probability-based online panel is the data collection method for the future.”

The quality of the Bilang Pilipino panel. When surveyed face-to-face on March 8-11, the voting percentages of the complete original panel were Poe 29, Duterte 23, Binay 22, Roxas 19, and Santiago 3—close enough to the numbers in the standard Social Weather Survey of March 4-7: Poe 27, Duterte 21, Binay 24, Roxas 22, and Santiago 4. The lead of Poe over Duterte was 6 points in both surveys.

When surveyed online, the response rate of the panel started at 71 percent on March 14, but had fallen to 55 percent by April 14, after 24 “waves.” The fall was not smooth; recoveries to 74 and 76 occurred. Attrition in panel response is normal; hence the need for a large starting size.

(In a panel recruited by telephone from the American National Election Survey data base, only 68 percent cooperated when first surveyed online, for demographic profiling. In 21 monthly waves from January 2008 to September 2009, the response percentage fell from 67 in wave 1 to 54 in wave 21, even going as low as 47 and 49.)

In the online (small-scale) surveys of the Bilang Pilipino panel thus far, Poe has kept a lead over Duterte: it was nine points on March 18, and nine points again on March 22, but dropped to three points on March 30, and to only one point on April 13. The SWS reports give details on how members of the panel changed their minds from one candidate to another across time.

In the meantime, the full-scale Social Weather Survey of March 30 to April 2 found Duterte up by 4 points. SWS will conduct two more of its own standard surveys before May 9, for first publication by BusinessWorld.

* * *

Contact [email protected].