Subjective data, objective science

Subjectivity in survey data cannot be avoided. The simple act of asking a sample of people certain questions, and recording their subjective answers, entails faith that the data will be useful to you.

Surveys of voters’ intentions have been proven to work: The survey respondents answer honestly, and then mostly stick to their intentions at the precincts; they help candidates even if they know that voters might change their minds at the last minute.

Surveys of consumers are constantly helping private businesses to design products that will appeal to buyers. Yet businesses know that buyers’ tastes and needs change over time.The government’s monthly labor force surveys report on how many are looking for different work because they are dissatisfied with what they presently do. Such subjective dissatisfaction is not merely a matter of whim. The Bangko Sentral’s quarterly Consumer Expectations Surveys look into people’s subjective thinking on what the future has in store for them.

The government isn’t about to survey public sector corruption, though. It can’t be expected to produce data to criticize itself. The famous Corruption Perceptions Index (CPI), initiated by Transparency International, relies on ratings done by private companies that have dealings with the government. Such ratings are subjective judgment calls. The movements in CPI country rankings are treated very seriously by transnational investors and multilateral financial agencies.

Why do some journalists pooh-pooh surveys about how many people are feeling poor, or experiencing hunger or any type of pain, as merely “claims,” suggesting that the respondents are faking it, so as to be rewarded somehow? Don’t they know that the United Nations’ Sustainable Development Goals are topped by the eradication of economic suffering?There is justifiable steady interest in the World Happiness Report, which relies on national surveys of individuals rating their current position on a ladder between the worst possible life and the best possible life they can imagine. What is more subjective than happiness (on the positive side, that is)?

On the negative side of subjectivity are the people’s intense fears about infectious diseases coming from abroad, as was shown by many surveys even before the COVID-19 pandemic. (The latest report is “Social Weather Report: 69 percent of unvaccinated adult Filipinos are still skeptical to get vaccinated against COVID-19,” www.sws.org.ph, 3/16/23.)

Though a topic is subjective, scientific research should nevertheless be objective. The fundamental way to confirm social survey findings is to have multiple independent surveys on the same topic. But an original researcher is not obligated to do this.

It is up to those who question a survey’s findings to do their own surveys—and shoulder the corresponding costs, of course—so as to check for themselves if the original findings are replicated. Scientific validation is done by competition.

What a statistical test of “95 percent significance” means is that, if 20 surveys of similar sample size are scientifically done, then 19 of the 20 surveys can be expected to reach the same conclusion.

Political parties competing in election campaigns all do their own surveys, with their own survey professionals, in order to get private data that they can trust. Naturally, they will not share their own data with their rivals. The same is true for competitors in business.

The people’s feelings about a topic might fluctuate radically over time, not because the subject matter itself is subjective, but because relevant circumstances are extremely volatile. Inflation in consumer prices, in particular, spikes from time to time. Natural disasters and calamities strike without warning. The Philippines needs agility in its survey capacity to keep up with the circumstances as they come.

——————

Contact: [email protected]