Surveys old and new
I’m partial to old-fashioned, tried and true, survey research methodology.
The basic data unit of this research is a friendly, personal, private, and confidential face-to-face oral encounter of a field interviewer (FI) with a respondent (R).
R’s sole compensation is the feeling of having done a civic service by assisting in research. The FI asks R a series of questions, as scripted by a questionnaire. R’s participation is purely voluntary.
Assuming R is willing to be interviewed, R may choose not to answer a particular question, or may even decide to terminate the interview. Thus, a questionnaire designer does not ask questions that are self-incriminatory or otherwise too uncomfortable to answer.
A question may offer a choice of answers (and sometimes it may accept more than one answer from the list) or it may call for an unrestricted response in R’s own words; either way, there is no right or wrong answer to any question. All the researcher wants is the answer that is most truthful for R. A successful interview calls for considerable goodwill and trust between the FI and R.
Random sampling. Survey research requires interviewing many Rs, not only one. It needs a large enough sample of Rs, drawn at random, or based on probability, to represent the group that the researcher wants to study. It does not need all members of R’s group, since that would make it an entire census, instead of a survey.
Random sampling does not mean doing it carelessly or sloppily. Randomization in survey research is actually done systematically and carefully, not unlike the shuffling of cards in a serious card game. Without sampling on the basis of probability, the science of statistics cannot be applied, no matter how many thousands of cases a study has.
Remote interviewing. Surveys by telephone or else online were developed abroad as alternative means of interviewing because they promise to save on transport costs and on the length of time for completion. Obviously, they raise the risk of a break-off in the interview. It’s too easy to hang up a phone or a gadget, without face-to-face contact with the FI. A remote interview cannot go into as many details, or take as much time, as the old-fashioned type; 15 minutes is about the upper limit for a remote interview.
There used to be directories for landline phones, from which one could draw a sample. But now landlines have given way to mobile phones, which lack area codes to hint at the user’s location. Mobile phones have no directories. I have tried, without success, to get cooperation from the telcos to access a database of their mobile numbers, for the purpose of sampling.
To construct its own database of random mobile numbers, in 2019 Social Weather Stations started asking the Rs of its face-to-face surveys if they would be willing to be surveyed by SWS again, by mobile phone. By 2020 we had accumulated over 31,000 mobile numbers of willing Rs, stratified into Metro Manila, Balance Luzon, Visayas, and Mindanao. We sampled from these numbers to do our mobile phone surveys in the months of 2020 when face-to-face work was still restricted by the COVID-19 lockdown (see my “Restarting the SWS surveys,” 5/23/20). Then in December 2020, when transportation opened up again, we resumed our F2F work.
The guarantee of survey quality is its replication. The proof that surveys are correct is their consistency with other surveys independently done. For instance, the SWS surveys on joblessness, gaining versus losing, and optimism versus pessimism do track closely to the Labor Force Survey’s rates of un- plus under-employment, and to the trends in the Bangko Sentral ng Pilipinas’ consumer expectations surveys.
Multiplicity of measurement of a phenomenon is not a waste.
The results of differently-authored surveys, if scientifically done, will tend to be similar and thus validate each other; that’s how science works. How might artificial intelligence do surveys? Suppose there could be a bot—called QBOT—with knowledge of all the survey questions in history.
Then QBOT could be tasked to design questionnaires for eliciting all the information needed on any new issues. QBOT could also be told to translate them into all the languages needed to communicate with prospective Rs.Suppose a statistics company loads up another bot—called SAMPLEBOT—with knowledge of the contact details, such as phone numbers and email addresses, of all persons having the information needed to deal with those new issues. Then SAMPLEBOT could be tasked to draw up random samples of Rs to be interviewed using the questionnaires designed by QBOT.
Suppose a fieldwork company creates an army of interviewer bots—call them INTERBOTs—and loads them with speaking ability in the relevant languages. Then it tasks them to simultaneously contact the samples of Rs identified by SAMPLEBOT, and to interview them using the questionnaires of QBOT.
I think I will create my own avatar—called MEBOT—to hold public records about me, and none of my secrets, and let it handle any queries addressed to me by the INTERBOTs.
—————-
Contact: [email protected].
