The DIY social survey

Some days ago, three social science members of the National Academy of Science and Technology, including me, were invited by the Region 7 office of the Department of Science and Technology to talk to 400 senior high school students, on whatever we liked. These are students with the potential to become scientists, who might be interested in what we do.

I chose “The DIY (Do It Yourself) scientific social survey” as my topic. I think doing such a survey is well within the students’ capability. A scientific social survey is a basic resource for social science inquiry. For example, if the inquiry is, “What factors lead young people towards science,” the straightforward DIY thing to do is: ASK SOME YOUNG PEOPLE ABOUT IT.

A social science inquiry requires data that are about people, that come from people, and that are acquired by communicating with people. (“Data” is a plural word; its singular is “datum”.) Data are subjective from the standpoint of those answering the questions, but objective from the standpoint of the questioner; hence doing surveys is objective research.

Data specifically pertinent to one’s inquiry are typically not found in the wild, waiting to be harvested. Such data usually need to be cultivated, as in a garden. The people who accept the role of survey respondents (Rs) are like the garden soil and its environment from which data can grow. Having good relations with Rs is like maintaining the fertility of the garden. Respect their privacy; do no harm.

Ask WHAT? The way to design survey questions (Qs) is to anticipate the range of possible answers (As), and to be prepared for all of them. The Qs should not call for any answers that are “wrong,” or self-incriminating. Neither should any answers promise a reward; a scientist doesn’t do marketing in the guise of survey research.

An open-ended Q, like “Why are you thinking about majoring in science?” may seem straightforward, but is actually very problematic. It is like an essay question in an exam: easy to ask, but hard to grade.

Multiple-choice Qs that have a closed set of answers (hence called closed-ended, not “close-ended”) are painstaking to design, but easy to grade. A single Q will not suffice for a complex topic; the way to proceed is to disassemble the inquiry into various dimensions, and ask separate Qs for each dimension. An efficient questionnaire usually has very many closed-ended Qs, and only a few open-ended ones. Write out the questionnaire in full, in the language(s) used in the interview. Follow the script strictly, with no ad libs.

The researcher may want to ask about the R’s knowledge, values, beliefs, attitudes, opinions, and practices pertaining to the subject matter. How about backgrounders on the R, his/her family, and social connections? The number of Qs can add up fast. Ask the Rs if they can spare the time needed for the interview.

Ask WHO? Define your survey population (the people to whom your study will apply) and how you propose to sample from that population. Statistical quality requires that sampling be random. Random sampling is not a careless process; it’s as careful as shuffling the cards in a serious poker game.

Take whatever sample size you can afford. Even a sample of 100 will work, as long as you understand that the sampling error margin for a proportion is plus/minus 10 percentage points.

WHAT did you find? Arrange the data in matrix (spreadsheet) form: N rows by K columns. The rows are the observations, i.e. the Rs, and the columns are the variables, i.e. the answers to the Qs. Computations will be less tedious if you have appropriate software; but, ultimately, it’s only arithmetic. Use cross tabulations and charts to effectively communicate the findings.

Preserve the survey data; it’s quantitative history. Deeper analysis may reveal more findings. Data do not depreciate by being used and reused, any more than a book depreciates by being read and reread. Welcome replication by others. Challenge those who criticize your findings to produce their own data from an independent survey. That’s how science works.

——————

Contact [email protected].