2.1 Limitations of Phone Surveys

BOX 2.1

Limitations of Phone Surveys

Phone interviews have turned out to be a valuable alternative to primary data collection using face-to-face surveys, particularly during the pandemic. In normal circumstances, these surveys would be a useful complement to other ways of collecting data, but they would not replace face-to-face surveys, because phone surveys come with a number of limitations.

First, areas or respondents with limited network coverage or no access to phones, typically the poorest segments of the population, will be under covered in the sample.

Second, indicators that are measured at the individual level (such as employment and unemployment) may be biased because of respondent selection. Especially in countries where high-frequency phone surveys (HFPSs) are sampled from an existing nationally representative (prepandemic) survey, the respondent is often the head of household, and thus some statistics (such as employment rates) would differ from those estimated by a conventional labor force survey, which collects information from all household members.

Third, women tend to be under represented because they are less likely to be the head of household, or, in instances where a random digit dialing approach is used, they are less likely to own a phone or respond to an unknown caller.

Fourth, the length of a phone interview is limited, making it challenging to design an effective survey because the number of questions that can be asked is small and because the questions need to be short and precise for easy comprehension. One implication is that, in many instances, the ability to consider distributional impacts is limited. That occurs because the phone surveys lack the ability to generate estimates of poverty (as doing so would require a long list of consumption questions for which these surveys lack sufficient time). To estimate distributional impacts, proxy variables would have to be calculated (such as wealth quintiles) from the limited information on wealth that is collected in the phone surveys themselves. However, phone surveys that draw their samples from preexisting welfare surveys could derive the pre-COVID-19 poverty status.

Fifth, sample sizes are typically relatively small—often less than 1,500 people—to allow for a rapid turnaround, with the exception of the West Bank and Gaza survey, with more than 9,000 observations. Small sample sizes make it more difficult to break down results by subgroups.

Despite these limitations, phone surveys have demonstrated their ability to collect high-quality data. Their agility and the ability to collect data rapidly, without the need for personal presence by an enumerator, makes phone surveys a valuable tool for specific situations, such as emergencies, dangerous situations, or situations in which the respondent is mobile.