1.1 What do Service Delivery Indicators surveys measure?

BOX 1.1 What do Service Delivery Indicators surveys measure?

• What do providers know? Service Delivery Indicators (SDI) surveys directly measure teachers’ knowledge by assessing how teachers answer questions on the fourthgrade material they are supposed to teach. Health workers are presented with patient case simulations (“vignettes”) on symptoms of high-burden diseases such as malaria, diabetes, and pneumonia. Using the innovative methodology of vignettes, the SDI surveys are the only surveys of this scale that measure the knowledge of health care providers in a comparable way across countries.

• What do providers do? Visiting unannounced, SDI survey enumerators assess the level of absence among teachers by observing whether classes are unattended and among health workers by cross-checking the presence of providers with the registry of personnel. With this pioneering approach, SDI surveys have generated the only internationally comparable data on provider absence in the education and health sectors, quantifying what many have only hypothesized.

• What do providers have to work with? The quality of services is limited not only by the technical capacity and behavior of providers but also by physical resources—for example, access to clean water, electricity, and improved toilets. Although vaccines may be available in most health facilities, in many cases refrigerators cannot maintain the temperatures required to store those vaccines properly. In schools, enumerators visually check that children have textbooks with them and that each school has toilets separated by gender.

• What are children learning? Establishing a direct link between accessing care in a clinic and patients’ health status is complex. No single summary measure of health status is available in the literature, and, even if health status were easily and reliably measurable, after having seen a doctor, a patient could choose to be seen by a different doctor in a different clinic. Hence SDI surveys do not include measures of health outcomes. In contrast, the link between teachers’ quality and knowledge and children’s learning has been established in the literature. Moreover, children’s numeracy and literacy can be measured in comparable ways. When fielded in schools, SDI surveys therefore include a measure of education outcomes—children’s learning.

William and Flora Hewlett Foundation and the African Development Bank, worked together to develop novel survey tools and a methodology for measuring comprehensively the delivery of primary health care and primary school services. The first SDI health and education surveys were piloted in 2010 in Senegal and Tanzania. Since then, they have been scaled up to cover more than 15 African countries and adapted to settings outside Africa, such as Afghanistan, Armenia, Bhutan, Guatemala, Indonesia, Iraq, Moldova, and Pakistan, where work is ongoing or recently completed.

Are systems working for citizens?

SDI surveys adopt the perspective of an average patient or student, with a focus on indicators of provider presence, provider knowledge, and physical inputs that seek to proxy from different angles the quality of everyday services. The surveys are undertaken through enumerator visits to a representative sample of health facilities or schools in each country, including both public and

private facilities. SDI surveys report information about the availability and functionality of infrastructure, equipment, and other physical assets via enumerators’ direct observation. In health, for example, survey enumerators ask about and observe the condition of water sources and sanitation facilities and whether health care providers have stethoscopes, antimalarial medicines, and other essential inputs to treat patients. During education surveys, enumerators check that blackboards are visible to all pupils and that students have a pen and paper to write with, as well as other materials for proper learning. Provider presence is measured in both health and education surveys with an unannounced visit to facilities, and providers’ professional knowledge is directly tested. Finally, in schools, a sample of fourth-grade children is tested in math and language (both local vernacular and language of instruction).

SDI surveys are meant to complement routine information systems and provide greater depth. Routine information systems are the backbone of monitoring and evaluation in most countries and capture crucial information for the day-to-day management of systems, including tracking medicine stocks at health clinics and student attendance at schools. However, routine systems are unable to capture more detailed features, such as the knowledge and behavior of staff and multifaceted indicators of quality. In addition, self-reported routine data can be unreliable or missing altogether, as information management systems are still emerging in many of the countries that conduct SDI surveys.

Accountability for quality

The SDI initiative is premised on the concept of making services work for the poor, as outlined in the World Development Report 2004 (World Bank 2004). This framing emphasizes the idea that measurement and information on providers’ performance enable beneficiaries—students, parents, and patients—to demand better quality of services, thus holding frontline providers accountable. Using this “short route to accountability,” health and education systems can be improved not just through government decisions but also through an active process of citizen engagement. This principle is echoed in a recent report of the Lancet Global Health Commission on High-Quality Health Systems, which notes, “Governments and civil society should ignite demand for quality in the population to empower people to hold systems accountable and actively seek high-quality care” (Kruk et al. 2018).

Low- and middle-income countries currently allocate roughly one-third of their budgets to human development sectors. Given the magnitude of this outlay, citizens should be engaged in demanding accountability for the efficient use of public resources and ensuring that these resources are distributed effectively and fairly. But, without consistent and accurate information on the quality of services, it is difficult for citizens or political leaders to assess how service providers are performing and to bring about improvements.