World Bank supported Community Driven Development (CDD) Programs Measuring the Impact of ‘Engaged’ Initiatives Presented by Natasha Hayward World Bank Indonesia Nov 2014
Photo: Myanmar National Community Development Project
“People are the ends and means of development, and the impact of development on people and their societies is the measure of its success� WB Task Force on Social Development (1995)
Photo: Central African Republic: Support to Vulnerable Groups Community Development Project
What is Community-Driven Development (CDD)? “Engaged” = Empowered communities who plan, implement, and monitor investments that respond to their own priority development needs Principles • Participation: development priorities determined by communities • Empowerment: resources managed by communities • Accountability: communities monitor delivery, program tracks grievances
CDD as an ‘Engaged’ Initiative 5 KEY ELEMENTS
Categories of Projects/Contexts in which CDD Operates
Community/ Group Focus (not HH/indiv.)
Multi-sector/Integrated Service Delivery programs
Common Pool Resource/NRM Projects Participatory Monitoring
Participatory Planning
Social Funds Local Government Support/ Decentralization Projects
Community involvement in implementation and O&M
Community Control of Resources
Livelihoods/Micro-credit Projects Emergency/Post-Conflict/Post Disaster Projects
Active CDD Projects around the World* (Number and level of CDD financing by Region)
Europe and Central Asia 25 projects, US$ 0.37 billion
Middle East and North Africa 18 projects, US$ 0.81 billion
East Asia and Pacific 37 projects, US$ 2.8 billion
Latin America and Caribbean 45 projects, US$ 1.66 billion Africa 101 projects, US$ 5.6 billion
- c.300 active projects in 84 countries, all regions - Total CDD investments: $17b, averaging ~$2b per year
South Asia 50 projects, US$ 4.85 billion
* World Bank CDD database as of August 2014, which includes projects approved through June 2013.
Where is the evidence base…? 1. 2. 3. 4. 5. 6.
What is the poverty or socioeconomic welfare impact of CDD/Social Fund programs? Who benefits from these program interventions (poorest quintiles, women, ethnic groups)? Do they reach poor areas and poor households? Do the programs improve access to and use of basic services? Do they improve social capital (using the standard proxy measurements for social capital – trust, collective action, association, groups and networks)? Do they improve local governance (participation in local meetings, satisfaction and increased confidence with government officials, awareness of program activities, etc.)? In conflict‐affected areas, do CDD operations have any impact on violent conflict?
A WB report (2012, led by Susan Wong) attempted to review the evidence, looking at 17 World Bank CDD programs in South and East Asia, Africa, Latin America, and Central Asia, with ‘robust’ IEs
Demonstrated Impacts of CDD Programs • Positive economic welfare (income and consumption) outcomes in several programs • Significant improvements in education, health, and drinking water service access outcomes • Generally reaching more poor than non-poor households • Effective in delivering services quickly in post-conflict settings • Participation improves construction quality, maintenance, and lowers unit costs • However: – Limited impacts on social capital and little evidence of greater collective action outside projects – Mixed impacts on local governance
Select examples of results on poverty reduction and cost effectiveness Poverty reduction/Consumption impacts: • Indonesia: 5 – 11% increase in per capita consumption •Philippines: 6% increase in per capita consumption •India (Andhra Pradesh): 26% increase in household assets •Senegal: 65% increase in household expenditure Cost effectiveness: Philippines, Indonesia, and Nepal: between 13 to 39% lower costs of infrastructure than central government depending on the type of investments
Service delivery: Nicaragua: Net primary enrollment rates was 5 to 10 percentage points higher in social funds than in non-social fund communities Zambia: Increased use of primary health facilities compared with control Bolivia: Share of women receiving prenatal care and the share of attended births increased significantly
CDD in Indonesia - Overview GoI’s Program Nasional Pemberdayaan Masyarakat - PNPM • • • • •
World’s largest CDD program & GoI’s flagship for poverty reduction Covering all 70,000 rural villages and 11,000 urban wards in Indonesia Disburses over $2 billion annually Benefits about 1/3 of the total population Replicated around the world
PNPM provides direct financial and technical support to communities for basic infrastructure and services • • • •
Annual block grant process Facilitated participatory planning Embedded governance and fiduciary oversight mechanisms Outputs over last 5 years – 132,397 kms of roads built/upgraded, 21,234 clean water systems, 20,181 bridges, 34,443 schools
Evaluating PNPM’s Impacts? PNPM Support Facility (PSF) – a multi donor Trust Fund, led by GoI, managed by the WB, has enabled resources for a program of high quality analytic work and evaluation to accompany PNPM implementation. Analytic objectives: To provide stakeholders with empirical data regarding results and impact of PNPM Mandiri; To conduct research in greater depth regarding special topics of concern to PNPM and the field of social development; and, To support capacity building efforts for Indonesian research organizations.
Approach/skill mix: mixed methods – qualitative and quantitative Working with local partners: Akatiga; BPS; SMERU; Survey meter, etc
Evaluation Findings
Lessons Learnt: Evaluation Challenges for ‘engaged’ programs Rigorous impact evaluations take an enormous amount of time, resources, and specialized expertise Formidable technical, financing and capacity challenges Timeliness for policy/program needs Choosing tools to address complex and context specific social issues rarely have singular clear ‘solutions’ often complex and unpredictable; affected by local norms and practices can have unintended consequences – so monitoring can be as important as final outcomes the need to understand program variation Policy makers need for real time, ongoing information about policy effects and social change Ability to detect when things work; or, make mid-point corrections and adaptations when they do not Learning to inform the next generation of reform (moving beyond the question of whether a particular project worked)
So, how to both improve monitoring efficiency and impact evaluation quality and quantity?
The value of mixed methods… • Different methods give us different insights into whether, why and how a policy is working • Mitigating the challenges of internal validity and external validity Internal Validity: The challenge of attribution – is there direct project cause and effect? External Validity: Will conclusions hold for other people, in other locations and at other times?
• While robust IE is desirable, in addition: effective monitoring/reporting systems; ethnographic work; qualitative research; ‘small n’ evaluation approaches – combined in an integrated framework approach – may better serve to answer questions of why and how, in addition to whether and how much, an intervention is working
Thank you!