1.4 Evaluation methodology The evaluation follows a theory-based approach in which the major change pathways in the UNDP energy portfolio are mapped, assessed and tested. It considers the role of UNDP in these pathways through the principles of contribution analysis,2 and aims to produce a plausible, evidence-based narrative to help explain how and why changes occurred, rather than to isolate and quantify the extent to which results can be attributed to UNDP. An initial theory of change (see annex 4) was developed using the UNDP Strategic Plan, 2018-2021 and the UNDP strategy note on sustainable energy.3 This was used to categorize UNDP initiatives, direct evaluative inquiry towards different sub-areas and analyse and synthesize results along the change pathways. Over the course of the evaluation, the team focused on more detailed, nested theories of change appropriate to the sub-areas of policy and technology adoption, scaling, de-risking and the ways in which often-excluded groups engage with and benefit from these initiatives. Having consolidated the project databases, the team tagged each output against the major intervention components in the theory of change. The database was then used for sampling.
1.4.1 Sampling and coverage In an initial step, the evaluation covered the full portfolio with descriptive analysis of its major components. It then purposively sampled the most mature energy access and transition projects delivered over the signature solution period. These examples provide the best demonstration of whether UNDP is helping to create lasting change and offer a longer duration through which to assess how change happened. Within this broad category, the evaluation applied further purposive sampling to ensure it represented all geographic regions, both access and transition and conflict/post-conflict settings and to ensure that it could test the UNDP approach to investment de-risking. Finally, the evaluation reviewed the findings from the mature projects against the wider portfolio to understand whether the themes remained relevant to newer initiatives and the portfolio as a whole. In total, 88 countries are the included in the sample covered by the evaluation, along with all UNDP regional bureaux and headquarters.
2 3
Mayne, John. “Contribution Analysis: An approach to exploring cause and effect.” (2008). UNDP (2016). Delivering Sustainable Energy in a Changing Climate: Strategy Note on Sustainable Energy, 2017-2021.
Chapter 1. INTRODUC TION
4
