2.2 Evaluation analytical framework, approach and questions

The evaluation was primarily formative and forward-looking, for two main reasons. First, it occurred while practical measures to integrate LNOB into UNDP programmes and operations were being designed or actively implemented, and therefore an organizational learning focus was timely. Second, there were evaluability challenges that limited the feasibility of a summative, or cumulative to date, assessment.35 All IEO evaluations – formative, summative or otherwise – follow the same methodologically rigorous analytical steps to derive findings, conclusions and recommendations and require a management response by UNDP.

The evaluation included a series of targeted consultations for adequate contextualization and nuancing of the proposed forward-looking strategies:

• UNDP staff (co-identified by the IEO and UNDP management) accompanied the evaluation from beginning to end through a bespoke evaluation learning group.36 • Staff from the Accelerator Labs – considered important vehicles for organizational change in UNDP – were consulted through a dedicated Accelerator Lab drop-in session on LNOB. • United Nations/UNDP staff shared insights through a SparkBlue consultation and discussion.37 • Civil society organizations from the Global South, brought together by the umbrella think-tank

Southern Voice,38 participated in two roundtable discussions (25 participants total). • A three-member advisory panel of external experts provided quality assurance and state-of-the-art benchmarking at key stages of the evaluation process (inception, data collection/analysis, draft report stage, final report stage).

In addition to the usual consultations with staff, government counterparts, United Nations agencies and donors, the evaluation was designed to capture the perspective of those left furthest behind, notably by focusing primary data collection at the community level, both rural and urban, as described below (section 2.3). Figure 3 shows a reconstructed programme-impact pathway for LNOB integration in UNDP and details the scope of the evaluation.39

35 Key evaluability challenges are: 1. An implicit theory of change exists but is not sufficiently clear to serve as a basis for analysis.

Therefore, the evaluation team developed an abridged results pathway for analysis. 2. The boundaries of “what is LNOB” and who is responsible for implementing it in UNDP are unclear. 3. The link between programmatic intent (per LNOB marker) and expenditures cannot be made due to systems limitations. 4. Monitoring data on results at intermediate steps of the results chain are lacking, as is impact-level data. 36 Evaluation learning group ToR can be found in Annex 7. The group supported ToR development and participated in key workshops, at the inception and emerging findings stages of the evaluation. 37 A moderated online consultation was held on the SparkBlue platform from 14 January to 22 February 2022, which gathered comments from UN/UNDP staff on “empowering marginalized groups”. Its summary is available on the SparkBlue page. 38 Southern Voice is an open platform bringing together 59 think-tanks from Africa, Latin America and the Caribbean and Asia. Its aim is to address the existing ‘knowledge asymmetry’ and ‘participation deficit’ in the dialogue on development by producing, promoting and disseminating evidence-based policy analysis by researchers from Global South countries. See: Southern Voice. 39 This framework was elaborated by the evaluation team in collaboration with the evaluation learning group during a workshop held on 1 February 2022. For the purpose of this evaluation, the definition of LNOB is as per the UNDP Strategic Plan, 2022-2025, p.7.: "Leaving no one behind [is] a rights-based approach centered on empowerment, inclusion, equity, human agency and human development capabilities which recognizes that poverty and inequality are multidimensional”. UNDP Strategic Plan, 2022-2025.

Integration of LNOB principles in UNDP (Equality – Non-discrimination – Equity) UNDP CORE INVESTMENTS

Programmatic

Data and analytics: Providing conceptual clarity, metrics and assessments Services: Contributing to strengthening the quality of public services; delivering services as provider of last resort, or for demonstration purposes upon request Influencing: Catalysing global partnerships; acting as a neutral broker around sensitive issues; promoting integration; leveraging resources Capacities/capabilities: Strengthening government capacities, including at decentralized level upon request; strengthening capacities of civil society organizations and national human rights institutions upon request; improving the opportunities and capabilities of the furthest behind people, groups and communities SDGs: Promoting SDG localization and people-centred SDG reporting Participation: Supporting inclusion of civil society; expanding spaces for participation of communities left furthest behind

Institutional

Conceptual clarity/programme guidance on LNOB Operationalization through various entry points, including the human rights-based approach LNOB-responsive management systems, including people management External coherence, coordination and partnerships

EXCLUDED FROM EVALUATION SCOPE: LNOB-responsive operations (procurement, logistics, fiduciary services, etc.)

INTERMEDIATE OUTCOMES

Programmatic

Data systems and knowledge Capacities to “examine, empower and enact” Access to services and quality of services Regulatory framework and oversight structures Social participation

Institutional

Organizational culture supportive of diversity, equity and inclusion Enabling environment, incl. mechanisms and resources Staff capacities and behaviour change OUTCOMES

Governments have the ability to identify who is left behind, the political will, budget and capacity to effectively include those left furthest behind, and civil society is holding them accountable IMPACT

No one is left behind The furthest behind are reached first

ASSUMPTIONS TO BE ASSESSED Clear mandate for LNOB Effective organizational leadership and support Corporate integrity/accountability mechanism Sufficient financial and human resources for LNOB integration Adequacy and comprehensiveness of interventions: (e.g. using poverty depth instead of head count, gender-responsiveness) Coherent contribution of other (national and international) actors Functioning civil society Coverage and scale (geographical) Reach (marginalized groups)

ASSUMPTIONS TO BE ASSESSED ASSUMPTIONS TO BE ASSESSED

ENABLERS Context indicators: better governance; higher human development; higher per capita income; higher gender equality