iterative process and triangulated across sources and methods. The evaluation included an innovative multi-stakeholder consultation process. 55 Protocols were developed for each data collection method to ensure rigour and suitability to the audience, especially when consulting with youth (see annex 3 and annex 9). The evaluation adhered to the United Nations Evaluation Group (UNEG) Ethical Guidelines for Evaluation (2020) and underwent detailed ethical review (see annex 3). Ethical approval for the proposed methodology was granted by an external ethical review board prior to fieldwork involving youth. The ethical protocol included, among others, a verbal informed consent provision for all key informants, written consent for “SenseMaking” and survey participants, and a protection protocol for data and human subjects (annex 10). All evaluators engaged in the projects signed the UNEG Pledge of Commitment to Ethical Conduct in Evaluation. The obligations of individual evaluators included integrity, respect, accountability and beneficence.
Data collection The evaluation collected data through five main channels: 1. Analytics of corporate data, including “web scraping” of social media. Data were extracted from the UNDP result-based management system and its financial reporting system.56 A sample of planning and monitoring reports of projects that are exclusively/partially focused on youth economic empowerment (see annex 11 for the list of sampled projects and selection criteria) for more in-depth analysis, and information available on UNDP social media and knowledge platforms, were extracted and judged against the analytical framework based on the evaluation questions. A moderated online consultation was held on SparkBlue57 from 20 May to 11 June 2021 on “youth skills development for employment”; see annex 4. 2. Comprehensive document review. The evaluation reviewed documentary evidence pertaining to youth economic empowerment interventions at country, regional and global levels. These secondary data informed interviews and focus group guides and survey questionnaires, as well as triangulation for evaluation findings. Sources are listed in annex 5. 3. Key informant interviews. Remote semi-structured interviews were conducted on a range of topics, including implementation of youth economic empowerment efforts and their results, effects and relevance in various contexts. Interviews collected feedback at country, regional and global levels from: (a) UNDP officers, including youth programme officers and focal points and selected governance and poverty reduction/sustainable development programme officers; (b) implementing partners, including national Governments and civil society; (c) peers, including representatives of other United Nations agencies, bilateral and international development partners; and (d) other partners including private sector and civil society organizations with which UNDP has worked, 55
56
57
Key stakeholders of this evaluation were engaged in three ways: (i) senior officer focal points were appointed by the UNDP Administrator to support the evaluation and answer technical queries; (ii) an external advisory panel weighed in at key stages of the evaluation process to provide state-of-the-art benchmarking with academia and comparator organizations and facilitate access to literature and expertise on youth economic empowerment; and (iii) the UNDP youth advisory panel commented on emerging findings and recommendations, making sure these would ultimately improve young people’s lives and livelihoods. Integrated results and resources framework (IRRF) reports, portfolio analysis dashboard and financial information from the Atlas planning system, along with various programmatic markers UNDP routinely collects (gender marker, disability marker, leaving no one behind marker, innovation marker). An online community engagement platform for UNDP.
CHAPTER 2. EVALUATION SCOPE AND METHODOLOGY
12
