II. WORLD BANK GROUP
World Bank Group (WBG) Independent Evaluation Group (IEG) Evaluation Mandate The Independent Evaluation Group (IEG) is responsible for assessing the relevance, efficacy, and efficiency of the World Bank Group’s operational programmes and activities and their contributions to development effectiveness. IEG’s mandate is to carry out independent and objective evaluation of the strategies, policies, programmes, projects, and corporate activities of the World Bank Group, which includes the International Bank for Reconstruction and Development (IBRD), the International Development Association (IDA), the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA). Independent evaluation is undertaken to improve accountability and inform the formulation of new directions; policies and procedures; country, sector, and thematic strategies; lending operations; and technical co-operation. With its accumulated knowledge of successes and failures at the project, country, sector, corporate, regional, and global levels, the IEG distils lessons and shares the knowledge gained from its evaluations across the World Bank Group and the wider international development community. In 2013, IEG set out a new results framework that outlines two key areas of focus to better align our objectives and work with the new World Bank Group strategic direction. •
What Works: Deepening evidence about the results of the WBG programme and activities and their effectiveness for accelerating growth, inclusiveness, and sustainability to contribute to the achievement of the WBG’s interim target of 9% poverty and progress on shared prosperity by 2020.
•
Real-time Learning: Generating evidence on the early implementation experience of the WBG Strategy to enable mid-course corrections and promote a stronger internal culture for results, accountability, and learning.
Responsibility and scope of activities Self-evaluation (decentralised) is commissioned by the programme/project managers and carried out by operational units responsible for programmes, for example project Implementation Completion Reports (ICRS), Expanded Project Supervision Reports (XPSRs), Country Program Performance Reviews, and impact evaluations. The IEG validates evaluation work carried out by the programme units, and conducts independent evaluations of projects, country programmes, sector/thematic areas, corporate processes, and global programmes. The IEG routinely evaluates the quality of self-evaluations as part of its validation work.
Organisational Structure and Reporting Lines The Independent Evaluation Group (IEG) is an independent unit within the World Bank Group. The Director-General, Evaluation (DGE) is the head of IEG and oversees IEG’s evaluation work, which includes all independent evaluation work as well as assessment of the Bank Group‘s self-evaluation systems. The DGE reports directly to the Board of Executive Directors, which oversees IEG’s work through its Committee on Development Effectiveness (CODE). CODE has an oversight function over the IEG.
EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016
225
II. WORLD BANK GROUP
Work plans are formulated through an annual consultation process with stakeholders at various levels both externally and internally. They are prepared independently of the WBG management under the oversight of the DGE, for endorsement by CODE, and approval by the Board. While evaluation in the WBG once had separate departments for private and public sector evaluation work, this is no longer the case. Under a new structure, IEG now has an integrated Bank Group-wide structure designed to improve co-ordination and synergy between private and public sector evaluation. In October 2015, IEG restructured its General Directorate with the objectives of ensuring clearer interface and engagement with the new WBG organisational structure, facilitating increased staff collaboration, and reducing overhead. Central/main evaluation units
Programme/operational units
Other units with evaluation functions
Reporting line
High level policy groups or ministries Lines of communication
Board of Executive Directors
Committee on Development Effectiveness (CODE)
Director-General, Evaluation Independent Evaluation Group (IEG)
Operations Units/ Programme Units (conducts self-evalautions and they are assessed by the IEG)
The new structure entails closer integration of country programs, economic management, human development and corporate evaluation in one department; and finance, private sector development, sustainable development in another department.
Types of Evaluation • Thematic and crosscutting evaluations • Organisational performance evaluations • Sector-wide evaluations • Programme evaluations • Country evaluations • Policy/strategy evaluations • Project/ activity evaluations
226
II. WORLD BANK GROUP
The majority of evaluations are ex-post project assessments. Larger evaluations (thematic, sector, country) cover ongoing projects in addition to completed ones. Recently, the IEG has started to conduct impact evaluations and systematic reviews to complement larger evaluations. Ex-ante evaluations or evaluability assessments are the responsibility of World Bank Group Management.
Resources The IEG’s budget is proposed by the DirectorGeneral and approved by the board of Executive Directors. Centralised and decentralised evaluations have separate budget lines. IEG’s work plan for 2016 is based on a total budget request of USD 35.9 million, comprising a USD 34 million regular budget to be approved by the Board, and USD 1.9 million in trust funds and externally funded outputs. In 2016-2017, about 80% of resources will support the first objective of deepening evidence about the results of the WBG programmes and activities; and about 20% will focus on the second objective of generating evidence on the early implementation experience of the WBG Strategy.
Snapshot of evaluation resources World Bank
Head / Director / Assistant Director
Professional evaluation staff
Principles of Evaluation Independence The Independent Evaluation Group (IEG) exhibits independence on four criteria, namely organisational independence; behavioural independence; protection from external influence; and avoidance of conflicts of interest.
Administrative / Support staff EUR 30 600 000 (USD 34 000 000) 1% of the ODA budget 220
Average evaluations produced per year The principles of organisational and behavioural independence are supported by the fact that the IEG reports directly to the Board of Executive Directors through CODE and is thus organisationally independent from management and operational staff whose activities are being evaluated. The IEG reports its findings to the Board without the Bank Group management’s pre-clearance.
In terms of protection from external influence, the IEG is protected in many ways. (i) the evaluation unit has the ability to decide on the design, conduct, and content of evaluations without interference; (ii) the content and recommendations of the evaluations cannot be changed by an outside authority; (iii) there are adequate resources to carry out the mandated responsibilities effectively; (iv) the head of evaluation is not threatened by real or perceived interference by management concerning his or her appointment or renewal, annual performance appraisal, or compensation; and (v) the head of evaluation has final authority over personnel matters subject to following the principles of the human resource policies in the organisation.
EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016
227
II. WORLD BANK GROUP
Competence and capacity building The IEG’s evaluation capacity development programme covers three areas: (i) providing technical assistance and advice to countries and Bank staff on M&E systems and approaches to evaluations; (ii) developing resource and reference materials; and (iii) providing training/ capacity building services through International Program for Development Evaluation Training/Shanghai International Development Program for Development Evaluation Training and Regional Centers for Learning on Evaluation and Results, and limited capacity building sessions at M&E network meetings (IEG 2011). The evaluation staff in the IEG receives internal and external training as well as mentoring and coaching to address skills constraints. Professionalisation has become a key issue for the IEG. The result-based management evaluation stream aims at the harmonisation of procedures, processes and techniques to measure results and develop evidence. IEG is strengthening efforts to advocate professionalisation through creating a dedicated methods advisor function and the provision of training materials across the directorate for every evaluation work. To the operations staff, the IEG provides trainings on how to write a good self-evaluation report and how to share lessons on the quality of result frameworks. The IEG however does not provide technical assistance to specific decentralised evaluations (i.e. self-evaluations), as the IEG is quality assurer and validator of decentralised evaluations. To help partner countries develop their own monitoring and evaluation capacity, the IEG is working with the Regional Centers for Learning on Evaluation and Results Initiative, and the International Program for Development Evaluation Training.
Transparency and participation In line with the World Bank’s Policy on Access to Information, which took effect on 1 July 2010, the IEG replaced its earlier mix of disclosure policies with a single Access to Information Policy (2011) that offers greater transparency and consistency of access across the IEG units. This policy shares the five guiding principles of the World Bank’s Access to Information Policy: i) maximising access to information; ii) setting out a clear list of exceptions; iii) safeguarding the deliberative process; iv) providing clear procedures for making information available; and v) recognising requesters’ right to an appeal process (IEG 2011). Furthermore, the IEG started to make the outreach work more strategic to get greater mileage out of the evaluations. Today the IEG disseminate the information not only on the website, but also through the social media and the blog. The IEG organises and records outreach events, which are subsequently uploaded online. Currently the IEG is piloting using the shorter recorded versions to share highlights in shorter and easily accessible formats. The IEG has a formalised management response system. At completion of the evaluation, the Management has to submit a draft response to CODE and a final response after the discussion. Within 90 days, the Management has to come up with an action plan in consultation with the IEG. The IEG tracks the implementation of the recommendation for the next four years once per year.
228
II. WORLD BANK GROUP
Knowledge management The IEG has accelerated its focus on learning and knowledge sharing. After a period of piloting in 2014-2015, the IEG developed clear guidelines for the definition, selection, and processing of learning products. Also in 2015, the IEG delivered 15 major learning products that synthesised the IEG’s evaluation findings. In addition to these major products, the IEG continued to deliver a wide range of brief and focused learning products. The IEG attaches great importance to disseminating knowledge from evaluations, including through development of new knowledge management products and processes that share lessons, good practices, and key evaluation findings. Key actions recently completed in 2015 include an improved online portal for lessons learnt, implementation of workflow automation for reporting providing better work programme tracking and management, and new easier-to-use data management services which are helping IEG teams to gain access to portfolio data.
Co-ordination with donors and country recipients Over the last five years, three joint evaluations have been conducted with other multilateral development banks, the European Commission, and other bilateral donors.
Quality assurance Since 2008 the IEG has put in place a comprehensive quality assurance framework for the evaluations conducted by the IEG. This has been further refined in 2015 where the IEG continued with two new quality assessment mechanisms after evaluation completion focusing on, firstly, structured debriefing processes and, secondly, meta-evaluations performed by an independent panel to assess the quality, strengths and weaknesses, and overall conduct of a sample of individual IEG evaluations. The panel is assessing utility, feasibility, propriety and validity, drawing on standards from the Joint Committee Standards and the ECG Big Book on Evaluation Good Practice Standard.
Note to reader: The section at the beginning of Part II entitled “Introduction and key for the member profiles” provides explanatory notes on the profiles.
EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016
229