II. FINLAND
Finland Unit for Development Evaluation (EVA–11), Ministry for Foreign Affairs Evaluation Mandate The Ministry for Foreign Affairs (MFA) manages and co-ordinates most of the Finnish development co-operation programme. Under the MFA, the Department for Development Policy is responsible for providing overall guidance on the implementation, planning and monitoring of Finland’s development co-operation policy, and holds direct responsibility for the operational activities for multilateral and civil society organisations development co-operation as well as humanitarian aid. Regional departments are responsible for the implementation of bilateral co-operation. The institutional arrangements and responsibilities for evaluation are defined in the Decree on the Ministry for Foreign Affairs 550/2008(1280/2013) and the Evaluation Norm 1/2015. The Unit for Development Evaluation (EVA-11) is an independent administrative unit formally responsible for the development of the evaluation system, commissioning large-scale evaluations, and ensuring its effective use.
Responsibility and scope of activities According to the Decree on MFA and the Development Evaluation Norm, EVA-11’s responsibilities include (MFA 2015): •
implementation of centralised evaluations, e.g. strategic and policy-level evaluations concerning development policy and development co-operation
•
evaluation guidance concerning development policy and development co-operation
•
capacity building and training
•
dissemination of information on evaluation results
•
overall development of the evaluation function of development policy and development co-operation, and participation in international evaluation networks and joint evaluations.
A combination of centralised and decentralised evaluations is used to measure performance at different levels. EVA-11 undertakes comprehensive and strategic evaluations to assess policy effectiveness, country strategies, financing instruments, processes, results, theme or sector-based programmes, and partner programmes (MFA 2015b). The decentralised evaluations conducted by the operational units focus on project or programme performance (e.g. appraisals/ex-ante evaluations, mid-term and final evaluations).
Organisational Structure and Reporting Lines EVA-11 became an independent administrative unit in January 2014 and reports to the Under Secretary of State for Development Cooperation and Development Policy. The evaluation unit is part of the Intervention/Project Quality Assurance Board of the MFA. EVA-11 is expected to ensure that evaluations are used for intervention planning, that the evaluability is high and that an initial evaluation plan is integrated in the funding proposal. Furthermore, EVA-11 is also part of the Development Policy Steering Group of the MFA and an expert member of the national Development Policy Committee to facilitate participatory evaluation planning and provide advice on evaluation-related issues. The planning cycle covers three years, the first of which is elaborated in detail. The ministry-level evaluation plan is prepared by EVA-11 in co-operation with policy makers, senior management, as
EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016
121
II. FINLAND
well as implementing units. The evaluation plan is discussed with the Development Policy Steering Group (KEPO) and later presented to the Under Secretary of State for approval. Both centralised and decentralised evaluations are included in the plan. The practical planning of the decentralised evaluations is done by the implementing units themselves. Central/main evaluation units
Programme/operational units
Other units with evaluation functions
Reporting line
Minister for Trade and Development
Minister for Foreign Affairs
Ministry of Foreign Affairs Under-Secretary of State for Development Policy and Co-operation
Secretary of State & other Under-Secretaries of State
High level policy groups or ministries Lines of communication
National Development Policy Committee
Expert member Development Policy Steering Group (KEPO)
Unit for Development Evaluation (EVA-11) (conducts centralised evaluations)
Intervention/Project Quality Assurance Board Meta-evaluation
Expert member Policy and Regional Departments
Operational Units (conducts decentralised evaluations)
External Experts (they are competitively recruited and carry out evaluations)
Types of Evaluation Finland applies the criteria and standards of the OECD DAC and the EU in all centralised and decentralised evaluations. The commissioner of the evaluation specifies what evaluation criteria should and should not to be applied in the Terms of Reference for each evaluation. • Thematic/cross-cutting evaluations • Sector-wide evaluations • Programme evaluations • Policy/strategy evaluations • Project/activity evaluations
122
II. FINLAND
Resources EVA-11 is currently staffed by five full-time evaluation staff. Within EVA-11, some of the fulltime positions are staffed by secondees from other departments. Centralised evaluations are funded through a specific budget, while decentralised evaluations are included in the programme budget. The indicative evaluation plan and budget are included in the project proposal stage. During the past 5 years, 15 ex-post, 5 evaluability assessments, 2 process, 1 real-time, and 3 other type of evaluations were conducted. From 2016, EVA-11 resources will be reduced by 20%. This will result in a decrease in evaluation conducted per year from 4 or 5 to between 1 and 3.
Snapshot of evaluation resources Finland
Head / Director / Assistant Director
Professional evaluation staff
Administrative / Support staff EUR 2 000 000 0.25% of the ODA budget 4-5
Average evaluations produced per year
Principles of Evaluation Independence Independence of EVA-11 is supported by the direct line of reporting between EVA-11 and the Under-Secretary of State for Development, as well as by a definition of responsibilities of the evaluation unit in the Decree on the Ministry for Foreign Affairs. All evaluations are conducted by external consultants and potential conflicts of interests are strictly assessed. The evaluation unit is responsible for the overall management of centralised evaluations. However, the decision-making mandate for the overall budget for human and financial resource rests with the MFA.
Competence and capacity building EVA-11 organises evaluation training for the MFA staff located in Helsinki and provides web-based learning also for the staff in local embassies, as well as development partners. This is done in collaboration with the MFA Staff Development Unit. A training plan is produced and regularly updated. The EVA-11 provides help desk support to other operational units and embassies with evaluation issues.
Transparency and participation Finland is committed to the International Aid Transparency Initiative (IATI) and promotes transparency and openness of information. According to the evaluation norm (1/2015), all evaluation reports are made available on the website. Public presentations are organised when more comprehensive and strategic evaluations are conducted. These presentation and related materials are also available on line on the MFA website. Policies, guidelines and plans, including the evaluation manual, evaluation norm, and evaluation plan are also publicly accessible although some of them are only available in Finnish (see references for document links). A management response is provided to all evaluations carried out by the MFA. The response to centralised evaluations is produced by a working group that represents the responsible departments or units. The Evaluation Unit acts as a secretariat, facilitating the
EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW Š OECD 2016
123
II. FINLAND
work and following up on the implementation of the recommendations. The Chair of the working group takes the prepared response, actions and status to the Under Secretary of State for her formal approval. The decentralised evaluations use the management response as part of the management cycle of the project. It is discussed with the steering groups/management groups of the relevant programmes, or included in the preparation of the following phases of the programme.
Knowledge management All programme-related and evaluation-related documents are managed and stored in the internal information management system. A database specifically for the purposes of sharing information on evaluations is under development. The database will include a search function using key words and other parameters to ensure that information is accessible to different departments, stakeholders and citizens. When conducting actual evaluations, all stakeholders are requested to support in the collection of documents and other materials for the evaluation.
Co-ordination with donors and country recipients Finland has conducted two joint evaluations in the past five years with the EU and Sweden. The MFA advocates for the development of evaluation capacity in partner countries and their participation in joint evaluations as equal partners. Two partner-led evaluations were conducted in 2012, and the use of country systems in development co-operation was assessed with counterpart evaluation bodies in 2015. EVA-11 promotes the improvement of the evaluation function through participation in international networks and conferences.
Quality assurance All evaluators are required to follow the clear guidelines on quality assurance and they are often requested to attach a quality assurance report. EVA-11 does not assess individual reports, but encourages the managers to make use of peer reviews instead.
Note to reader: The section at the beginning of Part II entitled “Introduction and key for the member profiles� provides explanatory notes on the profiles.
124