Spain Evaluation Profile - Evaluation Systems Review 2016

II. SPAIN

Spain Division of Development Policy Evaluation and Knowledge Management (DEGCO), Ministry of Foreign Affairs and Cooperation (MAEC) Evaluation Mandate In 2012 the Ministry of Foreign Affairs and Cooperation (MAEC) was restructured.

As a result, the Directorate-General for Development Policy Planning and Evaluation (DGPOLDE), to which the evaluation unit the Division of Development Policy Evaluation and Knowledge Management (DEGCO) was attached, was dissolved. The General Secretariat for International Development Cooperation (SGCID) assumed the mandates and competencies of the former DGPOLDE, including the evaluation function.

The new, well-established Evaluation Policy (2013) is the result of the fourth Master Plan of the Spanish Cooperation 2013-2016 seeking for a better evaluation process, focusing more on outcomes, and promoting greater transparency and accountability. The overall mandate to evaluate the Spanish development co-operation policy and state-funded operations rests with the State Secretariat for International Cooperation and Ibero-America (SECIPI). The Royal Decree 342 / 2012 further provides a clear mandate around evaluation to the DEGCO.

Responsibility and scope of activities The DEGCO manages centralised strategic evaluations, which cover policy or strategic evaluations. Decentralised evaluations such as project and programme evaluations are conducted and commissioned by the Spanish Agency for International Development Cooperation’s (AECID) country offices and the operational units in the headquarters.

The DEGCO’s responsibilities include: •

leading the development of the Evaluation Policy and the Evaluation Plan

•

managing centralised evaluations

•

managing and centralise knowledge gained from the evaluation results

•

publishing the evaluation results

•

participating in joint evaluations with other international donors

•

monitoring and quality assure all evaluations within the scope of the SECIPI

•

conducting meta-evaluation of the evaluations of Spanish Cooperation

•

creating manuals, guides, methodological tools

•

providing capacity building in the area of evaluation.

The DEGCO provides the AECID operations units with advice on management of decentralised evaluations. Furthermore, some of the evaluations conducted by the AECID operations units are quality assured by the DEGCO, and the same unit also co-ordinates knowledge management and dissemination of evaluation reports. Evaluation will be integrated into all the interventions and evaluability needs to be assessed from the early phase of project cycle.

Organisational Structure and Reporting Lines The DEGCO is managed by the SGCID, who subsequently reports to the SECIPI, while the AECID is directly attached to the SECIPI. The AECID’s operations units reports directly to the Agency Management. In order to facilitate information exchange between the SGCID

EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016

195

II. SPAIN

and the AECID, a network of evaluation focal points has recently been created and is still an early stage. This network will not only disseminate evaluation results and events, but also provide capacity training to the AECID staff responsible for decentralised evaluations.

Central/main evaluation units

Programme/operational units

Other units with evaluation functions

High level policy groups or ministries

Reporting line

Lines of communication

State Secretariat for International Cooperation and Ibero-America (SECIPI)

Spanish Co-operation Agency (AECID) Main executing agency for development cooperation

Secretary-General for Development Cooperation (SGCID)

AECID Country Offices and Operations Units

Division of Development Policy Evaluation and Knowledge Management (DEGCO)

DEGCO as trainer and facilitator Network of Evaluation Focal Point

When the formulation of the Biennial Evaluation Plan commences, the SGCID informs the AECID and the other stakeholders in Spanish development co-operation. This includes the Council for Development Cooperation which is a consultative body incorporating representatives of other ministries, universities, NGOs and the private sector. The Plan includes an estimate of the costs and the timeline when the final evaluation reports should be available. The AECID submits its proposal to the SGCID for their evaluations to be included in the Plan, and the DEGCO co-ordinates the consultation process. When the consultation is finalised, the Head of the SGCID formally approves the Plan and the information is also shared to the Council and the Parliament.

196

II. SPAIN

Types of Evaluation • Thematic and crosscutting evaluations • Programme evaluations • Country evaluations • Policy/strategy evaluations • Project/activity evaluations

Spain follows the OECD DAC standards and criteria, while on a case-by-case basis other criteria are also applied.

The Evaluation Plan takes into account a balanced geographical distribution and implementation modalities. Priorities are given to the projects with the possibility of expansion and replication and with potential to generate meaningful information, to feed decision making and to contribute to accountability.

During the past five years, 500 ex-post, 50 process, 1 ex-ante evaluations, 1 evaluability assessment were conducted. The DEGCO is considering conducting more real-time evaluations, impact evaluations, evaluability assessments and synthesis of evaluations in order to approach results better.

Resources Snapshot of evaluation resources Spain

The DEGCO does not have a budget line dedicated for centralised evaluations. Both centralised and decentralised evaluations are funded within the operational budgets. In 2014, DEGCO spent EUR 98 663 from the general budget of SGCID-MAEC and managed centralised evaluations for a total of EUR 2 065 340. The DEGCO is aware of the need for a separate budget and intends to create it as recommended by the OECD Peer Review.

Head / Director / Assistant Director

Professional evaluation staff

Administrative / Support staff 8

The budget has increased through a temporary agreement with a public foundation. This has Average evaluations produced per year allowed for significant changes in the number of evaluations and new processes. Currently four staff are employed on a full-time basis for evaluation, and the remaining three staff are external consultants with non-permanent contracts. The human resources for the centralised evaluations are considered insufficient to cover various duties.

Principles of Evaluation Independence The DEGCO’s independence is supported through the organisational set-up, where it directly reports to the head of the SGCID. The organisational separation of the DEGCO is guaranteed and the validation of evaluation reports does not depend on those directly responsible for the design, management or implementation of the interventions that are subject to evaluation. The DEGCO is not to be pressured to change their reports or ratings.

EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016

197

II. SPAIN

The operations units in the AECID report to the Agency Management, which does not provide them with independence, but the decentralised evaluations are considered to be a means of learning rather than accountability. The external evaluators are recruited with clear and pre-defined criteria and procedures.

However, the DEGCO and the operations units are not completely independent in terms of budget. They report to the management that is responsible for general management.

The SECIPI is at the same time the President of the AECID and the SGCID Director has been acting AECID Director.

Competence and capacity building The DEGCO evaluation staff participate in external seminars and networks or take university courses on an individual basis. Not all evaluation staff have sufficient skills for conducting complex evaluations. Evaluations are managed in pairs and two staff members are always involved in an evaluation, which mutually strengthens the skills of the team. The DEGCO is conscious of the need for further development of internal evaluation capacities.

Discussions are underway with the National Evaluation Agency and with universities to develop a structured training plan in evaluation for the DEGCO or for AECID staff.

The DEGCO’s tasks also cover provision of training and capacity building in the area of ODA evaluation for those interested, including staff of other ministries and operations units. The aforementioned network of evaluation focal points is created as a platform for training, but the means of delivery is not yet clearly determined and no specific budget is allocated.

Transparency and participation In order to ensure transparency of the evaluation process, all evaluation-related information, from evaluation tools and plans to evaluation reports and other studies, is gathered in one place and made available on the website. The recent change in the Development Aid Law has also made compulsory the presentation of the Annual Evaluation Report to the Parliament. The AECID has a library with books and journals in the field of development co-operation that are available for own employees, researchers, students and the general public.

Spain has recently further simplified the management response system and the procedure is now being institutionalised for management. Once the evaluation report is presented, those who are responsible for the evaluation are required to objectively assess and comment on the findings and recommendations produced and prepare an improvement plan when needed.

Knowledge management The importance of an information exchange system and co-ordination between the evaluation units is acknowledged in the Policy to reflect the findings to future projects and programmes, and facilitate learning.

The Director of Evaluation participates in the annual meetings of country offices and Department Directors of the AECID. An informal network of evaluation focal points is being created to improve information sharing, evaluation culture and capacities, and knowledge management. The operational units participate in centralised evaluations through reference groups. Workshops, presentations, video conferences and country visits are organised for each centralised evaluation.

198

II. SPAIN

Co-ordination with donors and country recipients Participation in joint evaluations is encouraged for promotion of mutual accountability, contribution to evaluation capacity development and better evaluation products, and facilitating dialogue with stakeholders. Joint evaluations have recently been conducted with the European Commission, UN Women, UNICEF, UNFPA, the AFD (Agence Française de Développment) and Spain’s partner countries, such as Central American Integration System and the Government of Morocco.

When evaluations are conducted in partner countries, the evaluation is carried out in the language of the country. Spain promotes the participation of local evaluators in evaluation teams, and partner institutions participate in the reference group. There is usually a debriefing presentation in the country. Once the evaluation is finished a presentation is routinely organised in Spain and the evaluation results are disseminated in both countries.

Quality assurance The SGCID bears the responsibility of promoting the implementation of a quality control system for evaluations, such as establishing criteria, procedures and tools for reviewing the quality of evaluations and meta-evaluations. Having two evaluation staff overseeing an evaluation also supports the quality of the evaluation reports.

Note to reader: The section at the beginning of Part II entitled “Introduction and key for the member profiles” provides explanatory notes on the profiles.

EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016

199