Switzerland Evaluation Profile - Evaluation Systems Review 2016

II. SWITZERLAND

Switzerland Department of Economic Cooperation and Development (WE), State Secretary for Economic Affairs (SECO) Evaluation and Corporate Controlling Division, Swiss Agency for Development Cooperation (SDC) Evaluation Mandate SECO’s evaluation system has since 2009 shifted from a strong accountability focus to one more on learning (OECD 2013). The Evaluation Policy drafted in the same year declares that evaluation serves for learning and accountability, but it also indicates that evaluation is not just about producing reports, but to contribute to SECO’s decision-making process and to foster continuous improvements. The Policy also defines the responsibilities and organisational arrangements guiding the evaluation function in the Department of Economic Cooperation and Development (WE).

SECO clearly distinguishes external from internal evaluation. The Policy provides different mandates to the Evaluation Officer (EO) within WE and the divisions implementing operational activities.

•

The EO is mandated to propose, commission and manage independent evaluations.

Typically independent evaluations are large-scale, such as country assistance strategies, cross-cutting issues or themes, economic co-operation instruments and impact evaluations. Independent evaluations are decided on by the external Evaluation Committee and are executed by external consultants.

•

The EO is also responsible for publishing the annual report on effectiveness, summarising the results of all external evaluations and internal reviews conducted during the elapsed year.

Other responsibilities of the EO include inter alia consolidating the overall evaluation programme for SECO; reporting on the results of evaluations; disseminating the results of evaluations (including independent evaluations and Effectiveness Reports); and organising and providing training and support to operations divisions.

On the other hand, the operations divisions are responsible for project and programmelevel evaluations (external evaluations and internal reviews). They plan evaluations and internal reviews on an annual basis, integrate the lessons learned from evaluations into the division’s work, and ensure adequate follow-up for their evaluations.

•

External evaluations are decided by the heads of operational divisions, commissioned and managed by the SECO Program Officers in charge of specific projects and programmes in the operations divisions, and executed by external consultants.

•

Internal reviews are decided by the heads of operational divisions and executed by the SECO Program Officer or by the project manager him/herself.

Organisational Structure and Reporting Lines The EO is integrated in WE and belongs to the Quality and Resources Division. The EO acts as the Secretariat for an external and independent body, the Evaluation Committee.

The EO reports to the Committee which provides oversight of the evaluation function and

204

II. SWITZERLAND

subsequently reports to the State Secretary (SECO Director). The Committee consists of evaluation experts, development experts with operational experience, civil society and parliamentarian representatives appointed by the State Secretary.

The SECO/WE evaluation programme covers all of the above-mentioned evaluations for the coming four-year period. In principal, independent evaluations are planned by the EO and approved by the external Evaluation Committee, whereas the operations divisions plan external evaluations and internal reviews with assistance of the EO. Final approval of the consolidated evaluation programme is provided by the external Evaluation Committee.

Central/main evaluation units

Programme/operational units

Other units with evaluation functions

Reporting line

High level policy groups or ministries Lines of communication

State Secretary

Evaluation Committee

Quality Management Division Evaluation Officer

Operations Divisions

External Consultants

External Consultants

Types of Evaluation • Thematic/crosscutting evaluations • Sector-wide evaluations • Programme evaluations • Project/activity evaluations

EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016

205

II. SWITZERLAND

During the past 11 years, 230 external project/programme, 11 independent, and 5 other types of evaluation were conducted. Both independent and external evaluations are in line with Swiss Evaluation Society (SEVAL) and OECD DAC standards.

Resources There are three staff in the Quality and Resources Division, working for evaluations, altogether one to two full-time staff equivalents.

Independent evaluations are financed through a separate budget line. Other evaluations (external evaluations and internal reviews) are financed within the operational budget.

Snapshot of evaluation resources Switzerland SECO

Head / Director / Assistant Director

Professional evaluation staff

Administrative / Support staff 1-2

Principles of Evaluation Independence

Average evaluations produced per year (16 evaluations during the past 11 years)

The structural independence is assured by placing the EO in a dedicated unit without operational activities, reporting directly to the Evaluation Committee. The Committee provides methodological advice to the EO, which strengthens scientific independence.

Moreover, the Committee has a final say on the budget allocation to the independent evaluations and is able to recommend to WE management on additional budget allocations, when it is considered necessary. The use of external consultants also supports independence.

Competence and capacity building Skills constraints are dealt with by external training such as the International Program for Development Evaluation Training or coaching by experienced evaluation staff or external experts. Know-how transfer and experience sharing is considered more effective than professionalisation. The Evaluation Officer also participates in a project/programme approval process to assess evaluability and promote greater learning within SECO (OECD 2013). The demand from the operations divisions for evaluation expertise is high, but the evaluation function with its limited resources (one to two full-time equivalents), has not always been able to provide all the requested support to operations divisions.

Transparency and participation Transparency is recognised to be essential for the credibility of the evaluation function in WE. Independent evaluations and reports on effectiveness are always made public with management responses and the position of the external Evaluation Committee.

The external evaluations and internal reviews are summarised in the Annual Report on Effectiveness which is made public on the website. Although management responses are standard practice for both independent and external evaluations, systematic follow-up is limited to independent evaluations.

Knowledge management All relevant web-based information, such as the Evaluation Policy and Evaluation Guidelines, are made available for the SECO staff and a wider public. SECO also has a file archive with a key word search function and has access to a detailed description of

206

II. SWITZERLAND

processes and activities of evaluations. However, an even better knowledge management system is considered to be necessary and is currently being discussed.

The EO ensures that results and recommendations from the annual report on effectiveness (summary of all external evaluations and internal reviews) are systematically discussed during workshops with the operations divisions. For independent and external evaluations, so called “capitalisation workshops” are conducted, in which the concerned operational division discusses the evaluation findings with the external consultants, encouraging learning from findings and conclusions. The EO is the moderator in these workshops.

Co-ordination with donors and country recipients The EO is responsible for exploring opportunities to undertake joint evaluations.

During the last five years, joint evaluations with other donors such as SDC, International Finance Corporation, African Development Bank, Asian Development Bank, Inter-American Development Bank, and European Bank for Reconstruction and Development have been conducted. Joint evaluations with SDC are considered as a formal interaction channel where strategic issues can be discussed together. Informal interaction between SECO and SDC is also maintained.

Quality assurance WE was one of the first federal offices in Switzerland to implement an ISO 9001 certified Quality Management System. This system serves to determine the most important and suitable processes and work methods, clearly distributes tasks and responsibilities, and documents these methods.

Furthermore, the Evaluation Committee ensures the quality of the evaluation function, while the EO checks the quality of all types of evaluation reports.

Evaluation and Corporate Controlling Division, SDC Evaluation Mandate The Evaluation and Corporate Controlling Division (E+C Division) is the main unit in charge of independent evaluation activities at SDC. The Evaluation Policy 2013 provides an overview of the evaluation architecture in SDC and a framework for evaluation standards.

The Strategy 2016-2019 for Independent Evaluation provides information on challenges to be met, lessons learned and sets out the strategic objectives and results expected for the period 2016-19. The SDC also distinguishes external from internal evaluation. The Evaluation Policy mandates the E+C Division to conduct external evaluations, while the Operational Line Units are mandated to conduct internal evaluations.

•

The SDC Directorate commissions the external evaluations (centralised evaluations) and entrusts the responsibility for planning, organisation and implementation of these to the E+C Division. The external evaluations are detached from the operational lines and typically address overarching thematic and institutional issues. Additionally, E+C Division commissions impact evaluations and at least two country evaluations per year.

Other duties of the E+C Division are to participate in international joint evaluations representing SDC; disseminate the evaluation results to the public and parliamentary commissions; and ensure transparent access to results. The Division also contributes

EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016

207

II. SWITZERLAND

to strengthening the evaluation capacities within SDC. Furthermore, E+C is in charge of conducting strategic controlling and elaborates regular steering reports for the Board of Directors. •

The Operational Line Units co-ordinate internal evaluations. Internal evaluations (decentralised evaluations) include reviews of programmes and projects, selfevaluations or impact studies. As the considered aim is to increase project performance as part of Project Cycle Management, the evaluations are commissioned by the Operational Units Lines or relevant field offices.

The Operational Line Units also contribute to corporate knowledge management and support the E+C Division in external evaluations when necessary. •

A blended approach is adopted for country strategy evaluations. They are conducted by a mixed group of external evaluators and internal peers (SDC staff). Internal peers are SDC employees with operational experience selected by the E+C Division.

Involving peers facilitates learning within SDC.

Organisational Structure and Reporting Lines The E+C Division is part of the Staff of the Directorate positioned right under the SDC Directorate. The Head of the Division directly reports to the Board of Directors (led by the Director General).

SDC has a dedicated Quality Assurance Unit that advises the Operational Line Units on conceptual and methodical aspects for internal evaluations and also provides training.

The E+C Division drafts an evaluation rolling work plan covering up to four years in consultation with the Operational Line Units. The evaluation work plan is approved every year by the SDC senior management.

Central/main evaluation units

Programme/operational units

Other units with evaluation functions

Reporting line

High level policy groups or ministries Lines of communication

SDC Director General Board of Directors

Quality Assurance Unit

Operational Line Units

208

Evaluation and Corporate Controlling Division (E+C Division)

External Evaluators

II. SWITZERLAND

Types of Evaluation • Thematic and crosscutting evaluations • Organisational performance evaluations • Sector-wide evaluations • Programme evaluations • Country evaluations • Policy/strategy evaluations

The E+C Division conducts evaluations in line with the guidance of the OECD DAC and the standards of the SEVAL and the standards established by the Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP). For the past five years, 19 process evaluations were conducted. The Division envisages more impact evaluations.

Resources The financial resources have been remained the same during the last five years. The staff increased in 2013 by one additional evaluation manager. The Division is staffed by five full-time employees of whom four are evaluation staff (three full-time equivalents).

Centralised evaluations are financed through the Division’s evaluation budget, while decentralised evaluations are funded from the project/programme budgets.

Evaluation principles

Snapshot of evaluation resources Switzerland SDC

Head / Director / Assistant Director

Professional evaluation staff

Administrative / Support staff EUR 1 000 000 0.05% of the ODA budget 4-5

Average evaluations produced per year

Independence The E+C Division’s independence is supported by its direct reporting line to the Board of Directors. The centralised evaluations are fully separated from the delivery of operations.

Competence and capacity building Skills constraints in the Division are dealt with on a team or individual basis. The Division sees an external designation or accreditation scheme for the staff as a possible means for capacity development. For the SDC headquarters and field staff, training courses focusing on essential skills such as project cycle management are provided.

SDC has piloted a new approach to the evaluation of country strategies where an independent, external evaluator will lead and the SDC staff will participate in the evaluation team to facilitate learning (OECD 2013).

Transparency and participation The SDC’s external evaluations are published on the SDC website. The evaluation reports are complemented by a management response. The management response includes an implementation plan, which is monitored on a yearly basis and reported on to the Board of Directors.

EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016

209

II. SWITZERLAND

Knowledge management The Evaluation Policy clearly states that the Terms of References, methodical approaches, evaluation reports and the management response of all external evaluations must be available across SDC. Core Learning Groups or steering groups are established in each evaluation in order to promote learning and the implementation of agreed recommendations. Regular knowledge exchanges with SDC’s thematic networks during the evaluation process also promote institutional learning.

Co-ordination with donors and country recipients One joint external evaluation has been conducted with the Netherlands during the past five years. The involvement of local consultants to participate in project and country evaluations is encouraged to strengthen the capacity of the partner countries.

Note to reader: The section at the beginning of Part II entitled “Introduction and key for the member profiles” provides explanatory notes on the profiles.

210