Canada Evaluation Profile - Evaluation Systems Review 2016

II. CANADA

Canada Development Evaluation Division (DED), Global Affairs Canada Evaluation Mandate In July 2013, Canadian International Development Agency (CIDA) and the Department of Foreign Affairs and International Trade (DFAIT) amalgamated to create the Department of Foreign Affairs, Trade and Development (DFATD). In November 2015, the DFATD was renamed Global Affairs Canada.

The two evaluation teams that existed previously in the different departments are maintained, keeping their complementary functions. One unit focuses on evaluating foreign affairs and trade-related issues, the other on development programming. Both of them have some distinct accountability as well as some that overlap. The modalities of their co-operation are currently being defined.

The Treasury Board Policy on Evaluation (2009) provides each department in the government the mandate to conduct evaluations for its own activities. The Policy states that evaluation should provide an evidence-based, neutral assessment of the value for money (i.e. relevance and performance [effectiveness, efficiency and economy]), of the government programmes to inform decision making and to support accountability, managing for results and policy and programme improvements based on lessons learned and best practices.

Global Affairs Canada’s Development Evaluation Division (DED) is part of its Strategic Policy Branch. The DED evaluations inform programme effectiveness and programming tools, as well as priority and policy setting.

The Policy on Evaluation requires that 100% of direct programme spending be evaluated over a five-year cycle. This Policy is applied to all departments across the government and is in the midst of being updated. The new policy is expected to bring performance monitoring and evaluation closer together and attempt to increase the flexibility with which it can be applied in different contexts.

Responsibility and scope of work The DED’s responsibilities cover: •

developing the Evaluation Plan which focuses on corporate evaluations, but also includes the decentralised evaluations planned by Programme Branches

•

ensuring neutral, rigorous and cost-effective corporate development evaluations

•

disseminating evaluation knowledge across the Department to promote organisational learning

•

instituting a systematic approach to track the implementation of recommendations in management response and action plans

•

providing technical advice, training and quality assurance services to Branches

•

revising and advising on the accountability and performance provisions to be included in Cabinet documents

•

ensuring continuous access to external qualified evaluation consultants

•

developing and maintaining beneficial strategic alliances with key stakeholders internally and externally

•

supporting the Development Evaluation Committee to ensure effective governance and oversight.

EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016

97

II. CANADA

While Programme Branches fund and manage their own project-level evaluations, the DED provides technical advice and quality assurance services for them. The decentralised evaluations are at times jointly, co-managed with other donors and/or development organisations.

All, or part, of corporate and branch-led evaluations are usually contracted to external consultants. Evaluations are always carried out in collaboration with DED.

Organisational Structure and Reporting Lines Central/main evaluation units

Programme/operational units

Other units with evaluation functions

Reporting line

High level policy groups or ministries Lines of communication

Deputy Minister of International Development

Strategic Policy Branch Assistant Deputy Minister

Unencumbered access (when required)

Development Evaluation Committee

Programme Branches (Each branch has its own Assistant Deputy Minister reporting to relevant Deputy Minister)

secretariat

International Assistance Envelope Management

Development Evaluation Division (DED)

The Head of Development Evaluation reports to the Director-General of the International Assistance Envelope Directorate (IAED) who subsequently reports to the Assistant Deputy Minister of the Strategic Policy Branch. The International Assistance Envelope is a dedicated pool of resources that enables the Government of Canada to deploy its international assistance nimbly and responsively. The IAED determines international assistance priorities and make broad funding decisions. As per the Policy on Evaluation, the Head of Evaluation has unencumbered access to the Deputy Minister in charge of development co-operation in Global Affairs Canada, as required.

As per the Policy on Evaluation, Global Affairs Canada has a Development Evaluation Committee which is assigned the responsibility for advising the Deputy Minister on all development evaluation and evaluation-related activities of the department. It is chaired by the Deputy Minister, supported by the Head of Evaluation on evaluation matters and structured with specific roles and responsibilities.

The Committee comprises external experts and senior executives who are appointed by the Deputy Minister on the advice of the Head of Development Evaluation. While the Committee functions as an advisory body to the Deputy Minister on development evaluation activities, the Committee supervises the DED as its secretary. The Committee gives external expertise and perspectives to strengthen evaluation-related deliverables, serves to help ensure neutrality and an additional layer of quality control. For instance it provides independent review and advice on evaluation and recommends approval of final evaluation reports. The Committee also helps leverage evaluation results to strengthen policy and programming, and position the department’s development evaluation approach within a broader international context, and reviews of the adequacy of the resources allocated.

98

II. CANADA

The Development Evaluation Plan covers five years and involves various actors in the formulation process. Programme Branches review the units of accountability to be evaluated and negotiate the appropriate timing for evaluations within the five-year period, taking into account programme realities and the context in implicated countries. Policy makers help identify thematic areas requiring evaluation and advise on the prioritisation of other evaluations. At senior management level, all Assistant Deputy Ministers are required to approve their Branch-specific evaluations noted in the overall plan and the final plan is then approved by the Deputy Minister.

Types of Evaluation According to the Evaluation Plan for the past three years, the types of evaluation the DED carries out cover: country/regional programme evaluation, thematic evaluations, and reviews of the development effectiveness for multilateral organisations. DED follows the OECD DAC Principles for Evaluations of Development Assistance (for instance, in addition to the Treasury Board Secretariat (TBS) Policy on Evaluation requirements, the DED also considers the sustainability criteria laid out in the said principles).

Resources After the organisational restructure, the number of staff has been maintained. The DED has 19 full-time positions in total.

According to the Five-Year Development Evaluation Work Plan 2015-2016, in order to carry out the planned evaluations of the year, CAD 2 million for FY 2015-2016 is budgeted from a separate corporate budget for centralised evaluations, while decentralised evaluations are funded within the programme budget. For centralised evaluations CAD 1.7 million is allocated for the salary of 19 full-time equivalents.

Principles of Evaluation

Independence The DED is part of the Strategic Policy Branch and free from the operations branches and the decision-making process. The Head of Development Evaluation is given a direct access and reporting line to the Deputy Minister of International Development. The aforementioned Development Evaluation Committee provides independent review and advice on development evaluation, which ensures neutrality of evaluations.

Snapshot of evaluation resources Canada

Head / Director / Assistant Director

Professional evaluation staff

Administrative / Support staff EUR 1 300 000 0.1% of the ODA budget 8

Competence and capacity building Training and retention of trained staff is a Average evaluations produced per year priority. The DED has identified the need for more experienced evaluators due to the increasing complexity of the evaluations. The DED is considering engaging in an external designation or accreditation scheme for measuring the competence of the staff and exploring this issue with the broader Federal Government and the evaluation community.

EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW Š OECD 2016

99

II. CANADA

Transparency and participation Transparency is considered highly important in Canada and various efforts have been made since 2011. An Open Data Portal was launched, which made statistical data and other information on Canada’s international assistance available. As a member of International Aid Transparency Initiative (IATI) and Open Government Partnership, Canada is now also committed to broadly publicising more information on its development co-operation activities and results. However, not all decentralised evaluations are made available publicly and their dissemination is at the discretion of those who commissioned the evaluation.

All evaluations include recommendations. It is mandatory that management responses be provided for each recommendation and made public. They must include concrete and time-bound actions. Accountability for ensuring that actions are completed is at the Assistant Deputy Minister level, with ongoing monitoring by the Deputy Minister.

Knowledge management The Global Affairs Canada website makes a wide range of information on Canada’s development co-operation activities available. It includes documents such as country strategies, evaluation reports, plans and dissemination materials such as newsletters, photos, videos and an interactive map of projects funded by Canada.

Co-ordination with donors and country recipients Canada has participated in joint evaluations with other bilateral and multilateral organisations actively. Since the previous study, Canada participated in joint evaluations led by European Commission and the Netherlands. It also funded and jointly managed evaluations with multilateral organisations such as the current H4+1 Joint Programme Canada and Sweden (Sida) 2011-2016. The DED is actively engaged in the evaluation community as well. Not only in OECD DAC, but the DED is also a member of MOPAN’s Technical Working Group and Steering Committee and promotes an integrated methodology to assess the development effectiveness of multilateral organisations. Canada’s ongoing relationship with the Nordic+ Evaluation Group and 3ie (a network on impact evaluation) will continue.

Externally, the DED is aware of increased engagement with developing country evaluation functions, but evaluation capacity building is not part of the official responsibility of the DED.

Quality assurance DED provides the Programme Branches with the quality assurance in managing and implementing evaluations based on the OECD DAC Quality Standards for Development Evaluation. It has a team dedicated to provide support to those carrying out decentralised evaluations in reviewing Terms of Reference, work plans, evaluation reports, and management responses. The Evaluation Committee also functions as an external layer of quality control.

Notes to reader: The section at the beginning of Part II entitled “Introduction and key for the member profiles” provides explanatory notes on the profiles.

1. Joint Partnership of UN Agencies Working Together to Improve Women’s and Children’s Health.

100