systems (iSA-Gov

138 – 4 – SUPREME AUDIT INSTITUTIONS’ INPUT INTO POLICY EVALUATION AND OVERSIGHT

The SAI community is addressing the important question of “who controls the controllers” and is promoting and providing guidance for peer reviews and a performance management framework for self-evaluation. To date, many SAIs have been subject to a peer review by international organisations (including the OECD) as well as their peer SAIs. SAIs’ quality, objectivity and relevance of work is critical to them having impact and providing a complementary, rather than duplicative and inefficient, role in evaluating for accountability and results.

Case studies of SAI activities supporting policy evaluation and oversight

Evaluating for results

Box 4.1. The SAI of Canada – assessing programme evaluation in the Federal Government

Objective

The government of Canada has had several policies on the evaluation of programme effectiveness since the 1970s. The Office of the Auditor General of Canada has conducted audits of the evaluation function several times, most recently in 2009 with a follow-up in 2013. The objective of these audits was to determine whether the relevant central agency and affected departments were meeting needs for information on programme effectiveness, and whether they were identifying and making needed improvements to the function.

Type

Performance audit.

Scope and methodology

The audits covered the central agency responsible for the government's policy on programme evaluation (the Treasury Board of Canada Secretariat) and selected departments subject to the policy. These departments were selected to reflect a cross-section of programme types.

The initial planning phase of the first evaluation audit included broad consultations with experts in the field of programme evaluation and public administration. The decision on audit timing reflected the introduction of a revised policy on evaluation in 2009. The audit methodology included reviewing samples of completed evaluation reports, and assessing departmental compliance with the policy.

Criteria

Country laws (The Financial Administration Act requires evaluation of certain types of programmes) and key national indicators (The Treasury Board Policy on Evaluation and its associated directive and standard).

Resources

The resources committed to the initial 2009 audit were 9 100 staff-hours at a cost of CAD 1.4 million (Canadian dollars). The resources committed to the 2013 follow-up were 6 100 staffhours at a cost of CAD 1.1 million. The team consisted of approximately six members, one of whom was an experienced programme evaluator. Consultants were rarely used.

SUPREME AUDIT INSTITUTIONS AND GOOD GOVERNANCE: OVERSIGHT, INSIGHT AND FORESIGHT © OECD 2016