2.2. Innovative and joint approaches to policy-making: Peru’s “Edu-Lab”

2 – SUPREME AUDIT INSTITUTIONS’ INPUT INTO POLICY FORMULATION – 43

order to improve implementation capacity and, in some countries, monitors spending, delivery and outcomes, and steers whole-of-government evaluations (OECD, 2014a).

Despite playing an increasingly decisive role in policy-making, CoG institutions often suffer from limited capacity. Resource justification for strategic and steering activities can be difficult, particularly in a context of financial crisis. In times of fiscal prosperity, however, there may not be a focus on foresight and contingency planning. On average, OECD member countries’ CoG expenditure is an average of only 0.045% of government expenditure, and accounts for less than 0.1% of central government employment (OECD, 2014a). A lack of capacity limits the CoGs ability to systematically identify and assess risk, which highlights the need for a coherent and measured system of controls and regulations. Central steering may also be inhibited by the fact that CoG staffing, particularly at the top, is often linked to the electoral cycle. This emphasises that a longterm vision based on foresight is important for times of prosperity as well as times of constraint.

SAIs have recognised the increasing pressure that has been either assigned to or accepted by CoG institutions to ensure coherence across government. While CoG institutions fall under the audit scope of SAIs, some SAIs are moving beyond the basic requirement of auditing for financial regularity and compliance and are looking at the capacity and efficacy of CoG institutions. In doing so, SAIs are filling a critical need within resource-constrained CoGs by providing information and insights about strategic planning.

6/10

Of peer SAIs have looked at: The capacity and/or efficacy of central government units, including: ensuring the long-term vision is harmonised with other key documents (fiscal plans), and ensuring the long-term vision is reflected in ministries work plans.

7/10

Of peer SAIs have looked at: The division of responsibilities between key actors involved in strategic planning

Examples of SAI work in this area include:  Brazil’s Tribunal de Contas da União (TCU) has looked at the executive’s multiyear planning system, the PPA, from various angles. An evaluation of the department of planning and strategic investments, and other related entities, was undertaken to analyse the adequacy of programmes and indicators listed in the

PPA from 2008-2011 against objectives of government policy (TCU, 2009a).

Later, a performance audit was used to describe and analyse how the PPA was structured for the period 2012 to 2015. This audit checked the extent to which new concepts introduced would incorporate innovation and seek to correct deficiencies identified in previous plans.  The UK’s NAO synthesised years of work on centre of government institutions to create a centre of government report (NAO, 2014a). Recognising its critical role, the NAO examined the centre of government (HM Treasury and Cabinet Office) and changes in recent years, touching on the effectiveness of the CoG’s strategic planning, leadership and co-ordination functions.

SUPREME AUDIT INSTITUTIONS AND GOOD GOVERNANCE: OVERSIGHT, INSIGHT AND FORESIGHT © OECD 2016

44 – 2 – SUPREME AUDIT INSTITUTIONS’ INPUT INTO POLICY FORMULATION

In assessing the maturity of government processes related to steering and those responsible for steering, SAIs, and other evaluation institutions, may refer to the TCU’s Framework to Assess Public Policies (Box 3.1), the OECD’s Centre Stage Report (OECD, 2014a) and the IADB’s (2014) Framework for the Centre of Government.

The establishment of measurable indicators (Table 2.1, key element E)

Some governments have been integrating measurement criteria into planning documents so that programmes and policies can be evaluated against their original objectives in a way that provides a true picture of their strengths and weaknesses. However, national indictors in most countries appear to focus on establishing macroeconomic national indicators, which do not necessarily illustrate public sector progress towards inclusive growth (INTOSAI, 2010). Establishing relevant and targeted criteria during the planning phase that can be evaluated at a later phase, and fed back into future iterations of planning remains a difficult task. There is little evidence that evaluation currently feeds into policy-making processes and decisions, as countries struggle to streamline quality and clear performance information in a systematic way. Guidance exists on how to develop criteria, for example, the OECD’s seven steps for indicator development (OECD, 2014c).

6/10

Of peer SAIs looked at: The existence or development of government-wide evaluation criteria

Examples of SAI work in this area include:  In accordance with its mandate to assess the US Government Performance

Management Modernization Act (GRPAMA), the GAO employs innovative tools to foster linkages between ministries’ planning and performance information and

Congress, who use this information for decision-making. The GAO’s

Congressional Briefing on opportunities to address government performance issue, the Guide on Using GPRAMA to help inform Congressional decisionmaking, and the index on Use of Performance Information at the Agency Level, are discussed in more detail in Box 2.6. These activities are particularly notable as they have been largely at the request of Congress, who has recognised the value of GAO’s cross-cutting insight and foresight on these critical topics.

A reliable evidence base (Table 2.1, key element F)

Evidence-based decision making is a key tool that governments and public administrations use to gain strategic insight through examining and measuring the likely benefits, costs and effects of their decisions. Performing evaluation at a government-wide level helps to ensure coherence across government and provides a good understanding of whether government-wide objectives are being met. It clarifies the reality of the costs associated with government objectives and gives governments the tools to help prioritise competing objectives. There are numerous channels available to a head of government for policy related decision making, however the cabinet remains the principal vehicle.

SUPREME AUDIT INSTITUTIONS AND GOOD GOVERNANCE: OVERSIGHT, INSIGHT AND FORESIGHT © OECD 2016

2 – SUPREME AUDIT INSTITUTIONS’ INPUT INTO POLICY FORMULATION – 45

As part of high-level government meetings, one important objective of the centre of government’s management role is to ensure that harmonised consultation processes have been followed and that appropriate analytical tools have been used. These twin activities help to ensure that basic issues regarding the usefulness and cost-benefit analysis of an action can be judged easily by the head of government and cabinet. This analysis and consultation process usually extends beyond the sectoral perspective of the sponsoring agency or department and, as such, the analytical tools and consultation processes must also be broader. The resulting presentation of information to government should enable political leaders to understand the need for, and consequences of, proposed policy interventions. This includes understanding their costs and benefits based, in part, on weighing up evidence on the likely impacts of such interventions, and anticipating risks to the objectives and resistance to the policy. While individual departments might focus on their own evidence-gathering, sometimes through large internal research units, the CoG is obliged to canvass opinion and analysis more widely (OECD, 2014a).

Ensuring public sector success in the policy formulation stage requires having a thorough understanding of the nature of the problem that a policy or programme is designed to tackle. However, understanding how a policy or related programme will work in practice continues to be a challenge. Governments are employing new strategies in policy formulation to allow more space for innovation and piloting. Collaborations and sources of inspiration may come from outside government, particularly from the fields of science and academia where, for example, experiments aimed to fight poverty have incurred great success (Box 2.2).

Box 2.2. Innovative and joint approaches to policy-making: Peru’s “Edu-Lab”

With a view to designing innovative evidence-based policies for education in Peru, the strategic planning secretariat of the Peruvian Ministry of Education teamed up with the Adbul Latif Jameel Poverty Action Lab (J-PAL) and the Innovations for Poverty Action (IPA), to create an innovation lab for education – the “EduLab”.

The EduLab was created to identify problems and design potential innovations to implement low-cost interventions with big impact, merging the expertise of researchers with the political and operational experience of policy makers.

The strategic planning secretariat has worked collaboratively with J-PAL and IPA, as well as with Ministry units required to implement interventions, throughout the EduLab policy cycle of: designing cost effective interventions, implementing cost-effective interventions, impact evaluating using administrative data, scale-up of effective interventions, and identification of problems to be solved.

One intervention proposed seeks to improve the management of school maintenance resources (cash transfers) that are provided to schools to guarantee a minimum standard of safety, hygiene and comfort. Following lab research findings that carefully constructed SMS messages is an effective way to supervise and motivate school personnel, the EduLab is elaborating on the design, implementation and evaluation stages of an intervention that would rollout the use of SMS messages. The impact of the intervention can be monitored through the ministry’s administrative data on the execution of school maintenance budgets.

Source: JPAL (2015), Bringing Innovation to Education Policy in Peru, Adbul Latif Jameel Poverty Action Lab (-JPAL) and the Innovations for Poverty Action (IPA), www.povertyactionlab.org/doc/edulabpublication, (accessed 1 October, 2015).

SUPREME AUDIT INSTITUTIONS AND GOOD GOVERNANCE: OVERSIGHT, INSIGHT AND FORESIGHT © OECD 2016