Anecdotal insights from a quick comparative analysis of National Reform Programmes of the European Semester Introduction Every April, within the context of the European Semester, each Member State publishes its National Reform Programme (NRP), describing the structural reforms and measures undertaken or that will be undertaken in order to comply with the Country-Specific Recommendations (CSRs) received from the European Commission the year before. Until 2020, Member States also reported on the undertaken initiatives to meet the main objectives of the Europe 2020 Strategy, which is Green Deal and is -year strategy for smart, sustainable and inclusive growth, whose targets cover employment, climate change and energy sustainability and education, amongst others. The NRPs are a very important exercise for Member States to undertake on an annual basis: they incentivize Member States to set up policies and measures coherent with diverse EU objectives and aligned with the recommendations they receive on their macroeconomic and fiscal performances. However, NRPs remain a forward-looking exercise, as Member states are asked to report the foreseen impacts of their planned policies and measures. Member States are not required to undertake a backward-looking exercise of evaluating the exact performance of their policies, which would assess the causal relationship between the measures and the outcomes, nor to report on these results.1
Why is (ex-post) policy evaluation important? Monitoring and evaluating policies helps decision-making bodies to extract relevant information from past and ongoing activities that can be used as the basis for policy fine-tuning, reorientation and future policy planning2. Evaluating implemented measures provides information on the progress made towards certain targets, but more particularly establishes a causal link between implemented policies and certain results. Such analyses examine the relevance, effectiveness and impacts of regulatory decisions, as well as identify unintended outcomes, reasons for failure, and factors contributing to success3. They can also measure the cost-benefit of implementing particular policies, as policies and spending activities should be fit for purpose and deliver, at a minimum cost, the desired changes4. Without an in-depth evaluation of which policies and reforms conditioned and contributed to which outcomes, national governments and the Commission are unable to associate results with specific successful policies, which leads to the risk that decision-makers draw incorrect conclusions regarding the cause of improvements or deteriorations of policy domains. In the end, ex-post evaluations can serve multiple purposes. They can support decision-making processes by highlighting how policy implementation can be improved through a refinement or adjustment of policies and lead to general learning about which measures work and wh Policy evaluations can also provide accountability to the wider public, by providing transparent assessments of existing government policies.
European Commission (2015), Guidance on the content and format of the National Reform Programmes. GFA Consulting Group (2018), (Quick) Guide to Ex-post Policy Impact Assessment. 3 European Risk Forum (2009), Ex post evaluation assessing regulatory outcomes. 4 European Commission (2021), Better Regulation Guidelines. https://climandcomorg-my.sharepoint.com/personal/emma_climcom_org/Documents/Microsoft Teams Chat Files/2022.01.11_InsightsFromNRPs.docx 1 2
State of play of impact assessment / ex-post evaluation in the European Semester Climate & Company dived into some of the NRPs of Member States and the guidance given by the Commission in 2015 on how to write the NRP5, and made the following observations. In its guidance document, the Commission provides coordinator can use to facilitate their work. According to this template, the NRP should contain the elements depicted in Box 1. The only mention of an evaluation framework is as follows: [when providing a timetable on the progress achieved since the previous NRP], even if a measure has been fully implemented, relevant information on steps afterwards could be included (e.g. on evaluations of implementation). So, although -making processes and the adjustment of various measures, they do not appear as deemed necessary by the Commission in the context of the European Semester. So, it is not surprising that many Member States do not describe in their NRP concrete plans to evaluate their taken measures. What is asked of Member States, on the other hand, is to include the foreseen impacts of their planned measures. Nevertheless, after reviewing these expected impacts as reported by the countries, it becomes clear that Member States are not required to provide a justification/measurement of these impacts, so it is unclear to what extend the countries are basing their analysis on accurate causality methodologies. Furthermore, from our quick comparative analysis, it seems that most, but not all, Member States include in their annual NRP a brief description of the expected impacts of the planned and ongoing measures.
Box 1: Outline for the National Reform Programme 1. Introduction 2. Macroeconomic context and scenario a.
Macroeconomic outlook
b.
Macroeconomic impact of structural reforms
3. Key policy response to major economic challenges 4. Progress towards national Europe 2020 targets 5. EU funds 6. Institutional issues and stakeholder involvement
In their recent NRPs, many Member States stressed that ex-post evaluations will be caried out for some of the measures taken, and sometimes mentioned that evaluation was conducted, but they do not transparently reveal the methodology used and the results of the analyses. Furthermore, evaluations are apparently not foreseen for all types of measures and policy domains, or at least plans to conduct such analyses are not systematically stated by Member States in their NRP.
5
European Commission (2015), Guidance on the content and format of the National Reform Programmes.
https://climandcomorg-my.sharepoint.com/personal/emma_climcom_org/Documents/Microsoft Teams Chat Files/2022.01.11_InsightsFromNRPs.docx
Recommendations The European Commission should promote the practice of evaluating new policy measures at the national level and to require Member States to transparently report on the results of the evaluations in their NRPs. This way, the European Commission will be equipped with more accurate information on the performances of new measures undertaken by the national governments and will be able to provide more concrete country-specific recommendations in the European Semester cycle. The European Commission would also have the necessary information to promote best-practice examples of policies that have successfully worked in some of the Member States, and to disclose which policies have proven not to be effective and efficient. Although an ex-post evaluation is not required from Member States, it should in their own interest to conduct such analysis, as they should be aware of the successful and ineffective or even deteriorating policy choices they make. Disclosing these results creates more transparency for civil societies on the actions of national governments and their actual impacts. This gives the opportunity to Member States to provide justified reasoning for their political decisions.
Curious about our proposals to reform the European Semester? Find out more: At Climate & Company, we understand the unique potential of the European Semester for the climate and sustainability transition in the EU. With our project "Greening the European Semester", funded by the European Climate Foundation, we are exploring how the Semester can become a key tool for achieving the European Green Deal and the EU's sustainability goals. Stay up to date by visiting our project page and browsing our knowledge hub and its many valuable resources. For any questions or comments, get in touch with us:
Contact
Climate & Company
Oliver Herrmann - Oliver@climcom.org
Ahornallee 2,
Ingmar Juergens Ingmar@climcom.org
12632 Berlin, Germany
Laura Kaspar - Laura@climcom.org
www.climateandcompany.com
Louise Simon Louise@climcom.org
https://climandcomorg-my.sharepoint.com/personal/emma_climcom_org/Documents/Microsoft Teams Chat Files/2022.01.11_InsightsFromNRPs.docx