Evaluation of projects with climate adaptation co benefits in sub-Saharan Africa and Madagascar

Authors Olivier Beucher, Marie-Karin Godbout, Camille Palumbo (Baastel)

Evaluation of projects with climate adaptation cobenefits in sub-Saharan Africa and Madagascar (2007-2018)

Evaluation

DECEMBER 2023 No. 96

Overview

The complete evaluation report can be downloaded on the AFD website at : https://www.afd.fr/fr/ressources-accueil Other contributors Marion Douchin, Julie Fabre, Alain Lafontaine (Baastel) Ghislaine Tandonnet, Stéphane Simonet (Acterra) Ansoumana Bodian (Senegal case study) Alivony Ravelomanantsoa (Madagascar case study) Adboulaye Soumaila (Niger case study) Coordinated by Claire Cogoluenhes, Victor Fourcin, Léa Poulin (AFD) Translated from the French by Words in Action

Disclaimer The analyses and conclusions of this document are those of its authors. They do not necessarily reflect the official views of the Agence française de développement or its partner institutions.

Cover page Transplanting of onion plants, Podor Department (Senegal), middle valley of the Senegal River © Emmanuelle Andrianjafy, December 2017

Evaluation of projects with climate adaptation co-benefits in sub-Saharan Africa and Madagascar (2007-2018)

Contents Editorial

p. 4

A rapid rise in financial flows in support of adaptation p. 5 Evaluating adaptation: a major topic still being developed p. 7

Finding 4. More efforts required for sustainable results p. 19

Recommendations

p. 22

Acronyms and abbreviations

p. 28

Benchmarking other donor practices with a view to operationalising the recommendations p. 9

Finding 1. Climate risk still insufficiently taken into account in project design, despite progress over the period studied p. 10

Finding 2. Overall coherence of the tools and approaches promoted to adapt to the diversity of contexts p. 13

Finding 3. Already observed adaptation results need to be better characterised and measured p. 14

ExPost – 96 — 2023 – Page 3

Editorial

By Mathilde Bord-Laurans, Head of AFD’s Climate and Nature Division (CLN) Viewing climate and development as two interlinked emergencies, the French Development Agency (AFD) has set the climate change challenge as one of its priority areas of investment. Since 2018, this priority has taken the form of the “100% Paris Agreement” objective and integrated into the AFD Group Strategy. The Agency is thus the first development bank to have committed to aligning its financing with the 2015 Paris Agreement.

Since 2017, the Group’s climate commitments have reached a cumulative amount of €33.3 billion for 1,336 projects. AFD Group has thus achieved its “Climate and Development Strategy 2017–2022” objectives. Adaptation financing designed to address the current and future consequences of climate change totalled €11.3 billion over the period reviewed and is mainly channelled into Africa.

However, needs in the area of climate change adaptation are growing exponentially as droughts are worsening, flooding is increasing, and sea levels are rising. According to the latest reports from the Intergovernmental Panel on Climate Change (IPCC), over 300 million people in Africa have been impacted by natural disasters over the last 20 years. Of the 20 countries most vulnerable to climate change, 14 fall into the least developed countries (LDCs) category. As a result, it seems crucial to continue investing in climate change adaptation in Africa. The Summit on a New Global Financing Pact held in Paris in June 2023 reiterated the importance of France’s commitment to support the most economically and climatically vulnerable countries.

The growing needs for adaptation and the requirements for financial transparency at international level call our actions into question.

What lessons can be learnt from past investments to improve our impacts when it comes to climate change adaptation ? What conditions will enable projects to produce sustainable impacts ? What tools can we develop to raise and strengthen our ambitions for adaptation finance ? To answer these questions, a study was carried out on the relevance, coherence

ExPost – 96 — 2023 – Page 4

and impact of adaptation projects funded in sub-Saharan Africa (SSA) between 2005 and 2018. This analysis was supplemented by a benchmark of the practices followed by other donors in order to illustrate which measures still need to be improved.

The evaluation underscores AFD Group’s scale-up of adaptation financing and presents the results that are already visible on the ground.

Yet, AFD’s operational deployment in the area of adaptation still needs to be consolidated.

Although climate risk is better integrated into project design, this approach needs to be further strengthened. There is also room for improvement when it comes to targeting the populations most vulnerable to climate change and monitoring our operations and our impacts.

The findings of the evaluation have already helped to improve our operations in this area. For example, climate risk analysis is now systematically performed and laid out in an internal procedure. The evaluation’s strategic and operational recommendations call for mainstreaming adaptation more effectively into AFD’s activities. They will thus enable the Group to raise its ambition in the direction of high-quality climate change adaptation.

Evaluation of projects with climate adaptation co-benefits in sub-Saharan Africa and Madagascar (2007-2018)

A rapid rise in financial flows in support of adaptation The Agence française de development (AFD) took a stance on the climate change issue as early as 2005 when it prepared its first climate strategy in a Cross-cutting Intervention Framework (CIT). The Climate CIT 2012-2016 marked a key turning point for mainstreaming the fight against climate change into the Agency’s activities, by setting the target to ensure that at least half of its annual commitments (30% for PROPARCO, AFD’s private-sector financing arm) had climate co-benefits.

AFD’s third climate strategy (2017-2022) aims to reinforce the Agency’s positioning on climate change in response to three strategic challenges : 1. Ensure AFD Group’s activity is consistent with the 2015 Paris Agreement (COP21).

2. Maximise the impact on climate by identifying levers and knock-on effects.

3. Act as a platform for France’s international financial commitment to climate and development.

AFD thus wishes to position itself as the first bilateral development bank whose activities are required to be “100% compatible with the Paris Agreement” : each project financed by the Group must thus form part of the low-carbon and resilient development trajectory of the intervention country concerned. The 2017-2022 Climate CIT also maintains the target of dedicating 50% of financing to projects with climate co-benefits and extends it to the Group as a whole. In a context of rapidly growing financial commitments, this has resulted in an increase in the volumes of climate finance.

Another of the Agency’s flagship commitments is to redirect financing by setting up cooperation agreements with different types of actors (“a partnership-based AFD”) and particularly by mobilising the private sector. Finally, the

Agency sets itself the objective of co-building solutions with its partners and bringing influence to bear on standards by drawing on its network of agencies and capitalising on its experience in climate matters.

However, one of AFD’s remaining great challenges is to better define its future operational commitments in the area of climate change adaptation. This is the backdrop to the setting-up of the Adapt’Action Facility, a tool endowed with €30 million over four years (2017-2021), with an additional €15 million for the period 2022-2026.

For the last six years, the overarching objective of the Adapt’Action Facility has been to assist States in implementing their Nationally Determined Contributions (NDCs) through specific studies and support.

Projects with adaptation co-benefits are, in AFD’s understanding, projects that help reduce the vulnerability of property, populations or ecosystems to the effects of climate change.

The tight interlinking between development and adaptation objectives required establishing common principles for tracking adaptation financing, a process that AFD helped to develop through the IDFC[1] (Common Principles for Climate Change Adaptation Finance Tracking – MDBs[2]-IDFC) and implemented in late 2015.

According to these principles, a project (or one of its components) will be assigned to climate finance accounting if the three following conditions are met : a contextual analysis of the project’s implementation area shows vulnerabilities to climate change, the intent to address these climate vulnerabilities is stated in project documentation, the positive impact of the actions planned by the project on the issues of climate change vulnerabilities is soundly demonstrated.

• • •

The work of qualifying project adaptation co-benefits is carried out ex-ante by AFD’s Climate and Nature (CLN) Division.

The first projects with a potential for “adaptation” were identified in AFD’s project portfolio as of 2007.

[1] International Development Finance Club.

[2] Multilateral development banks.

ExPost – 96 — 2023 – Page 5

There has been a gradual increase in financing earmarked for projects with adaptation co-benefits as this accelerated from 2015, rising from €570 million to €2 billion in 2021. Over the period evaluated, AFD’s “adaptation” commitments mainly involved projects on water resource management, agriculture and biodiversity, as well as the prevention of extreme climate events.

The projects are supported by AFD’s technical divisions Agriculture, Rural Development and Biodiversity (ARB) and Water and Sanitation (EAA).

The other operations involve urban projects, bank credit lines or policy-based loans (PBLs) and are managed by various technical divisions. In 2018 (last year of the period reviewed), 55% of adaptation co-benefits commitments were directed to African countries.

ExPost – 96 — 2023 – Page 6

Evaluation of projects with climate adaptation co-benefits in sub-Saharan Africa and Madagascar (2007-2018)

Evaluating adaptation : a major topic still being developed The monitoring and evaluation of adaptation projects is a nascent topic for which no unified analysis framework or indicators exist, unlike mitigation where the indicator of the number of tons of CO2 avoided garners general consensus. Adaptation is a concept that practitioners define in diverse ways, especially when it comes to additionality[3] as compared to traditional development actions. The concept translates into different levels of ambition depending on the actors.

Furthermore, the long-term aspect of climate change and the uncertainties of its projections make the planning of adaptation interventions extremely difficult and highly context-dependent, particularly in the case of scaling long-term infrastructure. These interventions need to be continually adjusted in light of a baseline that is constantly changing. Evaluating the “success” of adaptation interventions, which only becomes visible retrospectively, is a major challenge that raises questions on how results should be attributed and thus requires specific methodological approaches. Various research studies and experimentation work highlight the specific adaptation-related issues and difficulties [4] facing decision-makers and funders. As a result, the methodological hurdles to adaptation interventions currently constitute an obstacle to conducting a high-quality ex-post evaluation.

It was in this context that the first evaluation of adaptation co-benefits projects implemented by AFD in the agricultural, water and sanitation sectors was commissioned in 2020. The goal was to take stock of the design, implementation and results of AFD’s interventions in the area of climate change adaptation.

This work has a twofold dimension as it is both retrospective and prospective.

On the one hand it aims to learn lessons from a sample of projects with adaptation co-benefits implemented between 2007 and 2018 (completed and not completed) in sub-Saharan Africa (SSA) and Madagascar. This work was carried out via an in-depth document review (sample of 35 projects out of a total 81 projects[5]), interviews with the different AFD divisions involved, and case studies in three of AFD’s intervention countries (9 projects in Madagascar, Senegal and Niger), focussing on the following questions : 1. To what extent are the risks linked to climate change and the vulnerability of the targeted groups or territories adequately mainstreamed into project design ? (relevance) 2. What is the internal and external coherence of AFD interventions in the area of adaptation to climate change ? (coherence) 3. To what extent have the implemented activities contributed to reducing the climate risk ?

What links are there between the development results obtained and the adaptation co-benefits achieved (effectiveness/impact) 4. How efficiently were projects with adaptation co-benefits executed ? (efficiency) 5. To what extent is the sustainability of the projects boosted ? (sustainability)

[3] The notion of additionality is core to the concept of adaptation and

aims

to

ensure

that

projects

programmed

under

adaptation – and financed within this objective – are defined and implemented in order to achieve a greater reduction of

climate risk than in traditional development projects ; the concept of additionality produces different interpretations in the field (Additionality in Adaptation Finance — Climate Policy Lab).

[4] Notably : Leiter T. , A. Olhoff, R. Al Azar, V. Barmby, D. Bours, V.W.C.

Clement, T.W. Dale, C. Davies, and H. Jacobs (2019). “Adaptation

metrics: current landscape and evolving practices.” Rotterdam and Washington, DC. Available online at www.gca.org).

[5] The sample was selected so as to capture a sectoral representativity (agriculture and water) and the different stages of project completion (completed/not completed), while also focussing on the countries with the highest numbers of adaptation projects (at least three projects).

ExPost – 96 — 2023 – Page 7

In addition, the findings, conclusions and recommendations of this evaluation should make it possible to mainstream the adaptation dimension throughout the entire project cycle at AFD, improve AFD’s interventions and provide input for the ongoing revision of the new intervention strategy regarding the fight against climate change – all of this in a context where adaptation is becoming a priority topic in SSA. The findings presented in this summary should be read in light of a fast-changing context, where adaptation was becoming an increasingly important topic over the 10 years covered by the evaluation. In fact, the selected sample comprises projects at different stages of maturity, some of which had been set up when the topic of adaptation was only just emerging.

Mainstreaming adaptation co-benefits into development projects, as well as evaluating adaptation interventions, are subjects that have only recently gained momentum and most donors are still in the process of developing the necessary methods. As a result, this evaluation is relatively innovative with respect to its purpose and scope.

ExPost – 96 — 2023 – Page 8

Evaluation of projects with climate adaptation co-benefits in sub-Saharan Africa and Madagascar (2007-2018)

Benchmarking other donor practices with a view to operationalising the recommendations Following the evaluation, the AFD teams decided to carry out a comparative analysis of the approaches and practices used to mainstream climate risk and adaptation into development projects, with the objective of identifying elements of good practice.

First, a rapid comparative analysis of the practices of ten organisations [6] jointly selected with AFD was carried out. A more in-depth document review and interviews with climate/ adaptation experts were then conducted in four organisations identified as having approaches or tools potentially relevant to AFD, notably : the World Bank (WB), the International Fund for Agricultural Development (IFAD), the United States Agency for International Development (USAID [7]) and the German development bank (KfW).

The main results of this benchmark are linked to the conclusions and recommendations formulated by this evaluation in the sections below.

It should be noted that the benchmark involves the practices in place in 2022 in the selected organisations and compares these to the AFD practices analysed by the evaluation over a period up to 2018. However, to obtain a comparison that better reflects the current situation, the approaches and tools more recently developed

by AFD were also taken into account.

[6] United Nations Development Programme (UNDP) ; Food and

Agriculture Organization of the United Nations (FAO) ; Green Climate Fund (GCF) ; Asian Development Bank (ADB) ; European

Investment Bank (EIB) ; World Bank (WB) ; International Fund for Agricultural Development (IFAD) ; Kreditanstalt für Wiederaufbau

(KfW) ; Swedish International Development Cooperation Agency

(SIDA) ; United States Agency for International Development (USAID).

[7] It was not possible to have an interview with USAID, but an indepth document review was carried out.

ExPost – 96 — 2023 – Page 9

Finding 1.

Climate risk still insufficiently taken into account in project design, despite progress over the period studied

In most of the 33 projects reviewed (70%), the risks linked to climate change (i.e., their potential effects on ecosystems and society) and/or to the vulnerability of the target groups or territories were identified during the design stage of projects included in AFD's adaptation co-benefits accounting. However, only a minority of these (18%) included a specific analysis to support a diagnostic of the climate risks during the projects’ design phase. The other projects integrate a relatively general and incomplete presentation of the current or future climatic hazards in the project’s area of intervention (i.e., the physical climate-related phenomena and trends or their physical impacts), or the vulnerabilities or exposure of the target populations.

Nearly one-third of the projects in the sample included in adaptation co-benefits accounting make no reference to climate risk in the project documentation.

The evaluation highlights that resources are generally too limited to undertake a robust ex-ante characterisation of climate risks and identify the optimal adaptation that the project can provide, despite the progress made over the period.

Box 1 – Components of climate risk according to the IPCC AR5 The reference used is based on the notion of climate risk proposed in the Fifth Assessment Report (AR5) of Working Group II (WGII) from the Intergovernmental Panel on Climate Change (IPCC). According to this reference, climate risk is the result of a hazard, exposure and vulnerability (sensitivity, capacity).

Analysis of hazards is the entry point for assessing climate-related risks : a hazard represents an external climate signal which does not depend on exposure or vulnerability and which cannot per se be influenced by adaptation or other measures seeking to deal with climate-related loss and damage.

Climate signal HAZARD physical impacts Sensitivity VULNERABILITY Capacity (Coping, Adaptive)

EXPOSURE

RISK

Ecosystems and societies

Source : GIZ and EURAC 2017, Risk Supplement to the Vulnerability Sourcebook. Guidance on how to apply the Vulnerability Sourcebook’s approach with the new IPCC AR5 concept of climate risk, p. 17, GIZ, Bonn.

ExPost – 96 — 2023 – Page 10

Evaluation of projects with climate adaptation co-benefits in sub-Saharan Africa and Madagascar (2007-2018)

However, the evaluation underlines that significant progress had been made over the period : for all of the projects funded after 2015, we found information enabling us to draw a partial picture of climate risk. Moreover, a far greater number of projects (40%) than before integrate formalised analyses or studies that help diagnose climate risk. As a result, accounting principles for climate co-benefits and climate risk screening tools (based on the IDFC common principles) emerged and contribute to a better understanding of climate risk in the intervention areas of projects financed post 2015.

The interventions include all activities considered to provide possible responses to climate risks, including development objectives that can significantly further the adaptation of the target populations (co-benefits logic).

However, the evaluation finds that (i) most of the interventions are not explicitly designed to tackle climate risks (i.e., with the intention to reduce the exposure or vulnerability to climate hazards clearly reflected in the intervention logics), even in cases where the risks had been quite well identified, and that (ii) the proportion of projects that explicitly respond to climate risks do not show a positive change over the period 2007-2018. Thinking on climate change adaptation is in fact still relatively limited or little documented for lack of adequate requirements at this level, and seldom results in the identification of adaptation-specific objectives at the time of project design : very often, the project documentation makes no reference to specific in-depth studies associated with the projects’ fields of intervention from the angle of climate change and proposes no analysis of possible alternative solutions to adaptation. The case studies showed that the design and implementation of the projects involving adaptation mainly rely on the experience of AFD or non-governmental organisations (NGOs) or agencies designated as contracting authorities, and that alternatives to the climate adaptation solutions proposed by the projects are not systematically considered and/or clearly documented and contextualised.

Yet, over half of the projects selected for case studies, i.e. five out of nine projects, had refined their risk diagnostic during project implementation (e.g., by including studies on how the target populations perceived the risk). Moreover, for most of the projects reviewed in-depth during the case studies, the diagnostic elements that were actually identified in the project documentation

are consistent with the onsite situation. In Senegal, the project to fight against flooding in Pikine Irrégulier Sud N°2 (PIS2) informed the structuring of the GCF-funded Integrated flood management project in Senegal (PGIIS) which aimed to map flood risk in a context of climate change, thus reinforcing the adaptation dimension of the PIS2 project.

Several factors identified during the evaluation, particularly by studying strategy documents and interviews with AFD’s Climate and Nature (CLN) division and operational divisions, can help to explain these shortcomings : 1. At the strategic level, a climate strategy geared to the institutional level is difficult to operationalise at project level, is highly focused on increasing adaptation financial commitments and promotes a quantitative approach to development projects where co-benefits are implicit (“What can be accounted ?”) to the detriment of an operational approach (How can project impacts on adaptation be maximised ?”) geared to improving the quality of the adaptation response implemented.

At the operational level, short appraisal times with multiple priorities do not afford project leaders enough time to work on adaptation aspects beyond what is strictly required. Although the appraisal documents and tools along with the related processes showed improvements over the period,[8] they remained insufficiently developed and integrated to facilitate a satisfactory analysis of climate risks (hazards, exposure, vulnerability ; see Box 1 above), to explain adaptation co-benefits, to maximise results and monitor impacts.

For the time being, the climate risk diagnostic is still not adequately defined and integrated into the appraisal phase of projects presenting adaptation co-benefits. Short appraisal times, limited resources and sometimes the absence of data (e.g., climate projections at the appropriate scales, vulnerability studies) are seen as major constraints to taking climate risk into account and conducting a diagnostic.

[8] The analysis by AFD’s Climate and Nature (CLN) division regarding

the percentage eligible for adaptation accounting, a screening

tool leading to a vulnerability sheet (Climate Screening tool with a more robust rating grid in the context of the specific “Sustainable Development

Analysis

and

Opinion”

production of country climate fact sheets).

ExPost – 96 — 2023 – Page 11

[SDAO]

mechanism,

This leads to a mismatch between AFD’s stated ambition for adaptation co-benefits and the resources actually implemented by the Agency to achieve these objectives : the growth of financing volumes eligible for “adaptation” accounting is not correlated to the development of an operational organisation that would help to ensure the quality of the response provided in terms of reducing climate risk. This creates an accountability deficit on one of AFD’s key commitments.

Operational organisation requires characterising the climate risk, identifying responses to this risk through an explicit intervention logic, then monitoring the implementation of these responses and their positive impact on the issues of climate vulnerability. To achieve this, resources and tools are needed, some being specific to adaptation (e.g., improved climate risk diagnostics) while others are cross-cutting, such as improved readability of the intervention logics and results monitoring (e.g., a logical framework with adaptation-related indicators and results-based management).

The benchmark study showed that in 2022 [9] the other donors interviewed, like AFD, had mainstreamed an approach to climate risk analysis into their project development cycle in order to bolster the resilience of projects[10] (climate-proofing). However, unlike AFD, most of the organisations studied (7 out of 10 organisations) also rely on climate-risk analysis procedures for the explicit purpose of optimising project results on climate change adaptation, i.e., bolstering resilience through projects.

In both cases referred to above, the climate-risk analysis procedures use similar methods and tools in terms of identifying current and future climate hazards in project intervention areas. On the other hand, these climaterisk analysis procedures to bolster resilience through the project take resilience a step further as they analyse the beneficiaries and damages they might suffer. This latter type of procedure is more recent and/or under development in all these organisations.

[9] The year in which the benchmark was conducted.

[10] For climate change adaptation in development projects, a

distinction is made between resilience through projects and the resilience of the projects themselves :

• Resilience through projects denotes the way in which projects

explicitly contribute to reducing the vulnerability of the beneficiaries, communities or countries.

• Resilience of projects refers to the capacity of projects to lastingly sustain their activities to tackle climate hazards.

ExPost – 96 — 2023 – Page 12

In theory, climate-risk analysis is systematically applied, as is the case at AFD, to all of the projects in most of the organisations. However, the benchmark shows that, in practice, some shortcomings persist with respect to systematically conducting these analyses (lack of time for the appraisal, partners’ various requirements).

Most of the organisations (7 out of 10) conduct the risk analysis in two main steps : a risk screening step in the very early stages of the project development cycle (i.e., at the project identification stage), followed by an in-depth analysis (when needed and according to criteria pre-defined by the project team) at the project appraisal or start-up phase.

Evaluation of projects with climate adaptation co-benefits in sub-Saharan Africa and Madagascar (2007-2018)

Finding 2.

Overall coherence of the tools and approaches promoted to adapt to the diversity of contexts The choices of financial instruments are consistent with the size, purpose and nature of the projects considered in this study, although these choices mainly depend on criteria linked to the intervention countries. A degree of convergence in fact appears between adaptation and grant funding, as those countries most vulnerable to climate change are generally the best endowed with grants. This is notably the case in Niger and Madagascar, where five of the evaluated projects are financed by grants. On the other hand, the sovereign-loan and grant combination used for a project in Madagascar (PIAA – Integrated Sanitation Program in Antananarivo) reflects AFD’s approach of accompanying physical investments with soft components related to capacity building and the production of the Master Plan and its supporting studies.

This approach, regularly used for infrastructure projects, seems relevant and consistent with needs. In Senegal, AFD operates primarily through sovereign loans, which generally seems justified as the types of projects are expected to deliver significant socioeconomic benefits. This is the case, for example, of development projects in the Senegal River Basin, which demonstrate the country’s determination to provide infrastructure with long-term socioeconomic benefits.

Project leaders can also request co-financing (GCF, EU – European Union) to obtain additional funding for their projects and thus complete their financing scheme, should they deem it appropriate. Such is the case of a project in Senegal (PGIIS) whose components are 100% grant-funded by the GCF. This is justified as the project not only involves study-research and capacity building activities to improve flood management planning in Senegal, but it is also difficult for a developing country facing multiple emergencies to finance research studies whose

short-term results are not necessarily apparent to the beneficiary communities, unlike on-theground infrastructure projects.

After analysis, it was difficult to confirm that the projects are consistent with the Climate Strategy as the strategy lacks precise and measurable objectives at project level beyond seeking to produce climate co-benefits. As a result, the “Coherence with the Climate Strategy” section, added to appraisal documents since 2016, for the most part presents generalities about the project, climate change mitigation and adaptation, rather than any real analysis of consistency with the strategy’s main components and objectives.

The benchmark study showed that only a minority of organisations (except USAID and IFAD) adapted their strategy/strategies to include an adaptation objective at operational level. In addition, only half of the organisations had chosen one or more adaptation result/impact indicators consistent with their strategy (beyond the financial volumes allocated).

The evaluation was unable to identify any very pronounced differences in the approach to climate risk analysis and identification of adaptation co-benefits between “traditional” AFD projects and projects of the AFD division responsible for partnering with civil society organisations (CSO).

Projects appraised by the French Facility for Global Environment (FFEM), whose Secretariat is entrusted to AFD, present a slightly different case as they focus on an environmental issue and are thus designed primarily around this theme. Even so, their climate risk analysis is not necessarily much more developed than in the AFD projects considered inasmuch as these projects do not generally propose an analytical diagnostic of climate risk in their documents.

In general, AFD projects are complementary to the interventions of other donors, as the Agency seeks to integrate its projects into a coherent national development momentum and, in the three case-study countries of this evaluation, it positions itself as a leading funding agency active in the dynamics of coordinating development aid. For example, synergies between AFD (the Lalankely and PIAA projects) and the World Bank (PRODUIR – Integrated Urban Development and Resilience Project for Greater Antananarivo) were identified in the Madagascar case study.

ExPost – 96 — 2023 – Page 13

Finding 3.

Already observed adaptation results need to be better characterised and measured Given the lack of explicit expected results for climate risk reduction in the logical frameworks of the projects studied, climate impact chains[11] relevant to the projects’ contexts (to illustrate how the components of the various projects provided, or not, a response to climate risks) had to be rebuilt during the evaluation process on the basis of the collected documentation and consultations conducted in the field. The case studies did indeed make it possible to gather elements – dispersed but present in the documentation – enabling climate risks to be described and impact chains to be reconstituted. The example of the PIS2/PGIIS project [12] in Senegal (Figure 1) shows, following the IPCC’s AR5 approach, how hazards combined with factors of exposure and vulnerability generate risks. Through this reconstitution, it can be seen on which exposure and vulnerability factors a project’s components will act. This helps to optimise these intervention levers, verify their relevance to climate risk, and potentially avoid situations of maladaptation.

However, these elements are still largely lacking in the project documents despite the fact that the formalisation of climate aspects in AFD appraisal documents has been continuously strengthened and that AFD’s expectations, commitments and modalities for adaptation made considerable headway over the period 2007-2018.

[11] According to the IPCC’s AR5 approach, an impact chain comprises risk components (hazard, vulnerability, exposure) and underlying factors.

[12] PIS2/PGIIS : Project to fight against flooding in Pikine Irrégulier

Sud n°2, associated with the Senegal Integrated Urban Flood Management project financed by GCF.

ExPost – 96 — 2023 – Page 14

The ex-post reconstitution of climate impact chains thus makes it possible to assess the adaptation co-benefits against the results achieved, whether or not these co-benefits were explicitly expressed in the intervention logic. Yet, this ex-post reinterpretation of projects through the prism of adaptation while the projects are ongoing or completed does not contribute to identifying better alternatives that could have been examined or to optimising adaptation results.

It is encouraging to note that the vast majority of the completed projects show a budget execution rate of over 90%. This suggests that most of the planned adaptation co-benefit activities were actually carried out for these projects. Nevertheless, the scant available data on the results achieved by the projects and the lack of explicit expected results on climate risk reduction do not allow for robust identification at portfolio scale of the extent to which the implemented actions have resulted in a reduction of climate risk.

The case studies did however make it possible to identify the achieved results (or in the pipeline) that were likely to contribute to reducing climate risks in the intervention areas for all of the nine projects selected. Relevant examples include reduced flood risk, improved water production and distribution capacity, and increased farm yields and income, thus providing households with greater resilience following the implementation of agroecological practices. For example, the GARIC project (Gender, Agriculture and Climate Risk Management Initiative) in Niger achieved results with significant co-benefits in the area of climate-risk reduction, as shown in Box 2.

Source : the authors.

ExPost – 96 — 2023 – Page 15

-

-

-

-

Sea level rise/Coastal erosion

Stagnant water that persists over many months/years

Increase in the frequency, intensity and duration of flooding

- Component 1 – Stormwater drainage infrastructure - Component 2 – Sewage treatment

PIS2

2.1 - Structural and non-structural measures for flood risk reduction 2.2 - Tools for adequate investment in flood management infrastructure

Accessibility limited in districts

Absence of real-time hazard monitoring

Weak capacity to anticipate the development of new residential areas

Lack of knowledge, characterisation and mapping of flood risk

Municipalities’ lack of resources

Lack of alternative residential areas for the populations

Weak urban planning capacity

Vulnerabilities - Capacity

- Component 3 – Resettlement Action Plan - Component 4 – Capacity building and Information, Education, Communication (IEC)

PIS 2

4.1 - Support to integrated flood risk management policymaking 4.2 - Institutional strengthening and capacity building

Component 4 – Governance

3.1 - Real-time hazard monitoring in Greater Dakar 3.2 - Protocols for infrastructure management under extreme rain events

Component 3: Risk prevention

PGIIS (GCF) - Component 1: Knowledge about the risk: 1.1 - Flood risk mapping 1.2 - Flood risk awareness

Abandonment of drinking water boreholes due to pollution

Unmanaged population growth and lack of land use planning

Absence of public amenities, notably stormwater drainage network, sewage network, household refuse collection

Constructions in flood-prone areas, without adequate urban planning

Soil sealing

Vulnerabilities – Sensitivity

PGIIS (GCF) - Component 2: Risk reduction

Rise in the level of the water table

Disturbance of the hydrographic network

Intermediate impacts

Increase in the frequency of significant/extreme rainfall events

Health problems (water-borne diseases affecting nearly 50% of the population in Pikine lrrégulier Sud) Overall unhealthy sanitary conditions Degradation of housing (forced abandonments), equipment and infrastructure Limited travel Reduced access to public services (schools, health services, transport, etc.)

Decline in the attractiveness of districts Decrease in economic activity

Risks

Region of “niayes”: depressions separated from the sea by a ridge of dunes where the water table rises to the surface

High proportion of disadvantaged populations

Densely populated areas

Exposure

Hazards/Risks

Evaluation of projects with climate adaptation co-benefits in sub-Saharan Africa and Madagascar (2007-2018)

Figure 1 – Climate impact chain of Project PIS2/PGIIS in Senegal according to the IPCC’s AR5 approach

Box 2 – Examples of achieved results likely to help mitigate climate-related risks for the GARIC project in Niger The GARIC project led by AFD’s ARB Division and implemented in Niger (2015-2018) Project objective : Increase the resilience of the rural producers who are the most vulnerable to climate risks in the Region of Maradi (Niger) Expected results : 1. Significantly and lastingly reduce the poverty level of 2,500 extremely poor and vulnerable rural households by strengthening women’s groups’ knowledge and skills on agro-sylvo-pastoral and fishing systems and practices that are not only adapted to the effects of climate change but also economically sustainable.

2. Strengthen gender equality and equity in the dynamics of support for the climate resilience of rural production systems for 2,500 households and 8 municipalities in the Region of Maradi.

Examples of the results obtained likely to contribute to reducing climate risks : − Reduction of food and nutritional vulnerability and gender inequalities

− 13.3% of women who had generated a farm income have recorded at least a 25% increase in their incomes

− Women’s Empowerment in Agriculture Index (WEAI [13]) rose from 0.767 in 2014 to 0.956 in April 2017

− 78.5% of women adopted adaptation-based agro-sylvo-pastoral practices (e.g., market gardening,

use of improved seeds for rain-fed and irrigated crops, production of plants to protect ecosystems,

use of services of para-veterinary auxiliaries, use of feed supplements for cattle (vitamins and mineral salts, etc.)

− Increased yields of millet, sorghum, cowpea and groundnut

− Increase in average yields of herds owned by women

− 337 beneficiaries of the GARIC project practice assisted natural regeneration in their family fields

− A 4.6% increase in producers who have adopted environment-friendly practices

− 90% of women’s groups are satisfied with their suppliers

− 77% of all members of these organisations were given knowledge on climate risk analysis

− 87.5% of women’s groups (i.e., 12 out of 16) integrated risk analysis into their activities

− 100% of municipalities took into account the community-based adaptation / disaster risk reduction (CBA/DRR) approach relying partly on traditional knowledge to reduce vulnerability

[13] Women's Empowerment in Agriculture Index.

ExPost – 96 — 2023 – Page 16

Evaluation of projects with climate adaptation co-benefits in sub-Saharan Africa and Madagascar (2007-2018)

Overall, however, the projects are not sufficiently rigorous in the area of monitoring-evaluation[14] and this is thus also the case for adaptation co-benefits. In the vast majority of projects, the monitoring-evaluation frameworks typically have no explicit expected results for climate risk reduction and no specific indicators to assess the reduction of climate risks or the vulnerability or exposure to climate hazards.

Yet, the capacity to credibly monitor and account for project results linked to adaptation co-benefits, ex post, is a key aspect in a context where AFD’s adaptation financing is rapidly increasing and where the Agency is highly visible on the international stage regarding this topic. The lack of documentation on risk diagnostics and of explicit adaptation objectives in the intervention logics of most projects with adaptation co-benefits makes assessing the relevance of the proposed performance indicators more complicated. As a result, their assessment can only be conducted through a granular case study.

This means that for most projects the extent to which the results on climate risk reduction have been achieved cannot be documented given that expected results have not been defined at this level and/or an adequate monitoring-evaluation framework is lacking. It is thus necessary to better identify adaptation results at the design stage, assign them specific indicators (which can be the same as some development results indicators), and monitor these indicators during project implementation to better qualify the results for the sake of continuous improvement and accountability reporting.

Furthermore, vulnerable populations are insufficiently identified and, most often, the monitoring indicators do not allow for assessment of the effects that the actions have on these populations. In addition, as work on identifying the components of climate risks (hazards, exposure, sensitivity/capacity) is insufficient, this means that the climate vulnerability specific to different categories of population is not analysed, which thus limits the capacity to target interventions on these populations.

Finally, as is often the case, measuring efficiency is not an easy matter absent a benchmark or specific economic analyses. The evaluation found that the activities with adaptation co-benefits do not incur any identifiable extra costs or delays. In fact, the projects with adaptation co-benefits examined in this study regularly involve delays, and sometimes cost overruns, but these are not a priori attributable to adaptation activities. Nonetheless, the efficiency of climate change adaptation should be measured through specific studies to compare, for example, the cost of adaptation relative to the cost of non-adaptation, as well as to alternative adaptation solutions, which points up the need for specific analyses during the project appraisal phase.

The benchmark study showed that the evaluations of climate change adaptation in other organisations (FAO – Food and Agriculture Organization of the United Nations, IFAD, UNDP), like those at AFD, find it difficult to characterise the results of adaptation interventions as the principles and tools of results-based management are insufficiently integrated.

Nonetheless, some of the organisations studied are more advanced regarding tools and guidelines for identifying the results of adaptation measures. Some donors (IFAD, USAID, GCF) have produced manuals on standardised climate indicators (including adaptation-related indicators) that can be aggregated at the level of an activities portfolio. Other organisations (WB, KfW, IFAD) require that at least one indicator on adaptation results be included for projects identified as contributing to adaptation.

Additionally, among the organisations reviewed in this benchmark, little evidence is found at this stage showing that specific tools or mechanisms are systematically applied to target climate-vulnerable populations during project set-up or to assess to what extent these populations will benefit from the projects.

[14] The absence or weakness of monitoring-evaluation frameworks

(definition of relevant monitoring indicators, use of the logical framework to manage and evaluate projects, and management

of all the documentation linked to monitoring-evaluation) is a general finding reported in the evaluation of AFD’s evaluation policy carried out in 2021 and thus not specific to adaptation projects.

ExPost – 96 — 2023 – Page 17

Some identified practices, however, could support the targeting of vulnerable populations. IFAD, for instance, has adopted an integrated approach that mainstreams climate adaptation in its interventions, as well as an approach to target priority beneficiaries by mainstreaming climate vulnerability as a selection criterion for priority project intervention areas. FAO and USAID have integrated questions into their climate-risk analysis tools, which helps identify the groups most vulnerable to climate change in a project intervention area. The process for climate risk analyses proposed by the WB comprises a step for identifying adaptation measures that can reduce risks specifically for the most vulnerable groups, including women, migrants and displaced populations. This step allows these groups’ specific vulnerability to be explicitly taken into account by identifying the adaptation measures likely to be of benefit to them.

At this stage, the evaluations reveal that these approaches have yielded mixed results overall and show that it is often difficult to isolate the impact of the interventions on the populations for want of a climate risk analysis that is applied to different population categories and dedicated monitoring-evaluation measures.

Interestingly, the GCF and IFAD carry out impact evaluations that allow for a detailed analysis of project impacts on vulnerable beneficiaries. Even if these impact evaluations are costly and can only be performed on a limited sample of projects, they nonetheless give better insights into what effects projects have on vulnerable beneficiaries in terms of improving their resilience.

ExPost – 96 — 2023 – Page 18

Evaluation of projects with climate adaptation co-benefits in sub-Saharan Africa and Madagascar (2007-2018)

Finding 4.

More efforts required for sustainable results The evaluation shows that progress still needs to be made on ensuring the sustainability of project results at the level of the portfolio studied.

The long-term horizon is not sufficiently taken into account, be it because climate projections informing long-term tendencies in the intervention areas are not considered or because an exit strategy has not been prepared right from the appraisal stage. In fact, in just under one-third of the projects studied, the project design documents reveal that long-term climate risks were taken into account and that the long-term sustainability of actions was anticipated. While recognising the potential adaptation function of many development activities (e.g., the idea that improving the populations’ living conditions increases their resilience to shocks, particularly climate shocks), it is also important to agree that these activities will not necessarily lead to adaptation co-benefits in all probable climate contexts and scenarios. Yet, without this work on long-term scenarios, it is impossible not only to ensure the relevance of operations over time and the sustainability of results, but also to avoid the risk of maladaptation. In fact, preparing exit strategies that describe how a project can ensure the continuity and sustainability of results after its completion is a component that can ensure greater sustainability of adaptation responses. In the projects studied, however, this element was for the most part lacking.

Several examples of actions aimed at ensuring the sustainability of results were identified in the case studies. The example of the GCF-funded PGIIS project in Senegal is interesting on this count : by grant-funding climate risk studies to establish an investment plan to address flooding, the project will help to provide the country with a targeted and optimised intervention strategy against this risk. Again, in Senegal, financing for a cluster of projects in the Senegal River Basin is an approach that supports the production of vulnerability studies (with Adapt’Action funding) for a group of projects and strengthens national institutions in the long term through a large capacity-building component. A key element for the sustainability

of results is that local actors take ownership of the interventions. This aspect could be strengthened, for example, by generalising participatory approaches given that the document review showed a positive correlation with the effective ownership of results.

The benchmark revealed that the long-term climate risks threatening the project intervention areas are currently taken into account in the risk analysis methodologies of the reviewed organisations. For all of these organisations, climate risk analysis covers current risks and future risks identified on the basis of climate projections (the recommended long-term time horizons range from 20 to 80 years).

However, the benchmark study did not allow for an in-depth examination of the way in which taking long-term projections into account would impact project design. What it did show overall was that most of the organisations studied called for long-term climate change horizons to be taken into account by promoting the integration of measures in favour of transformational change and/or by fostering “no-regrets”[15] measures that factor in the uncertainty inherent to climate projections.

Other organisations/donors have rating criteria (WB) or project selection tools (GCF, IFAD) similar to those at AFD : for example, AFD’s Sustainable Development Grid implemented since 2018[16] assigns a higher rating to projects that plan for transformational effects and/or factor in the uncertainty of climate projections (by ensuring a degree of flexibility in project components and activities).

The evaluation also noted that the potential influence on public policy of the portfolio of operations with adaptation co-benefits could be exploited still further. In fact, only a minority of the nine case-study projects had the objective of influencing public policy in the area of adaptation (at local and national level).

[15] A “no regret” measure is an adaptation measure that will have beneficial effect regardless of changes in the climate.

[16] The Sustainable Development Grid was not operational during the evaluation period.

ExPost – 96 — 2023 – Page 19

Box 3 – Advantages of programmatic financing in a territorial approach to climate change adaptation To achieve sustainability, and for the sake of effectiveness, increasing the duration of projects

or financing clusters of similar projects in a given geographical area can offer many advantages as much in terms of development as adaptation co-benefits. Longer-term programming helps to better ensure the sustainability of the results obtained, to test and adjust approaches but, more importantly, to enhance the

capacity of local partners who will then be better able to pursue the interventions. AFD’s efforts over several decades in the Senegal River Basin provide useful lessons on this count, as it has had a territorial approach which has evolved over time to integrate different challenges that now go beyond the aspects of irrigated agricultural production.

In terms of climate change adaptation, short projects are clearly ill-adapted unless they “solve” a

precise problem, for example, by building infrastructure. Adaptation is a process that needs to be planned

for the long term due to the very fact that climate trajectories are uncertain and require adjustments, and because adapting also means changing habits, adopting new practices, using new technologies, all

of which need time. Financing clusters of projects, or programmes grouping together several projects, also has the advantage of helping rationalise the costs of vulnerability studies specific to such programmes,

and above all specific to the territories and populations targeted by the interventions. This helps to focus these interventions more precisely on the adaptation solutions that are most effective and best able to

benefit the most vulnerable populations. This is the case in Senegal, where a vulnerability study was funded

by Adapt’Action in order to establish long-term climate-risk scenarios in the area concerned, thus helping to adjust the interventions accordingly.

In a context where it is difficult for AFD to include climate risk analyses in the project appraisal

process, this kind of approach would offer clear advantages. It is also an approach that could prove more adapted to large-scale climate co-financing initiatives like those with GCF for instance.

ExPost – 96 — 2023 – Page 20

Evaluation of projects with climate adaptation co-benefits in sub-Saharan Africa and Madagascar (2007-2018)

To continue implementing the first major commitment of its climate strategy, i.e., ensure its activities are 100% aligned with the 2015 Paris Agreement, AFD Group now relies on the Operationalization Framework developed

by CPI[17] and I4CE[18] for IDFC members, which highlights inter alia the importance of public policy dialogue with the country partners.[19] In fact, going beyond AFD’s climate strategies, what should receive special attention is the coherence of the Group’s projects with the strategies and planning documents of the intervention countries. This would help ensure that AFD’s actions are integrated into the climate-related frameworks established at national and international levels, notably by contributing to the implementation of nationally defined contributions (NDCs), thereby orienting them toward a better integration of adaptation.

The analysis of the climate strategies of the organisations covered in the benchmark shows that most of them aim to con­tribute, through their activities, to implementing the national strategies and international commitments of their partner countries when it comes to adaptation, particularly the adaptation component in the Nationally Determined Contributions (NDCs) and the National Adaptation Plans (NAPs). Like AFD, some of them, notably the WB and EIB, state in their climate strategy the determination to gradually bring all of their activities into line with the Paris Agreement, to help further low-carbon and resilient development trajectories. Several of the organisations reviewed (USAID, KfW, IFAD) have developed tools or adopted guidelines to encourage coherence between their projects and the climate strategies and planning documents of their intervention countries, but results have been mixed.

[17] Climate Policy Initiative.

[18] Institute for Climate Economics.

[19] https://www.afd.fr/en/ressources/climate-development-20172022-strategy

ExPost – 96 — 2023 – Page 21

Recommendations The evaluation process has helped shed light on what still needs to be done to ensure that AFD’s portfolios of projects with adaptation co-benefits in the agricultural and water and sanitation sectors respond more explicitly and directly to the challenges of climate change adaptation and bring more sustainable benefits to the most vulnerable populations. The evaluation identified three adaptation-specific challenges that are broken down into five operational recommendations.

These proposals should help not only to optimise potential co-benefits and avoid situations of maladaptation, but also to improve the metrics and reporting of the results achieved, as well as consolidate the Agency’s discourse on adaptation interventions and its ex-post accountability. Now that the challenges are better known and shared with partners, the implementation of these recommendations will make it possible to continue scaling up the financial flows dedicated to adaptation while also maximising and improving the adaptation response.

Each challenge and recommendation is linked to the good practices identified in the benchmark carried out after completion of the evaluation.

ExPost – 96 — 2023 – Page 22

Challenge 1. Improve the design of adaptation co-benefits projects to optimise the co-benefits and strengthen the evaluability of interventions Recommendation 1 : Establish a diagnostic of climate risks right from project appraisal The climate analysis chain in projects needs to be improved, starting from the climate risk diagnostic (including vulnerable populations) through to the definition of expected adaptation results. For this objective, a climate risk diagnostic must be put in place at the project appraisal stage based on the following three actions : 1.1 Determine the risk diagnostic components (hazards, vulnerability and exposure) and the specific questions needed to establish the diagnostic and develop a format for presenting the diagnostic in the project documentation (model format).

1.2 Better mobilise existing internal and external sectoral knowledge at country level to reduce the workload for each project.

1.3 Identify whether a complementary analysis is required to establish the diagnostic : when necessary, foresee a more robust diagnostic either during the feasibility study or at the beginning of the project.

This means developing a relatively simple model (toolbox) and making it easier to apply, but which nonetheless allows for characterisation of the climate risk in order to ensure some coherence between projects at this level. Today, AFD has different tools that could be adjusted or enhanced to facilitate this risk diagnostic for projects included under adaptation co-benefits accounting.

Evaluation of projects with climate adaptation co-benefits in sub-Saharan Africa and Madagascar (2007-2018)

RECOMMENDATION

GOOD PRACTICES IDENTIFIED IN THE BENCHMARK

Recommendation 1 : Establish a diagnostic of climate risks at the project appraisal stage • Determine the risk diagnostic components (hazards, vulnerability and exposure) and the specific questions needed to establish the diagnostic and set up a format for presenting the diagnostic in the project documentation (a model format).

•

documentation were found, notably :

− “climate annexes” in project documents (KfW, USAID).

− detailed instructions for integrating the analysis into project documents (KfW, WB, IFAD).

− USAID has defined indicators for monitoring the effective implementation of the climate risk analysis procedure.

• Better mobilise existing internal and external sectoral knowledge at country level to reduce the workload for each project.

Identify whether additional analysis is required to establish the diagnostic : when necessary, foresee a more robust diagnostic either during the feasibility study or at the beginning of the project.

Screening tools (IFAD, KfW, USAID) or complementary tools (WB) allow climate risks threatening targeted populations to be taken into account.

Among the benchmarked organisations, examples of relevant formats or guidelines for presenting the diagnostic in project

•

•

Most of the organisations (IFAD, KfW, WB and USAID) have developed guides or manuals to help identify existing sectoral knowledge to facilitate analysis of the risks and/or identification of the best adaptation options.

The WB’s Climate Risk Country Profiles and USAID’s Climate Change Risk Profile help identify the risks specific to priority adaptation sectors at national level for nearly 80 countries.

The WB, USAID and IFAD have developed tools to classify climate risks (low, moderate or high). This classification is used to identify the needs for in-depth analysis at the appraisal stage.

ExPost – 96 — 2023 – Page 23

Recommendation 2 : Clearly explain the intervention logic for adaptation and identify the relevant indicators The intervention logic for adaptation must also be clearly explained, and relevant indicators must be systematically identified so as to ensure that results are monitored for co-benefits. To achieve this, for each project : 2.1 Establish a simple impact chain that links the risk elements identified by the diagnostic with the project activities.

Also, the accounting performed by AFD’s Climate and Nature division (CLN) must be as detailed as possible.

2.2 Ensure that one or more indicators for monitoring the achievement of the expected adaptation co-benefits are included in the logical framework, including at least one indicator at the results level.

RECOMMENDATION

GOOD PRACTICES IDENTIFIED IN THE BENCHMARK

Recommendation 2 : Clearly explain the intervention logic for adaptation and identify the relevant indicators Establish a simple impact chain linking the risk elements identified by the diagnostic with the project activities. Also, the accounting performed by AFD’s Climate and Nature division (CLN) must be as detailed as possible.

•

reduce identified risks or vulnerability is applied (pilot phase). projects..

Ensure that one or more indicators • for monitoring the achievement of the expected adaptation co-benefits are included in the • logical framework, including at

ExPost – 96 — 2023 – Page 24

− a resilience rating system that qualifies projects designed to

− a guide to help identify adaptation co-benefits generated by

•

least one indicator at the results level.

The WB has developed two tools relevant to this aspect :

•

Some donors have developed manuals on standardised climate indicators that can be aggregated at the level of an activities portfolio (IFAD, USAID, GCF).

KfW and the WB have determined the conditions under which the inclusion of at least one adaptation indicator at project level is mandatory.

KfW has integrated into its climate mainstreaming tool a mandatory step related to determining performance indicators linked to the identified risks and the selected adaptation actions.

IFAD’s Results Management Framework includes a project portfolio-level indicator : “percentage of projects rated moderately satisfactory or better for adaptation to climate change”.

Evaluation of projects with climate adaptation co-benefits in sub-Saharan Africa and Madagascar (2007-2018)

Challenge 2. Promote the mainstreaming of projects with adaptation co-benefits in the public policy dialogue on climate Recommendation 3 : Ensure that projects with adaptation co-benefits contribute to the international and national strategies and commitments of AFD and its partner countries

RECOMMENDATION

The contribution of projects with adaptation co-benefits to the strategies and international and national commitments of AFD and its partner countries must be strengthened and demonstrated. To achieve this, AFD and its network of agencies should : 3.1 Ensure that the projects with adaptation co-benefits contribute to the Agency’s climate strategy, enabling these commitments to materialise as actions and concrete benefits in the area of adaptation.

3.2 Check that the projects are consistent with the climate strategies and planning documents of the intervention countries and that the projects’ contribution to these documents and commitments (particularly NDCs) is understood and monitored by the competent authorities (notably within the NDC monitoring framework).

GOOD PRACTICES IDENTIFIED IN THE BENCHMARK

Recommendation 3 : Ensure that the projects with adaptation co-benefits contribute to the international and national strategies of AFD and its partner countries − IFAD and USAID are the two reviewed organisations whose

Ensure that the projects with adaptation co-benefits contribute to the Agency’s climate strategy, enabling these commitments to materialise as actions and concrete benefits in the area of adaptation.

Check that the projects are consistent with the climate strategies and planning documents of the intervention countries, and that the projects’ contribution to these documents and commitments (particularly NDCs) is understood and monitored by the competent authorities (notably within the NDC monitoring framework).

specific objectives in terms of adaptation are the most

explicitly formulated in their climate strategy and whose results frameworks (including aggregatable adaptation-related

indicators) and the frameworks for monitoring and evaluating

these expected results are in place and regularly documented (Annual Report on Development Effectiveness (IFAD) the

Progress Report issued at least once every three years (USAID)).

•

•

• •

USAID’s contribution to defining or implementing the climate strategies and commitments of beneficiary countries is one of the targets laid out in its Climate Strategy 2022-2030. Aggregatable monitoring indicators specifically enable the measurement of this aspect.

The climate mainstreaming tool set up by KfW includes a section on analysing a project’s contribution to the national climate plan of the country concerned.

IFAD’s guidance for identifying adaptation options indicate that the NDCs must be considered as a source of adaptation options.

In its annual climate report, IFAD describes how projects have been designed in order to contribute to NDCs.

ExPost – 96 — 2023 – Page 25

Challenge 3. Ensure that the projects with adaptation co-benefits reduce the vulnerability to climate change in the long term for the most vulnerable populations Recommendation 4 : Better identify and target vulnerable populations AFD projects have a priori the objective of improving populations’ living conditions. Yet, vulnerability to climate change differs greatly among social groups, and some populations are much more vulnerable than others, faced with the same exposure to climate hazards. This is often the case for women and younger generations, for example. For this reason, any project with adaptation objectives needs to ensure that it effectively identifies and targets climate-vulnerable populations through the two main actions below : 4.1 Identify, at the time of project appraisal, objective criteria to define population groups considered as vulnerable, with a special focus on gender-related vulnerability. In parallel, determine overall criteria enabling an AFD-wide definition of vulnerable populations. These overall criteria would provide a first level of targeting that could be fine-tuned for each project.

4.2 Ensure that the intervention’s relevant components integrate these vulnerable populations, and disaggregate and/ or target relevant indicators that allow project results for these populations to be monitored.

RECOMMENDATION

GOOD PRACTICES IDENTIFIED IN THE BENCHMARK

Recommendation 4 : Better identify and target vulnerable populations • • •

FAO and USAID have integrated questions enabling identification of the most climate-vulnerable groups in their climate-risk screening tools.

The WB’s processes for climate-risk analyses include a step for identifying adaptation measures that can reduce risks for the most vulnerable groups (particularly women, migrants and displaced populations).

In its strategy to target priority beneficiaries, IFAD has recently incorporated climate vulnerability as a criterion for selecting priority intervention areas for its projects.

ExPost – 96 — 2023 – Page 26

Evaluation of projects with climate adaptation co-benefits in sub-Saharan Africa and Madagascar (2007-2018)

Recommendation 5 : Take better account of a long-term time horizon and anticipate exit strategies to foster the sustainability of results

5.2 In particular, for infrastructure projects, include in the climate diagnostic the long-term climate change projections requiring consideration during the infrastructure design phase, covering at least the estimated lifetime of the project’s outputs (linked to Recommendation 5.1).

5.3 Integrate thinking on the exit strategy right from project appraisal.

Adaptation is an evolving and gradual process that needs to be thought out over the long term, even more so than classic development actions. For this, the evaluation recommends that the long-term time horizon be better taken into account and that exit strategies be anticipated in order to foster the sustainability of results, through the following actions : 5.1 Envisage programmatic approaches with 10-15-year commitment phases (e.g., with three 5-year project cycles laid out from the start). This justifies conducting vulnerability studies at a territorial scale and using these to integrate this long-term horizon and better ensure the sustainability of results, while also strengthening the capacities of the relevant public actors to tackle adaptation challenges.

RECOMMENDATION

GOOD PRACTICES IDENTIFIED IN THE BENCHMARK

Recommendation 5 : Take better account of a long-term time horizon and anticipate exit strategies to foster the sustainability of results •

•

Some organisations encourage the integration of a long-term climate change horizon by fostering the integration of measures that promote transformational changes likely to produce long-term results beyond the direct beneficiaries of the projects.

Other approaches, promoting both the integration of a long-term climate change horizon and the uncertain character of climate projections, advocate the development of “no-regret” and/or flexible interventions and encourage adaptative management. − The decision tool developed by IFAD to help select the “best adaptation options” integrated a flexibility criterion.

− USAID’s climate-risk management tool states the need to take different climate change scenarios into account in the risk analysis and the uncertainty inherent to the analysis, to identify the adaptation options that are “no-regret’ or sufficiently flexible.

ExPost – 96 — 2023 – Page 27

Acronyms and abbreviations ADB Asian Development Bank AFD Agence française de développement (the Agency) ARB Agriculture, Rural Development and Biodiversity (AFD division) CBA Community-Based Adaptation CIT Cross-cutting Intervention Framework (AFD) CLN Climate and Nature (AFD division) CO2 Carbon dioxide

COP Conference of the Parties (United Nations) CPI Climate Policy Initiative CSO Civil society organisation DC Developing country DRR Disaster risk reduction EAA Water and Sanitation (AFD division) EIB European Investment Bank EU European Union EUR (€) Euro FAO Food and Agriculture Organization of the United Nations FFEM French Facility for Global Environment GARIC Gender, Agriculture and Climate Risk Management Project in Niger GCF Green Climate Fund GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit (German Development Cooperation) I4CE Institute for Climate Economics IDFC International Development Finance Club IFAD International Fund for Agricultural Development (United Nations) IPCC Intergovernmental Panel on Climate Change KfW Kreditanstalt für Wiederaufbau (German development agency) M Million MDB Multilateral Development Bank NAP National Adaptation Plan NDC Nationally Determined Contribution NGO Non-governmental organisation OSC Civil Society (AFD division) PGIIS Integrated Flood Management Project in Senegal PIAA Integrated Sanitation Program in Antananarivo (Madagascar)

ExPost – 96 — 2023 – Page 28

Evaluation of projects with climate adaptation co-benefits in sub-Saharan Africa and Madagascar (2007-2018)

PIS2 Project to fight against flooding in Pikine Irrégulier Sud N° 2 (Senegal) PPL Public policy loan PRODUIR Integrated Urban Development and Resilience Project for Greater Antananarivo (Madagascar) SDAO Sustainable development analysis and opinion (AFD mechanism) SIDA Swedish International Development Cooperation Agency SSA Sub-Saharan Africa UNDP United Nations Development Programme USAID United States Agency for International Development WB World Bank WEAI Women's Empowerment in Agriculture Index

ExPost – 96 — 2023 – Page 29

Agence française de développement 5, rue Roland Barthes 75012 Paris l France www.afd.fr Innovation, Research, and Knowledge Directorate.

Evaluation and Knowledge Capitalization (EVA) Department

Agence Française de Développement (AFD) Group implements France's policy on development and international solidarity. Comprising AFD, which finances the public sector and NGOs; Proparco, which finances the private sector; and soon, Expertise France for technical cooperation, the Group finances, supports and accelerates transitions towards a more resilient and sustainable world.

We are building - with our partners - shared solutions, with and for the people of the Global South. Our teams are active in more than 4,000 projects in the field, in the French overseas departments and some 115 countries, including areas in crisis.

We strive to protect the common good - promoting peace, biodiversity and a stable climate, as well as gender equality, health and education. It's our way of contributing to the commitment that France and the French people have made to fulfill the Sustainable Development Goals. Towards a world in common.

Publication Director Rémy Rioux Editor-in-Chief Jean-Claude Pires Graphic Design MeMo, Juliegilles, D. Cazeils Design and Production Comme un Arbre !

Legal Deposit 4th quarter 2023 ISSN 2680-3844 Printed by AFD's reprography service To browse our other publications: www.afd.fr/fr/collection/evaluations-ex-post

Credits and authorisations License Creative Commons Attribution – Non Commercial –No Derivatives https://creativecommons.org/licenses/ by-nc-nd/4.0/