EPTISA ROM MANUAL by roberto canessa

ROM MANUAL

01/01/2012

Result Oriented Monitoring System of the Implementation of Projects and Programmes of external aid financed by the European Union in Sub-Saharan Africa, Caribbean, Pacific, Indian Ocean and Overseas Countries and Territories Lot 2 ACPEVA/2010/256-098

guidelines for mission planning and execution, report writing and quality control

This manual has been prepared by the external monitoring team for EC projects and programmes in the ACP and does not necessarily reflect the opinions of the European Union

NAVIGATE THROUGH THE MANUAL USE THE BUTTONS BELO W TO NAVIGATE THROUG H THE DIFFERENT PART S OF THIS MANUAL

Instructions

ROM BACKGROUND

MISSION CYCLE

ROM CONCEPTS

COLLECTION OF DATA

Products

PROJECT SYNOPSIS (PS)

BACKGROUND CONCLUSION SHEET (BCS) (On going projects) MONITORING REPORT

DE-BRIEFING FICHES

INDEPENDENCE AND AVOIDANCE OF CONFLICTS OF INTEREST

LIST OF CRS PURPOSE CODES

NAVIGATOR

others

GUIDELINES FOR ROM MISSION LEADERS

Use the buttons below to navigate through the different parts of this manual

FOREWORD

The present ROM Manual is an internal document developed by Eptisa ROM team for the use of its team of monitors and quality control. Its aim is to complete the instructions already available in EC issued documents (ROM manual, BCS rider, ex post and SPSP ROM guidelines1) and in no way does it substitute those documents. It is addressed to all monitors and quality controllers as a collective instrument, to be improved on the process. All remarks and suggestions are thus welcomed to improve these guidelines. We expect all ACP monitors to follow the instructions included in this Manual so as to increase the consistency and objectiveness of our work.

Brussels, January 2012

Frans Geilfus

Team Leader

In particular, for ROM SPSP refers exclusively to the specific Guidelines

FOREWARD

Monitoring System for the Implementation of Development Projects financed by the EC in SubSaharan Africa, Caribbean, Pacific, Indian Ocean and Overseas Countries and Territories.

Abbreviations & Acronyms

ABBREVIATIONS & ACRONYMS ACP BCS CA CARICOM CO CRIS CRS CSP D+3 DAC (DCD) DEVCO EC EDF EMN/EMR EU EU MS EUDEL FA GC HIV/AIDS ICT LF LFM M&E ML MR MT NAO NIP ODA OO OVI (IOV) PCM PFM PIU PP PS QC QSG ROM SC SMART SPSP TA TAPS TC TM ToR UN

Africa, Caribbean and Pacific Background Conclusion Sheet Contribution Agreement Caribbean Community Coordination Office (Brussels) EC Common Relex Information System Creditor Reporting System (CRS) aid activity database (OECD) Country Strategy Paper Decision date + 3 years Development Co-operation Directorate (OECD) Development Cooperation (EC Directorate) European Commission European Development Fund End Mission Note/Report European Union European Union Member States European Union Delegation Financing Agreement Grant Contract Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome Information and Communication Technology Logical Framework Logical Framework Matrix Monitoring and Evaluation Mission Leader Monitoring Report Mid Term (review) National Authorizing Officer National Indicative Program Official Development Assistance Overall Objective Objectively Verifiable Indicator Project Cycle Management Public Finance Management Project Implementation Unit Project Purpose Project Synopsis Quality Control Quality Support Group Result Oriented Monitoring Service Contract Specific, Measurable, Achievable, Relevant, Time-bound Sector Policy Support Program Technical Assistance Technical and Administrative Provisions (for Implementation) Technical Cooperation Task Manager Terms of Reference United Nations

TABLE OF CONTENTS

TABLE OF CONTENTS NAVIGATE THROUGH THE MANUAL ................................................................................ 3 FOREWORD............................................................................................................................... 5 TABLE OF CONTENTS ............................................................................................................ 7 ROM BACKGROUND ............................................................................................................ 11 ROM typology ...................................................................................................................................... 11 ROM service organization .................................................................................................................... 11 The ROM experts ................................................................................................................................. 14 ROM expert’s general obligations ........................................................................................................ 14

MISSION CYCLE ..................................................................................................................... 17 Mission Planning.................................................................................................................................. 19 Background documents analysis .......................................................................................................... 21 Mission start ........................................................................................................................................ 22 Pre briefing meeting ............................................................................................................................ 23 Briefing ................................................................................................................................................ 24 Monitoring visits .................................................................................................................................. 25 Debriefing ............................................................................................................................................ 26 Post mission accomplishments ............................................................................................................ 27 Writing of Reports, Quality Control and encoding in CRIS .................................................................... 28 The use of the templates ..................................................................................................................... 29

ROM CONCEPTS .................................................................................................................... 31 HOW to analyse the LF ......................................................................................................................... 31 WHICH are the most common lf deficiencies ....................................................................................... 31 Intervention logic indispensable elements........................................................................................... 31 Basic rom CRITERIA .............................................................................................................................. 32 Assessing the “quality of design” ......................................................................................................... 33 Suggesting “LF” improvements ............................................................................................................ 33

MONITORING STEP BY STEP ............................................................................................ 35 COLLECTION OF DATA ....................................................................................................... 36 Basic documents checklist ................................................................................................................... 36 Policy and country context documents .................................................................................................. 36 Project documents ................................................................................................................................. 36 Additional useful documents ............................................................................................................... 37 Policy and country context documents .................................................................................................. 37 Project documents ................................................................................................................................. 37 Who is responsible for collecting documents ....................................................................................... 38

TABLE OF CONTENTS

PROJECT SYNOPSIS (PS) .................................................................................................... 39 What is the PrOject Synopsis ............................................................................................................... 39 When to prepare a Project Synopsis .................................................................................................... 39 How to prepare the Project Synopsis ................................................................................................... 39 Administrative data ................................................................................................................................ 39 Text of the PS.......................................................................................................................................... 40 PROJECT SYNOPSIS (template) ............................................................................................................. 41

BACKGROUND CONCLUSION SHEET (BCS) .................................................................. 45 What is the BCS ................................................................................................................................... 45 Navigating tHrough the BCS instructions ............................................................................................. 45 How to fill the BCS ............................................................................................................................... 46 When to start filling the BCS ................................................................................................................ 47 The scoring system .............................................................................................................................. 47 1.Relevance and quality of project design ............................................................................................ 48 Sub criterion 1.1 - What is the present level of relevance of the project? .......................................... 48 Sub criterion 1.2 - As presently designed, is the intervention logic holding true? .............................. 49 Sub criterion 1.3 Is the current design sufficiently supported by all stakeholders?............................ 51 Sub criterion 1.4 - Is the current design sufficiently taking cross-cutting issues into account? ......... 52 Key lessons learned and/or action(s) recommended, by whom in order of priority .......................... 53 _Toc313549890 2.Efficiency .......................................................................................................................................... 56 Sub criterion 2.1 - How well is the availability/usage of means/inputs managed? ........................... 56 Sub criterion 2.2 - How well is the implementation of activities managed? ....................................... 57 Sub criterion 2.3 - How well are outputs achieved? ............................................................................. 58 Sub criterion 2.4 - how well is the Partner Contribution / Involvement working? ............................ 59 Key lessons learned and/or action(s) recommended, by whom in order of priority .......................... 60 3.Effectiveness ..................................................................................................................................... 64 Sub criterion 3.1 - How well is the project achieving its planned results? .......................................... 64 Subcriterion 3.2 - As presently implemented what is the likelihood of the PP to be achieved? ....... 65 Key lessons learned and/or action(s) recommended, by whom in order of priority. ............................ 67 4.Impact prospects .............................................................................................................................. 70 Sub criterion 4.1 - What are the direct impact prospects of the project at OO level? ........................ 70 Sub criterion 4.2. - To what extent will the project have any indirect positive/negative impacts? ... 71 Key lessons learned and/or action(s) recommended, by whom in order of priority .......................... 71 5.Potential sustainability ..................................................................................................................... 74 Sub criterion 5.1 - Financial / economic viability .................................................................................. 74 Sub criterion 5.2 - What is the level of ownership of the project by target group .............................. 75 Sub criterion 5.3. - What is the level of policy support provided? ....................................................... 76 Sub criterion 5.4 - How well is the project contributing to institutional/management capacity? ..... 77 Key lessons learned and/or action(s) recommended, by whom in order of priority. ......................... 78

TABLE OF CONTENTS

6.Horizontal issues ............................................................................................................................... 82 Review of Technical Cooperation (TC) Quality Criteria ........................................................................ 83 EC Visibility ............................................................................................................................................. 88 7.Cross cutting issues ........................................................................................................................... 92 7.1) Have practical and strategic gender interests been adequately considered in the strategy? ........ 92 7.2) Is the project respecting environmental needs? ............................................................................. 93 7.3) Has (good) governance been mainstreamed in the project? .......................................................... 94 7.4) Does the project actively contribute to the promotion of Human Rights? 94 8.List of persons interviewed /documents analysed ............................................................................ 96

DE-BRIEFING FICHES ........................................................................................................101 Why de-briefing fiches ....................................................................................................................... 101 How to do it ....................................................................................................................................... 101 When to do it ..................................................................................................................................... 101

MONITORING REPORT (MR) ..........................................................................................103 What is the Monitoring Report .......................................................................................................... 103 When to prepare the Monitoring Report ........................................................................................... 103 The different sections of the Monitoring Report ................................................................................ 103 Navigating tHrough the mr instructions ............................................................................................. 103 Intervention/financial data ................................................................................................................ 105 The grades in the MR ......................................................................................................................... 106 Summary of conclusions .................................................................................................................... 106 Relevance and quality of design .......................................................................................................... 107 Efficiency of implementation to date.................................................................................................. 108 Effectiveness to date ........................................................................................................................... 109 Impact prospects ................................................................................................................................. 110 Potential sustainability ........................................................................................................................ 111 Key observations and recommendations (by whom and in order of priority) .................................. 112

GUIDELINES FOR ROM MISSION LEADERS ................................................................120 1 Mission planning: ............................................................................................................................ 120 2- Mission field phase ........................................................................................................................ 121 3- Post-mission phase ........................................................................................................................ 123 4 - End mission report (EMR) ............................................................................................................. 123

INDEPENDENCE AND AVOIDANCE OF CONFLICTS OF INTEREST ......................124 LIST OF CRS PURPOSE CODES........................................................................................126

TABLE OF CONTENTS

ROM BACKGROUND

quick review of ended projects, with an ex post

ROM BACKGROUND

perspective. Result Oriented Monitoring (ROM) is the system

used

the

European

Union

Development Cooperation to follow up on a regular basis all its

external operations

worldwide. The service is outsourced to external contractors on a regional basis. The Eptisa Consortium (Eptisa, EPESMandala, ECO3, ISS) is running the ACP lot for the period 2011/2013. ROM principles and methodology are established in the ROM Handbook which is

Eligibility criteria for each typology are shown in the following table. ROM of Regional projects is regulated by a specific

Guideline .

programmes

with

Complex activities

regional different

countries may be monitored separately in more than one country, with an additional report to consolidate them (the consolidation report”. 3

the official guideline for all ROM activities and

For ROM of SPSPs a specific Guideline also

operators. ROM is based on the same criteria as

exists, although still being tested.

all the other M&E processes applied to the EU external cooperation and uses the Project Cycle Management

(PCM)

Criteria

and

ROM SERVICE ORGANIZATION

Logical

Framework (LF) parameters. ROM serves not

The ROM service for the ACP region is managed

only

project

through a service contract on behalf of the

management by informing stakeholders about

European Commission by a coordination office

the performance of a specific project, but it

(CO) in Brussels (see organization chart in the

also contributes to general EuropeAid policy

following page). The CO is responsible for

articulation, implementation and review. With

organizing all ROM activities in coordination

the monitoring data collected in EuropeAid’s

with the contracting agency and its delegations

Common Relex Information System (CRIS)

in the respective countries, the Consortium

database, ROM provides an overview of the

members and Eptisa’s main office in Madrid.

performance of the EC development aid

For all matters related to ROM ACP the CO is to

portfolio. Quantitative and qualitative studies

be contacted at valvareztirapu@eptisa.com or

based on the ROM database provide lessons

+32 (02) 3406025.

tool

for

day-to-day

learned which feed into the project cycle.

Originally designed to monitor classic projects at country level, the use of ROM has gradually been extended also to regional projects and to 2

SPSP (Sector Policy Support Program) and for a

Navigate TROUGH the manual

Guidelines for Monitors for Regional Programmes Guidelines for Monitors to assess Commission’s Sector Policy Support Programmes under Result Oriented Monitoring System

ROM BACKGROUND

ROM TYPOLOGY

Type of ROM and eligibility criteria

Type of ROM

Stage of implementation

EC contribution

Exceptions

Ongoing

6 months into operational Approximately 1 million â&#x201A;Ź Smaller projects: sampling phase, 6 months before end of and more: should be according to added value and operational phase (flexible if priority for ROM and representativeness (normally not operational phase is about to ideally monitored each more than 10% of total ROM be extended) year universe)

Ex post

Ideally 12 - 18 months after end Approximately 1 million â&#x201A;Ź of operations although and more: should be flexibility possible according to priority for ROM added value of ROM

SPSP

6 months into operational phase, 6 months before end of operational phase (flexible if operational phase is about to be extended)

Should be monitored yearly ideally

Smaller projects: sampling according to added value and representativeness

Evaluation

Representativeness

Minimum 3 months after mid term review

Sample should include different DAC sectors, thematic budget lines, different kinds of performance etc.

Sample should include Not after ex different DAC sectors, post evaluation thematic budget lines, unless special different kinds of performance need etc.

ROM SERVICE LOT 2 ORGANIZATIONAL STRUCTURE DEVCO

EPTISA

Consortium partners

ROM Contract Management Unit

Headquarters Direction and coordination

ECO3, EPESMandala, ISS

Madrid

EPTISA ROM MAMAGEMENT TEAM

Team Leader Frans Geilfus Quality Assurance Managers

Deputy Team Leader

Joao Guimaraes, Alain Blondel

Roberto Canessa

François Colson, Pietro Magini

ROM Operational Coordinators) Valentin Alvarez, Carlo Coppola,

Support staff Junior experts Juncal Baez (admin) Andrés Moya Amanda De Joncheere

Elena Caballero

ROM Mission Leaders Team

MONITORS TEAM ROM Monitors

THE ROM EXPERTS ROM experts are assigned to the different operations to be monitored taking into account their thematic, geographic and methodological experience.

ROM

contractually

level

classified

expertise

according

is the

following categories: Mission Leaders, Senior Monitors, Senior Monitors (SPSP), Medium monitors, Junior Monitors. ROM experts are required to guarantee their best quality services in accordance with their specific level of expertise, before, mission.

during and after the

ROM experts must have adequate

mastery of PCM and ROM. Moreover, in addition to their specific thematic background, they should be perfectly familiarized with the EU policy relating to the thematic area covered by the operation(s) they are going to monitor. Specific linguistic skills (read, spoken and written) related to the language(s) utilized by the EU in the country(ies) were the mission is going to be undertaken, are indispensable. .

absolutely

ROM EXPERT’S GENERAL OBLIGATIONS It is the ROM expert’s obligation to fulfill his assignment in compliance with these general ToRs and with the mission’s specific ToRs. He will prepare accurately his mission by following the requisites listed below. For any doubt arising during the preparation phase he will ask for clarifications from the CO. During the mission he will carry out his work diligently, in accordance with

the different

guidelines

applying to the specific ROM exercise and possibly the instructions received by the ML. He will report to the ML and/or to the CO any unexpected situation which might hinder his mission’s accomplishment and will follow their instructions to overcome it. He will participate in all the preparation and synthesis activities organized by the ML. He will comply with the security rules possibly communicated to him by the EUDEL, the ML or the CO. He will produce the ROM outputs by correctly using the specific tools applying to the ROM exercise to which he has been assigned, and guarantee their best quality.

will

maintain

the

required

objectivity and confidentiality during the whole mission and after it.

Navigate TROUGH the manual

Rom expert classification by levels of experience The level of experience is defined in accordance with the ROM Handbook in the following way: Mission Leaders are Senior Monitors (more than 10 years relevant M&E experience) with proven extensive leadership experience in ROM and European External Cooperation in general, as well as communication and leadership skills. They must be prepared to assume the leadership of missions as well as, when necessary, the monitoring of complex or sensitive projects. Senior Monitors (more than 10 years of relevant experience) combine at least some specific ROM experience and/or broad experience in evaluation and monitoring of projects over a range of geographical and thematic situations. They are expected to assume the monitoring of the most important, complex or sensitive projects in a mission. Senior Monitors (SPSP): a subset of senior experts with particularly strong experience in relevant thematic areas, preferably including sector policy experience, as well as specialists of Public Finance Management and institutional strengthening and reform. Medium Monitors are young professionals with between 5 and 10 years of relevant professional experience. They have a substantial thematic and PCM experience, in different degrees a ROM experience, and are expected to monitor the projects that are not particularly complex or sensitive and where their specific experience can be best put to use. Junior Monitors are young professionals with less than 5 years relevant experience. They bring in new perspectives from the openness to new ideas and recent academic background, but the lack of broad experience limits their responsibility to small and simpler projects.

ROM expert qualification according to thematic areas of experience In order to respond to the service’s thematic challenges, the EPTISA Consortium has built up its pool of experts trying to balance the experts areas of specialization with the ROM service expected needs. The reference framework for thematic areas in cooperation is the OECD DAC “CRS purpose codes” for sector classification, used by all aid agencies. The following table shows the classification into 16 clusters; these are designated with names to avoid confusion with DAC codes. The classification is only intended for internal management of our experts’ database, and is not aimed at substituting any official classification. Experts’ qualifications can be assigned to one or several thematic clusters. Knowledge of local context, language and culture is also an asset for monitoring of any operation. Previous geographical experience gives preference for project allocation if other requirements are met.

Classification of expertise in thematic clusters

Cluster

Thematic areas included

Main DAC codes (2009)

Education

110

Health and population

Health & population (including HIV/AIDS)

120,130, 16064

Water and sanitation

140

Policy & Government

Policy and government administration, economic planning, PFM, decentralization, law

1511,1512, 1513

Civil Society

Democratic participation, elections, media, HHRR, women equality, NGOs

1515, 1516, 1517, 920

Conflict, emergency

Conflict, emergency response

152, 720

Social welfare

Social welfare, employment, culture

160

Infrastructure

Transport and storage, construction, urban development and reconstruction)

210,323, 43030, 730

Communications

Communications (including ICT)

220

Energy

230

Business

Business, banking & trade, non food commodity

240,250, 331, 530

Industry and mining

Industry & mining

321, 322

Rural and agricultural development

Agriculture, food security, rural development

311, 520, 43040

Natural resources, environment

Forestry , environment, disaster prevention

312, 410, 740

Gender & development

Gender & Development

15164

Others

Other specific (Fisheries, Tourism, alternative development)

313, 322, 43050

MISSION CYCLE

group of countries is referred to as â&#x20AC;&#x153;ROM

MISSION CYCLE

Missionâ&#x20AC;?. Standard ROM missions last normally two weeks, during which each monitor expert

ROM exercises are carried out on a regular

covers two operations and subsequently

basis for all eligible European Cooperation

produces the respective reports. However, in

operations (programs, projects, contracts) and

the

are based on short visits lasting from

competences are particularly required for just

minimum of four to a maximum of fifteen

one project, the respective ROM expert might

working days, according to the type of

be asked to participate for only one week in the

operation to be monitored. Simultaneous visits

ROM mission. In this section the different

to several operations by a group of ROM

mission stages (navigable through the following

experts (Mission Team) in a single country or

buttons) are explained.

cases

where

specialist

professional

Mission Planning Background documents analysis

Writing of Reports, Quality Control and encoding in CRIS

Mission Start

Post mission accomplishments

Monitoring visits

Navigate TROUGH the manual

Briefing

MISSION CYCLE

Pre briefing meeting

Debriefing

Missions are planned every year by the CO in

also be part of a regular mission or can be

coordination with the EC Headquarters and the

arranged in a separate specific mission. When

respective EU Delegations in the recipient

regional projects are included, the mission

country(ies). Specific ToRs for each mission

might involve traveling to different countries

specifying the dates, the operations to be

for one or more of the participants. Traveling to

monitored and the assigned experts are

more than one country can also be necessary if

approved by the EU normally not later than one

the EU Delegation responsible is located in a

month prior to the mission departure. Those

country different from the one where the

specific ToRs are attached to the present

monitoring exercise operations is implemented.

standard ToRs and remain the official guide

Contractually, as ROM providers we are bound

during the whole mission (unless modified

to guarantee a minimum number of working

through an official communication by the CO).

days in country, varying according to the type

ROM missions are coordinated in the field by a

of ROM (see next page). The mission overall

Mission Leader (ML). According to the different

schedule is arranged in agreement between the

typology of operations to be monitored, or the

CO and the Delegation and the respective

ROM exercise perspective, the mission can

travel implications are communicated to the

include ongoing or ex post monitoring at

ML and all the mission members, as part of the

country or regional level. Monitoring SPSPs can

specific ToRs.

Setting the mission agenda

MISSION PLANNING

The agenda should include, according to the type of project

Preparing the mission accurately is a prerequisite for a correct mission execution and successful mission results. Once all the experts for a monitoring mission are identified, the ROM Eptisa Brussels Office finalizes the experts team and sends the Terms of Reference to DEVCO for approval. The monitorâ&#x20AC;&#x2122;s participation in the mission is confirmed contractually only when the ToRs are approved by DEVCO and after verification of the absence of any possible conflict of interest (see respective section at page 123).

a minimum of 7 field work days(a) for ex-post monitoring (from Monday to Friday plus 2 additional working days(b) in the previous or following week)

a minimum of 4 field work days for regional component monitoring

3 more work days for regional consolidated monitoring.

Agendas should include at least 50% of the time available for project field activities aimed at obtaining information from primary sources (the direct beneficiaries of the project).

ROM Eptisa Brussels Office makes the ticket reservation and purchase of the international (and domestic flights, whenever possible) through a travel agency in Belgium (see box in the next page).

MISSION CYCLE

The expert will work independently but during the visit, will need information and logistical support from the Project management to facilitate field visits at project sites. It is important to note that the monitor will need to meet at least the persons who are responsible for project management, the management of institutions in charge of the project, all other stakeholders involved in the project as far as possible, and visit a representative sample of activities and beneficiaries.

ROM Eptisa Brussels Coordination Office (CO) puts in contact the Project Managers and the monitors responsible to monitor the respective projects. At this point, each monitor sets his agenda (see box) of visits and receives more information about the project (Financing Agreement, Contracts and Riders, Overall Work Programme and Annual Work Programme or equivalent, Progress Reports, Logical Framework, etc).

The monitor is responsible for obtaining the visa(s) that allows his/her entry into the country(ies) and to carry out the mission. When needed the CO provides each monitor with a letter from the EU to present her(him)self for visa application. Visa costs and the shipping cost associated will be covered by Eptisa, but not other expenses associated with its acquisition (travel), and will be included in the final invoice. Besides, the monitor is responsible for having the required vaccinations certificates in order to enter into the country, when this is needed.

a minimum of 5 field work days for ongoing monitoring (from Monday to Friday)

The above quantity of days will be reflected in each expertâ&#x20AC;&#x2122;s Time Sheet.

Once approved, the administrative and logistics issues of the mission begin, including the following steps: 1.

-----------------------------------------------------------

Fieldwork day: days devoted to visit any actor involved in the Project.

Working day: weekly working days (no holidays, Saturday or Sunday)

ROM Eptisa Brussels Office sends to monitors 4 contracted by EPTISA the contracts, insurance, first pay invoice, and the web link in which documentation is made available from

Monitors employed by any firm of the Consortium, other than Eptisa (Epes Mandala, ECO3 or ISS), will refer to their direct contracting firm for all matters relating to contracts, insurance and first payment.

the EU CRIS database for each project as well as other documents conducive to quality monitoring. 5.

The EPTISA monitor will send by e-mail to the ROM Eptisa Brussels Office the following documents scanned and correctly completed and signed: a. First Payment Invoice b. Contract c. Insurance Annex I

Logistics Consult the following websites for any doubts about visas and vaccination requirements

visas : http://www.visahq.com/visas.php vaccinations: http://www.netdoctor.co.uk/travel/in dex.shtml

Transfer of the First Payment will be executed before the beginning of the mission.

Organizing the travel o

Eptisaâ&#x20AC;&#x2122;s ROM Brussels office is available to consider monitors specific requirements compatible with contract rules, however unnecessary complications should be avoided and flexibility prevail;

Identify suitable itineraries according to your preference but respecting class (only economy) and calendar prescriptions (no per diem is paid during unnecessary travel days);

Departure and return points must be the same, normally your place of residence (should any exception to the above be required consult feasibility with CO);

We suggest that you arrive in the country in time, in order to rest and adapt to the place. Whenever possible the flight schedule should be arranged that you may participate in a pre-briefing meeting to be organized by the ML, normally on the evening before the official briefing in the EUDEL.

Chose your accommodation at your convenience considering the DSA value. Staying together or close to the other team members facilitates communication and team building. The ML normally suggests a common solution.

Procure as soon as possible a local communication system (local SIM card for your mobile phone) and provide local contact to Brussels office, ML and the rest of the team members.

MISSION CYCLE

If traveling for the first time to a country it could be useful to consult the own countryâ&#x20AC;&#x2122;s Ministry of Foreign Affairs WEB (f.e. Belgium http://diplomatie.belgium.be/en/services/tra velling_abroad/) and check aspects like driving license requirements, driving side, climate, customary uses, dressing, voltage, electric plugs, currency and money change situation.

BACKGROUND DOCUMENTS ANALYSIS

o o

Available latest progress report; MT evaluation (if available).

C- Internet available information ROM interlocutors in the Delegation, the NAO, the Project and the National Institutions expect that the ROM expert is fully aware of all the project background since his first meeting. It happens frequently that an expert’s questions will be answered by “… we sent you all the related documents …did you read them?” Without the intention of overloading the expert with too much unnecessary reading before the mission, it is strongly recommended that each monitor feels sufficiently comfortable, before arriving in the country, about her/his knowledge of at least the following: A - Country general o Socio/political/economic situation o EU Cooperation Strategy and Indicative Plan B - Project specific o Government specific sector policy and institutional set-up; o General principles of the EU budget line under which the specific project is financed; o Contracting mechanism, contracted organization (check if implementer is different from contract’s signing organization, and related implications); o Procedures applying to the specific contractual arrangement (International NGOs, National NGOs, UN agencies, Regional Organizations, etc.); o Implementation method; o Up-to-date and complete legal 5 documentation (FA, GC, CA, +Addenda) ; o Up-to-date and complete financial documentation; o Previous ROM report(s);

FA – Financial Agreement GC – Grant Contract CA – Contribution Agreement

MISSION CYCLE

It can be useful to make a quick search on the Internet about the specific context of the project and/or the project implementing agency and partners. Less experienced monitors are strongly advised to do background research of this type to improve their preparation. D- Methodological tools Be sure that you know exactly where to find the monitoring templates (PS, BCS, MR) and how to use them. As several versions might have circulated, make sure that you are using the correct version (check with CO in case of doubt). ROM outputs produced in invalid formats will be rejected.

MISSION START The mission starts with the expert’s departure from his place of residence and ends at the time of his return to the place of origin. However for contractual purposes the time contractually allocated to the mission is striclty the one defined in the standard mission’s ToR. Most of the experts’ activities will be carried out individually (except in the case of SPSP ROM where a two expert team is foreseen). Team joint activities will be restricted to the pre-briefing and briefing meetings at the mission’s start and the pre-debriefing and debriefing at the mssion’s end. The ML will be responsble for convening and coordinating these activities. The monitoring team members will normally arrive in the city where the European Union Delegation (EUDEL) in the country is situated on the Sunday before the briefing with the EUDEL. In case a monitor cannot get there on the Sunday, he should arrive on the Saturday. In case of ex post monitoring, the monitor will start the mission the Thursday before the briefing (if it´s planned during the first week) or stay until the Tuesday after the debriefing (if it´s planned for the second week).

In case of traveling complications before or while going to the mission’s destination get immediately in contact with the Brussels Coordination Office about: o o o o

Whom to inform Travel rescheduling Agenda rescheduling Contract implications

MISSION CYCLE

Security ROM mission are normally not carried out in countries/regions with critical security problems EUDEL gives the main guidelines ML gives operational instructions Reduction of mission mobility for security problems need to be agreed with the EUDEL at the time of briefing. In these cases an alternative methodological approach needs to be agreed between the TM and the monitor and endorsed by the ML. Some websites provide useful official security advice: UK Foreign & Commonwealth Office Security Advice: http://www.fco.gov.uk/en/travel-andliving-abroad/travel-advice-by-country/ French Ministry of Foreign Affairs: http://www.diplomatie.gouv.fr/fr/conseil s-aux-voyageurs/

PRE BRIEFING MEETING Time should be allowed, on the arrival Sunday, for the team to meet altogether in a convenient place (hotel meeting room) and convenient time (2-3 hours before dinner). An example of the topics to

be covered in this meeting are those listed here below. Local monitors, if any, can also give an overlook on the country situation during this meeting.

Pre briefing meeting - possible topics to be covered Mission scope Projects sample Type of projects/ROM approach Interlocutors o EU delegation o Project implementer o Other Stakeholders o Beneficiaries Recall of Methodology o PS o BCS o MR Recall of Toolkit o Templates o CSP o Basic information per each project (Contract, Addenda, CRIS, QSG) o ROM Handbook o ROM Manual (this manual) Recall of Documents to analyze o Policy documents o Project formulation document o Logical Framework (outputs/outcomes) o Baseline o Progress reports  Technical  Internal monitoring  Calendar  Budget execution  Auditing reports Field visits approach and security aspects o Credibility of reported IOVs o Quality of outputs/outcomes o Beneficiary opinion Briefing organization (next day) o Collective briefing o Individual briefing (points of attention/interest for the Delegation) Debriefing organization (at mission end) o When and how to prepare it o Project ROM summary slide (debriefing fiche with preliminary scores) o Aggregate considerations (homogeneous groups of projects).

MISSION CYCLE

BRIEFING

Also to be taken into account for the briefing

The members of the monitoring team will attend the briefing meeting with the EUDEL normally on the Monday morning of the first week, followed by more detailed bilateral meetings with the 6 respective task managers . During the inception meeting, the Mission Leader will introduce the members of the monitoring team as well as present a brief overview of the methodology to be used in the monitoring exercise and define the content, structure and set the date and hour for the debriefing. If the Delegation staff is interested the ML can show a Power Point presentation about 7 ROM . More detailed briefings with the respective task managers in the Delegation will include an overview of the project background, a review of its progress and identification of any relevant issue of interest for the monitoring exercise. Briefing main content Recall of the ROM purposes and approach Presentation of the mission experts List of projects to be monitored Calendar of the mission Operational Aspects Individual briefings with task-managers Specific aspects of interest to the Delegation Coordination of field visits Planning of general debriefing and Individual de-briefings.

For those experts who will only participate in the mission for one week and whose incorporation in the mission is scheduled for the second week, an individual briefng will be organized by the expert with the respective TM, for the missionâ&#x20AC;&#x2122;s start day or whenever suggested by the TM.

PP Model presentations are available, in different languages (EN, FR, PT, ES), at the Brussels Office and can be sent to the ML.

MISSION CYCLE

The appointment (place, date and hour) for the briefing needs to be formally confirmed with the Delegationâ&#x20AC;&#x2122;s Focal Person in advance by the ML or CO. No Power Point presentation is normally needed, unless requested by the Delegation for introducing the ROM methodology to new staff or outside partners. The points to treat in the briefing, apart from those related to the mission execution, and the security conditions (see next box) are: o special aspects to be considered by all monitors in their analysis o end of mission debriefing format o time, place, duration of the collective debriefing o who will participate o form of presenting the conclusions o homogeneous groups of projects to be monitored, for which results could be presented in an aggregated form at the time of debriefing o individual debriefings versus collective one If the NAO representatives are not present in the briefing, the ML will organize a collective or individual visit to their office to brief with them. The ML takes notes about the subjects discussed during the briefing and makes to circulate a participation list where the participants in the briefing write their names and positions (standard form provided by CO). The notes and the list are subsequently sent by the ML to the CO in a scanned form, to be included in the End Mission Note (EMN).

MONITORING VISITS From Monday (or first official mission day) onwards, all monitors will conduct their field visits to projects until the Thursday before the debriefing (set on the second Friday). Each project will be normally monitored in a different week.

Mission interlocutors and activities Project authority meeting Project implementing team (technical/financial) meeting Interviews with beneficiaries Interviews with other donors, EU MS Field visits Debriefing with project’s interlocutors

Monitors’ contacts during the mission Individual agendas should be known by everybody as well as contact access (hotel phone, counterpart phone and possibly local cell phone number). Circulation of information by each team member should be suggested rather than adding this responsibility onto the ML. Intermediate meetings among the team members during the mission should be facilitated. The ML should promote them, if possible, and all available monitors would attend them diligently.

MISSION CYCLE

Mission logistics and local costs Local Air Travels can be arranged, if possible, by the Brussels’ office. Ask them in advance. Local traveling shouldn’t be a constraint for accomplishing with the mission activity schedule, therefore take the necessary arrangements in time. Per diems cover ordinary city transport costs Other eligible costs are only those related to extra urban transport (air, road, water), as well as the airport taxes if properly documented (specific receipts). Exceptionally, costs of interpretation to a local language can be accepted, where indispensable. Daily allowances for the project’s or counterpart institution accompanying staff are normally not eligible (consult the CO in case of doubt). Supporting documents need to be in the name of Eptisa or the monitor’s name and specify at least the date, type of service, itinerary, fiscal or personal identification of the receiver. In case of emergency assistance requirements (sickness, stealing, accident, etc.) inform the ML and the insurance company, where necessary (see modality in the respective insurance contract documents provided by Eptisa or other Consortium Member). In case of restricted traveling situations inform the ML and the TM.

DEBRIEFING Debriefing presentation preparation The debriefing will include a global presentation of the preliminary conclusions and recommendations by the Mission Leader and the individual experts followed by more detailed individual debriefings with the respective Task Managers. The opinions and comments made by the Task Managers / Delegation Advisors will be taken into account for the reports. The format of the debriefings can vary from Delegation to Delegation and is to be decided upon during the inception meeting (briefing).

Pre-debriefing meeting A preparatory meeting, prior to the formal debriefing is indispensable. The recommended starting time for it is not later than 17h on the Thursday before the debriefing day. A respective appointment should be arranged between the team members after the briefing, taking into account the agreed format for the debriefing. The ML would have circulated in advance the respective Power Point layout, indicating where and how each monitor should contribute. During the meeting the PP presentation will be finalized and roles assigned for presenting the various parts. Conclusions per theme or other homogeneous groups of monitored projects should be agreed between the respective team members, if the Delegation has expressed interest for it.

All monitors are required to prepare a debriefing fiche on each project to be presented at the debriefing.

MISSION CYCLE

The ML should be helped by one team colleague who has had experience in the preparation of the PP presentation (Brusselsâ&#x20AC;&#x2122; Office members when participating in the mission) . Examples of the PP can be obtained by the ML from the Brussels Office, in different possible formats.

Conducting the debriefing Leave a copy of the presentation at the Delegation Even if a format has been previously agreed but availability constraints by the Delegations Staff oblige to shorten or modify it, be flexible. If a project by project method has been requested for the debriefing, individual projects conclusions to be presented should focus only on strategic aspects. Avoid details that have been already (or will be) treated with the respective TM. Summary scoring should be presented first, highlighting main trends (verticalhorizontal), with respect to previous ROM exercises. Discussion should be favored throughout the presentation. Conclude with some strategic recommendations related to the EU overall strategic programming. End by asking if the Delegation has any suggestion relating to next yearâ&#x20AC;&#x2122;s ROM mission (projects to be monitored, timing and aspects to be taken into consideration). The ML takes note of the above and informs Brusselsâ&#x20AC;&#x2122; office by a brief note to be included in the EMN (End of Mission Note).

POST MISSION TASKS

Administrative Documentation. (to be sent by DHL to the office in Brussels)

Monitors main responsibilities post missions are: 1. 2.

To deliver their reports in timely fashion and collaborate with Quality Control (see further); To provide the necessary administrative follow-up.

Original Timesheet signed in both pages (CO will send the timesheets pre filled)

Boarding passes of all (international and national)

Final Invoice (original) and reimbursable costs supported by invoices and boarding passes (be reminded that we are talking about internal flights if you have paid them and/or transport from one city to another but not taxis â&#x20AC;&#x201C; the EU considers that this is included in the per diem). For the local costs in a currency other than EURO, you will have to apply INFOREURO rate (http://ec.europa.eu/budget/inforeuro/ind ex.cfm?Language=en) unless you can provide different exchange rate proof.

Original contract signed (two copies) and original pre-paid invoice signed (in case you have asked the pre-payment before the mission)

Although the expert main responsibility ends once her/his report(s) have been accepted by the Quality Control and uploaded to CRIS for further dissemination, it is important also to make sure that all the contractual obligations have been met in order to permit the formal closure of the contract. Therefore, after the return from mission, apart from concentrating on the Report writing and delivery, responding to possible quality control questions and providing the reports final version in time, the expert will: 1.

Collaborate with the CO for the missionâ&#x20AC;&#x2122;s contractual closure. The monitor will forward to the Brussels Office, by e-mail in a scanned form and duly signed, the documents indicated in the box in the right margin. Once approved by the CO, the originals of the same documents will be sent by DHL, or another 8 courier .

Return to the CO the collected documents for future re-use. All the useful information about the project that you have been given in electronic format will be sent by e-mail to the CO in order to provide further information to monitors over coming years.

Courier costs are paid by Eptisa only once per expert mission. If more courier services would be needed due to monitorâ&#x20AC;&#x2122;s errors in sending the documents, these will not be borne by the company. MISSION CYCLE

your

flights

Quality Control

WRITING OF REPORTS, QUALITY CONTROL AND ENCODING IN CRIS

Writing of reports The reports will be submitted by the monitors to the ROM Eptisa Brussels Coordination Office (CO) within the deadline set in their contracts. The reports will be written either in English, French, Portuguese or Spanish (depending on the mission), should be free of linguistic errors, clear and objective. The reports will contain the opinion of the monitor based on his/her findings in the field, conclusions and recommendations. For each project assigned, the monitor will be responsible for producing the following documents: The Project Synopsis (PS), according to standard format and respective instructions, that summarizes the highlights of the project (see suggestions in the respective section of this Manual). The Background Conclusion Sheet (BCS) that is the basis of the Monitoring Report and its ratings, as laid out in the ROM Handbook and the respective instructions set in the respective section of this Manual. The Monitoring Report (MR), in accordance with the format provided and respective instructions. It should be clear, concise, complete and consistent (see suggestions in the respective section of this Manual). In order to facilitate the quality control the expert will also deliver, together with the above reports, the following two documents obtained from the monitored project’s management: The updated Logical electronic version

Framework

The updated Schedule of Activities and project implementation (if available).

MISSION CYCLE

All reports will be submitted to a thorough quality control. The monitor will have to provide answers/commentaries to questions and suggestions which might be communicated within the two weeks following the end of the mission by Eptisa’s Quality Control Team. Reports not meeting the required quality and not complying with the respective instructions, will be sent back to the expert for modification and correction. The expert should answer those observations/questions as soon as possible in order not to delay the submission of the reports to the EU . The monitor will have accomplished all his/her responsibilities once the reports are finally accepted and uploaded onto the CRIS database. In the case where, after the expert’s correction, the reports are still be inadequate according to the required standards, they will be considered “not accepted” with the consequent contractual repercussions. All reports produced by the monitor remain the property of the UE and for no reason can be diffused by the author.

Encoding the ROM reports in CRIS The final Monitoring Reports, Background Conclusion Sheet and Project Synopsis documents will be forwarded to DEVCO headquarters (C3) after the mission has been completed. These documents plus the respective logical frameworks and work plans (if available) will then be uploaded onto CRIS, whilst the Delegation will be requested to fill out the TM/DA Response Sheets in order to provide constructive feedback for all parties concerned in the monitoring exercise. The CO will later forward the respective RS to the monitors for their information.

THE USE OF THE TEMPLATES

The expert is responsible for producing the standard reports according to the ROM methodology and instructions, in the form and within the terms established for each ROM mission. The ROM methodology foresees the use of standard templates according to the ROM typology as it is indicated in the table below. The expert will make sure that she/he is using the appropriate templates before starting the writing. Reading accurately and complying with the instructions contained in this Manual regarding the different type of report, is mandatory. In particular, a the MR is the most important output destined to the wider public, it needs to contain, in a clear and concise form, all the ROMâ&#x20AC;&#x2122;s major findings and recommendations.

Templates to be used for the different types of reports

Type of ROM

BCS

ongoing

PS/P/OG

BCS/P/OG

MR/P/OG

ex post

PS/P/XP

BCS/P/XP

MR/P/XP

PS/SPSP

BCS/SPSP

MR/SPSP

ongoing

PS/P/OG

BCS/P/OG

MR/P/OG

ex post

PS/P/XP

BCS/P/XP

MR/P/XP

ongoing

PS/P/OG

BCS/P/OG

MR/P/OG

Project approach Country level SPSP

Country component Regional Consolidated

ROM CONCEPTS

ROM is based on the systematic analysis of the project/programme intervention strategy and its performance. In order to apply the overall framework for evaluation which is the methodological basis of ROM, one needs to understand what the intervention strategy is. This preliminary exercise is necessary because, despite all the attempts to standardize the way project planning is done, in particular through the logical framework (logframe), there are variations in the way strategies are presented, and methodological mistakes are common.

HOW TO ANALYSE THE LF

The Logical Framework Matrix (LFM) is defined in the EU PCM Guidelines as “ a matrix which summarizes the key elements of a project plan, namely: the project’s hierarchy of objectives (Project Description or Intervention Logic); the key external factors critical to the project’s success (Assumptions); and how the project’s achievements will be monitored and evaluated (Indicators and Sources of Verification). To start doing a meaningful analysis, the Monitor needs to understand clearly what the Project intervention logic is. It is supposed to present a causal chain of actions and their intended effects, based on an analysis of the causal factors and central problems that the intervention addresses. The project strategy (the causal chain of actions and effects) needs to be translated in clear form in the LFM.

WHICH ARE THE MOST COMMON LF DEFICIENCIES

ROM is not always a straightforward exercise. The logframe is supposed to be the reference for structuring the monitoring exercise. However there

Navigate TROUGH the manual

are common occurrences that can make this exercise difficult: There is neither logical framework nor anything equivalent: this case is getting rarer as it has become an obligation for all projects but may still happen (i.e. regional projects components). The logframe is replaced by a different format or different terminology: this is the case when the project implementation is delegated to agencies with specific PCM instruments differing from the EC format, such as UN agencies . The logframe exists but does not reflect adequately the project strategy and therefore is not used as management tool: this is by far the most common occurrence. The logframe exists but the terminology and levels are confusing: elements of the causal chain are not included at the right level, for instance the LFM presents as results a mix of outputs, outcomes and impacts mixed at the same levels. This is also quite common.

INTERVENTION LOGIC INDISPENSABLE ELEMENTS

You will observe that the new BCS format does not stress anymore the logframe as sole reference but refers to “similar tools” or “project framework” or more generally “intervention logic”. In order to use a consistent approach for monitoring all interventions, the monitors have to discuss the same levels of intervention related to efficiency, effectiveness and impact: Efficiency is how well the project inputs are invested in activities in order to achieve planned outputs. Effectiveness is how well the project achieves its planned results/outcomes and how well these contribute to the project purpose. Impact is how well the project implementation is contributing to the overall objectives.

ROM CONCEPTS

Therefore, we all need to be extremely clear about what exactly are inputs, activities, outputs, outcomes/results, project purpose and overall objective. If you try to answer the questions in the BCS without understanding and making a clear distinction between the different levels of the intervention logic, you will most probably generate confusion between the criteria. Even seasoned monitors sometimes incur in this mistake. It is thus very important to discuss those different elements at the right level, so that even if the formal logframe is confusing, the discussion of ROM criteria will remain clearly focused.

BASIC ROM CRITERIA

The first step for a good ROM exercise is therefore to make sure that the intended intervention logic is organized in a way that allows applying the criteria for analysis in a meaningful form. In fact, a monitor is expected to be able to analyze performance without a proper LFM. We will always use the following definitions for the various levels of intervention, derived from the OECD DAC guidelines, whatever the terminology used by a specific project: OVERALL OBJECTIVE (Sometimes called Goal): It must correspond to broad, long term development changes– generally at a national or sector level (providing a link to the policy and/or sector programme context). The project is not supposed to deliver this objective fully, but to make significant contributions to it. If the highest level objective in the project logframe does not refer to broader development aspects, it is probably actually the project purpose. In that case the monitor must figure what are the existing corresponding development objectives (anyway you have to identify them under relevance). PROJECT PURPOSE (sometimes called Objective or Specific Objective, if only one). The central development outcome expected to be delivered at the end of the project – more specifically the main expected benefits

to the target group(s). Sometimes there is none. Often there are several, called specific objectives. The way to check is to assess whether one of these specific objectives is actually a consequence/effect of achieving the others, then this one is the PP and you can use it as such in the discussion (and recommend adjustment of the logframe). The others are then results/outcomes. PROJECT RESULTS (Sometimes called Specific objective or Outcome): the likely or achieved short-term and medium-term effects derived from the outputs delivered by the project, in terms of capacities and other benefits for target group(s), and which are necessary to achieve the purpose. There are normally several results which are supposed to be derived from the outputs delivered by the project. Outcomes are generally expressed in terms of capacities and other benefits for target group(s). The vertical logic supposes that in order to achieve the Project Purpose, the results are necessary. They are therefore, intermediate outcomes. There is often a logical confusion between results and project purpose (see above) and also, more often still, a confusion between results and outputs. The punctual delivery of outputs leads to the expected results. However, the benefits that accrue to beneficiaries are not fully under project management´s control. OUTPUT (sometimes called Result): direct/tangible goods and services that the project delivers, and which are largely under project management’s control. It is advisable to clearly differentiate outputs and outcomes when they are presented under the common heading of “project results” ACTIVITIES: Activities are the tasks (generally object of the work programme) that need to be carried out to deliver the planned outputs. They are fully under project management control. There is normally never any confusion about the activity level. INPUTS: the financial, human, and material resources used for the development intervention. They are contractual in nature.

ROM CONCEPTS

ASSESSING THE “QUALITY OF DESIGN”

It is very important, at the beginning of the monitoring visit, to assess the viability of the logical framework and reconstruct a functional logframe for monitoring purposes when the current document presents serious deficiencies. Such exercise is part of the assessment of the “quality of design” criterion anyway. The following cases are frequently observed: 1.

The LFM/planning framework clearly indicates the levels of intervention as defined before: there is thus no methodological issue and no need to fix the logical framework for monitoring purpose.

Some confusions in terminology or definition exist: for instance results mix concrete outputs and qualitative outcomes, etc. You just need to clarify what is what for evaluation purpose.

The reference planning is totally confused and confusing (unfortunately common!): you then have to rebuild a simple framework of the intervention logic .

SUGGESTING “LF” IMPROVEMENTS In order to use a consistent approach for monitoring all interventions, when the project documents do not provide a useful logframe for monitoring, try to reconstitute a simple framework of the intervention logic so that you are able to apply the ROM methodology. You are not supposed to elaborate a new LF matrix, but you just want to be clear of what is actually the OO, PP and results/outcomes of the project in case they are misplaced or inadequately expressed in the existing matrix. This “proxy” framework has to respect the levels of intervention as defined above.

By checking the vertical logic of the project and by setting the correct vertical chain of causality, you will be able to discuss the ROM criteria without confusion. Most probably you will also have

relevant recommendations for improving the logframe. This may look complicated and requires, over and above of some experience in PCM, a good dose of common sense. The following diagram illustrates the flow of the logic of intervention as a reference for the ROM exercise.

1.1 RELEVANCE

ROM CONCEPTS

OVERALL OBJECTIVE (Sometimes called Goal) The broad development changes to which the project contributes – generally at a national or sectoral level (provides the link to the policy and/or sector programme context)

PROJECT VERTICAL LOGIC and ROM

4. IMPACT

(With reference to PROJECT PURPOSE (sometimes called Objective or Specific Objective, if only one) The central development outcome expected to be delivered at the end of the project – more specifically the main expected benefits to the target group(s)

3.2 EFFECTIVENESS

PROJECT RESULT 1 (Sometimes called Specific objective or Outcome) The likely or achieved short-term and medium-term effects derived from the outputs delivered by the project, in terms of capacities and other benefits for target group(s), and which are necessary to achieve the purpose 3.1 EFFECTIVENESS

Outputs for result 1 (sometimes called Resut 1) Direct/tangible goods and services that the project delivers, and which are largely under project management’s control

2.3 EFFICIENCY

BCS)

3.2 EFFECTIVENESS

PROJECT RESULT 2 (Sometimes called Specific objective or Outcome) The likely or achieved short-term and medium-term effects derived from the outputs delivered by the project, in terms of capacities and other benefits for target group(s), and which are necessary to achieve the purpose 3.1 EFFECTIVENESS

Outputs for result 2 (sometimes called result 2) Direct/tangible goods and services that the project delivers, and which are largely under project management’s control

2.3 EFFICIENCY

Activities for result 1

Activities for result 2

The tasks (work programme)that need to be carried out to 2.2 EFFICIENCY deliver the planned outputs

Inputs for result 1

Inputs for result 2

The financial, human, and material resources used for the 2.1 EFFICIENCY development intervention.

5. SUSTAINABILITY

1.2 – 1.4 QUALITY OF DESIGN

MONITORING STEP BY STEP

MONITORING STEP BY STEP The ROM reporting work by itself can be divided in 5 steps. It is important, especially for less experienced monitors, to follow always the logical sequence as indicated below. This is essential for ensuring the consistency and homogeneity of the methodology.

IF YOU FOLLOW CLOSELY THE INSTRUCTIONS IN THIS MANUAL AND IN THE TEMPLATES, YOUR REPORTS WILL NOT NEED MAJOR IMPROVEMENTS AFTER YOU PRESENT THEM. SO PLEASE PAY ATTENTION TO DETAILS!

COLLECTION OF DATA FOR MONITORING

The preliminary stage is for the monitor to collect the necessary data and background information. This is done before the mission begins or right at the onset of the mission. With the background information the monitor is to prepare (or update) the Project Synopsis. It should be ready before the briefing at the EU Delegation.

PROJECT SYNOPSIS (PS) During the monitoring field work, the monitor collects information from stakeholders and field visits. In the meanwhile he should be able to start working on a draft Background Conclusion Sheet (BCS) with preliminary answers to the check list of questions, and preliminary scores. Before the EU Delegation debriefing the monitor must prepare a short Debriefing Fiche that the Mission Leader collates in a Power Point presentation. After debriefing and the end of the mission, the monitor reworks the Background Conclusion Sheet (BCS) with the information available, including feedback from debriefing, and produces the final scoring.

BACKGROUND CONCLUSION SHEET (BCS) (On going projects)

DE-BRIEFING FICHES

The Quality Control team reviews the three reporting outputs: PS, BCS and MR and if necessary, returns them to the monitor if improvements or clarifications are needed. The time left for final corrections is normally very short.

Preparation of Monitoring Report (MR)

MONITORING STEP BY STEP

The final output is the Monitoring Report (MR) which has to be ready after the mission, within contractual deadline. It is the main output of the ROM mission.

COLLECTION OF DATA

COLLECTION OF DATA BASIC DOCUMENTS CHECKLIST For each project monitoring activity a series of basic documents are necessary: policy and country context documents necessary to understand how the project fits in the policy context;

employment) and cross cutting issues (democracy, good governance, human rights, the rights of children and indigenous peoples; gender equality; environmental sustainability; fight against HIV/AIDS). Relevant national Policy Papers: useful documents are sectoral policy documents and Poverty Reduction Strategy (PRSP) documents.

PROJECT DOCUMENTS

project documents essential for monitoring. Some of these documents are retrievable prior to the mission, from the CRIS database or by e-mail from the EU Delegation task manager and project management. Others must be procured locally once the mission starts, either at briefing with the EU Delegation or in the introductory meeting with the project management.

Financing Agreement (EDF), Grant Contract (NGOs and thematic lines), Contribution Agreement (UN family). In the FA the Specific Conditions and the Annex II have the basic information on the project. In the Grant Contracts, Annex I describes the projects content (see table in the next page). Riders modifying the FA or Contract.

POLICY AND DOCUMENTS

COUNTRY

CONTEXT

Logical Framework (normally annexed to FA or Contract). Budget (normally annexed to FA or Contract)

EC Country Strategy Paper (CSP): CSPs serve as the bases for the EU development cooperation in the country and they spells out the rationale and main areas of intervention. They are available for almost every country th for 10 EDF. EC Country Indicative Programme: this is the financial planning corresponding to the CSP and normally annexed to it. EC Regional Strategy Papers: available for the th different ACP regions for 10 EDF. Sectoral EC development policy documents are available covering many of the EC priority policy areas (trade and regional integration; the environment and the sustainable management of natural resources; infrastructure, communications and transport; water; energy; rural development, territorial planning, agriculture and food security; governance, democracy, human rights and support for economic and institutional reforms; conflict prevention and fragile states; human development; social cohesion and

Navigate TROUGH the manual

CRIS fiche (normally provided by Eptisa before mission- to be checked for updates before final redaction). Quality Support Group fiche and checklist to be asked from the Delegation, if not provided by Brussels Office with other support documents. Additional contracts (TA contracts, and other sub contracts for services, works and supplies). Project annual and overall work plans including specific TA work plans if available. Project implementation reports, especially the latest if they contain accumulated data. Otherwise the whole set of progress reports will have to be reviewed. Project communication and visibility plan. Previous MR and PS. Mid term reviews or other related evaluations.

COLLECTION OF DATA

ROM official reference documents according to different types of operations to be monitored

Operation to be monitored Financial Agreements (FA)

Grant Contracts (GC)

Contribution Agreements (CA)

Service, work or supply contracts (SC)

Reference Number

Decision number

Contract Number

Budget lines

Type of partner

EDF, Facilities

Government or regional organizations

Thematic lines (mostly), Facilities

Nonprofit organizations, local authorities

Can apply to all lines, frequent in EDF

Mostly as an operational instrument of FAs

UN family organizations

Profit organization

ADDITIONAL USEFUL DOCUMENTS Additional documents may be useful. However, remember that ROM is not a full blown evaluation exercise and time is limited, so concentrate on the most important and concise sources of information..

POLICY AND COUNTRY CONTEXT DOCUMENTS

Country annual reviews are available for some countries. They are joint evaluation exercises with the partner countries. Other donorsâ&#x20AC;&#x2122; strategy documents may also provide important additional information.

ROM Reference Document

CRIS data

What to monitor

Legal

Technical

Special conditions

Annex II (Technical & Administrative Prescriptions)

Normally included in Annex II

Decision level

All that is prescribed in FA

Contract

Annex 1 (Description of the operation)

Normally included in Annex I

Contract level

All what is prescribed in Contract

Contract

Annex 1 (Description of the operation)

May be included in Annex II or not existing

Contract level

All what is prescribed in Contract

Annex 1 (Terms of Reference)

Normally refer to the LF of the respective FA

Contract

Contract level

always monitored as part of FA, rarely object of individual ROM

PROJECT DOCUMENTS

Project background documents. Thematic studies and consultant reports related to specific aspects of the project. Project communication publications.

materials

and

COLLECTION OF DATA

WHO IS RESPONSIBLE FOR COLLECTING DOCUMENTS The Brussels ROM office will collect prior to the mission all documents available on CRIS. The availability of useful documents varies a lot on CRIS. These documents are normally made available for the mission team through a specific web access, before the mission starts. The monitor is responsible for collecting all additional necessary documentation from the EC Delegation and project authorities. In exceptional cases when the preliminary CRIS search will not be possible (CRIS is sometime inaccessible for several days due to technical problems), the monitor cannot blame the CO for the lack of documentation and she/he will be responsible to obtain all the required documents in the field. At the end of the mission the Monitor will send to the ROM office digital copies of the most important documents, giving the priority to updated logframes and the latest reports.

PROJECT SYNOPSIS (PS)

The Project Synopsis is the first output that the monitor should prepare: it serves as background information to start the review process. You should prepare (or update, see below) the PS at the onset of the mission.

WHAT IS THE PROJECT SYNOPSIS? The Project Synopsis is a concise overview of the project background, logic of intervention and administrative data. It is used as a reference fiche and is uploaded in CRIS together with the Monitoring Report and the Background Conclusion Sheet. The PS is NOT supposed to include information or appreciations and observations on issues related to the project implementation. This has to be done in the Monitoring Report. The PS is basically an abstract of the project.

WHEN TO SYNOPSIS?

PREPARE

PROJECT

The Project Synopsis has to be prepared by the monitor for each project (or project component of regional programmes when monitored separately). When the project has been previously monitored, the existing PS has to be updated. The Project Synopsis is the first document to be prepared by the monitor and it serves as a check list. All the basic information regarding the project must be available in order to be able to elaborate the PS.

HOW TO PREPARE SYNOPSIS?

THE

PROJECT

The PS should be redacted using the template available for this purpose (see example in this section). The document should fit in one page. Please follow the instructions below to fill in the information.

Navigate TROUGH the manual

ADMINISTRATIVE DATA A part of the administrative data are provided before the mission in an Excel sheet. Other data are to be found in the CRIS fiche. You will need to review the CRIS fiche and FA in order to fill the project data. You are supposed to check CRIS data validity during the mission, preferably during the briefing with the project task manager. Update any data when needed.

Project Title: this is the official name appearing in the financing agreement/contract. Project Number: this is the CRIS number found in CRIS fiche. It can have different formats (see table in page 37). The operation you monitor is normally identified as a decision (common case for EDF projects), or a contract (common case for NGO projects), contribution agreements, or subcontracts under a decision. IF the operation monitored is a contract that is part of a wider decision, you can include both numbers, preceded by D (decision) or C (contract). Date Financing Agreement: this is the latest date that the Financing Agreement (FA) or Contract was signed by one of the parties. The most accurate way to check the exact date is look at the signatures section in the FA/Contract itself. Country: use the official country name or region name. Start Date â&#x20AC;&#x201C; actual: this is the actual start date of the operational phase (often the date of the last signature on FA- always check in the FA). This is NOT the decision date! End Date - planned: this is the date of the end of the operational phase as stated in the Financing Agreement or contract (see Art. 4 of specific conditions and do not put the final date for execution which is usually two years later for EDF projects); if there is an approved rider modifying this date use the new date.

PROJECT SYNOPSIS

PROJECT SYNOPSIS (PS)

End Date - likely: this date may be different from planned if an official decision to extend the project has been already taken and a rider to the FA/Contract is being processed or likely to be approved. In all other cases it is the same date as the planned one. Primary Commitment/ EC contribution: this is the global financial commitment of the EC as stated in Financing Agreement and possibly modified by riders.

Agreement (Annex 2) or the contract will provide all the necessary information. 2. Project Intervention Logic Summarize the project intervention logic: overall objective, project purpose, results or components. Summarize main activities if you have enough space. Do not cut and paste from project documents but provide a readable abstract.

CHECK LIST TEXT OF THE PS

The PS is a short fiche. It should not be more than a page long. Please respect the template font (Arial 11, justified).

1. Project Background Write a paragraph on the general context, the issues and problems that the project is intended to address. Indicate the target groups and implementation partners. Mention briefly the project origin, if relevant. Normally the Financing

Attached to the PS template you will find a table that we have added to check the correct filling of the document. The check list is to be used by the Quality Control, but it can also provide the monitor with a synthesis of quality control criteria that should be useful to prepare the report. In case your first version does not comply with those criteria, you will receive a list of observations about the necessary improvements. Please be aware that your report will not be accepted until you answer to all the observations sent by QC.

PROJECT SYNOPSIS (TEMPLATE) Project Title: Decision Number: Contract Number:

Date Financing Agreement:

Country: Start Date – actual: End Date - planned: End Date - likely: Primary Commitment:

€

1. Project Background

2. Project Intervention Logic:

PROJECT SYNOPSIS QUALITY ASSURANCE CHECKLIST (not to be erased before QC) Aspects of Project synopsis to be checked

Project data

Comments (reserved to QC) To be revised

Project data copied from Excel table data and pasted in corresponding cells in project data table in PS (use paste cells as RTF to avoid format problems) Data revised / updated according to latest available CRIS data

General aspects of text (compulsory)

To be revised

Template respected: Text Arial, 11 pt, English (UK), Justified space: simple. Title Arial 11 pt, bold, space below 6 pt Not more than 1 page for whole PS Only basic descriptive information, no comments or opinions If PS already available, check if need for update, in any case transfer info on new file using template Check orthography and syntax !!! For country components of regional programmes: specific PS with specific country information

Project background

Always: short presentation of issues and problems to be addressed by project (no general information on country if not relevant to project focus) If relevant, short reference to sector policy or other important aspect of context If relevant, short description of origin of project (call for proposals, new phase of previous project, government request etc)

To be revised

Always: target group of project Always: project intervention area Always: implementation agency (ies) To be revised

Project intervention logic

Always: as in Financial agreement or latest update, intervention logic must include overall objective, project purpose and all results If room available include some information on main activities, if relevant Indicate if some important modification to intervention logic, project implementation period etc by rider to financial agreement

Other comments by QC if relevant

BACKGROUND CONCLUSION SHEET (BCS)

BACKGROUND CONCLUSION SHEET (BCS) The Background Conclusion Sheet (BCS) is the basic ROM tool. This is the instrument that provides methodological consistency and coherence to the whole exercise. The BCS is of the utmost importance since it is also the basic reference for the Quality Control of reporting. Each project monitoring report (whether of a whole project or a component of a regional programme) must be based on and accompanied by a BCS.

WHAT IS THE BCS?

NAVIGATING THROUGH THE BCS INSTRUCTIONS Use the buttons below to navigate through the different sections (*)

1. RELEVANCE AND QUALITY OF DESIGN

2. EFFICIENCY

3. EFFECTIVENESS

It is the questionnaire that provides the methodological structure for the monitor’s analysis.

4. IMPACT PROSPECTS The latest version of the BCS 2009 (v1) is currently used. It is composed of 7 sections (see list at the side). Each section contains main questions and sub questions which need to be answered by the monitor in order to measure the degree of project’s performance. Answers to each question are to be scored according to a predetermined scoring system (see next page). Although it can be considered a tedious exercise, without it, the monitoring reports would lack methodological consistency. The scores given to each sub question and question would not have any objective reference without it. Detailed guidelines for answering the different questions and sub questions are included in the following chapters of this manual. Please read them carefully before filling your first BCS or consult them regularly to clarify any doubt for subsequent ROM exercises.

5. POTENTIAL SUSTAINABILITY

6. HORIZONTAL ISSUES

7. CROSS CUTTING ISSUES LIST OF PERSONS INTERVIEWED DOCUMENTS ANALYSED

bcs –/tE sEC MP TIO LAT N E2 bcs – TEM sEC PLA TIO TE N3 bcs – /TE sEC MP TIO LAT N4 E bcs /TE –MP sEC LAT TIO E N 5 bcs /TE – MP sEC LAT ETIO N6 bcs – sEC TIO N7

BCS

(*) left buttons = instructions right buttons = template examples

/tE MPL ATE bcs –/ sECT tEM PLAT ION 1 E

Navigate TROUGH the manual

BACKGROUND CONCLUSION SHEET (BCS)

HOW TO FILL THE BCS? Filling each sub section in four steps The BCS is an OFFICIAL DELIVERABLE PRODUCT, uploaded on CRIS and subject to quality control. Monitors are therefore advised to fill the BCS accurately. Although less demanding than the Monitoring Report (MR) from a formal point of view, the BCS needs to be filled with clear text, free from linguistic errors, in line with the following instructions :

In order to facilitate the work of monitors, we have included resumed “subtitles” in each text box to guide the answers to each sub question of the BCS. Each sub question or group of sub questions is to be scored separately so that the scoring of the whole question will be suggested automatically through a predetermined weighing system which takes into account the relative importance of each sub factor. The monitor will score each sub question first and select the resulting overall score in the corresponding buttons. However, if the resulting score does not express exactly his judgment, because in that specific case some sub factor weights differently from standard, the expert can put a different score in the button.

answer clearly each (relevant) question and sub question with short and specific sentences; include specific information on the project (see further) but do not just describe; justify clearly the scores (see further); do not answer by “yes” or “no”, be concise but specific; do not overflow the text boxes, nobody is going to read an overcharged BCS;

This information is also very important for later statistic analysis of ROM results.

do not “cut and paste” from project documents.

fill the text by answering to the related questions

weighing coefficients must never be changed in any of the sections

(see example here below)

choose a score from “a” to “d” according to your judgment

copy the resulting score by selecting the corresponding button

read the overall score calculated automatically by the template

BACKGROUND CONCLUSION SHEET (BCS)

WHEN TO START FILLING THE BCS Although some monitors are used to proceed differently, the correct method is to fill the BCS before the MR because the BCS must serve as background support to the observations and recommendations of the MR. It is recommended to start filling up the BCS as soon as possible during the mission. The monitor should try to have a first draft of the BCS ready before the mission debriefing. This will facilitate discussion of conclusions.

THE SCORING SYSTEM

Score

Detailed scoring reference table

(very good) means that for the criteria/sub criteria considered, the situation is highly satisfactory. All sub questions have fully positive answers or/and there is an exceptionally good aspect compensating other minor problems (possibly good practice). Recommendations related usually point to the need to learn, systematize or replicate the approach used.

(good) means that for the criteria/sub criteria considered, the situation is fairly satisfactory; whatever observations or possible improvements mentioned, the actual level of quality should not put into question the global performance of the project. Recommendations related point to improvements that are useful but not vital for project performance.

(problems) means that for the criteria/sub criteria considered, there are issues that need to be addressed, improvements necessary, otherwise the global performance of the project might be affected. The necessary improvements do not imply major revisions of the project strategy. Recommendations related must be addressed in priority

(serious deficiencies) means that for the criteria/sub criteria considered, there are deficiencies and issues of so serious a nature that if they are not addressed, the project may face overall failure. Major adjustments and revision of the strategy are necessary otherwise the project will not achieve its purpose.

The use of scores gives a lot of latitude for personal appreciation. In order to ensure the greatest consistency between monitors, the following criteria must always be applied when scoring (see more details in the table at the side): A: the sub question have fully positive answer or/and there is an exceptionally good aspect compensating other minor problems (possibly good practice). B: there are only minor issues not affecting performance significantly.

the

C: there are issues that need to be addressed, otherwise the performance could be affected. D: there are major issues that need to be addressed urgently, otherwise important aspects of performance, could be in jeopardy. IT IS VERY IMPORTANT THAT ALL MONITORS USE THESE REFERENCES CONSISTENTLY. Some common practices must be avoided because they affect objectiveness: Put a “D” score to “attract the attention” even if the project situation does not warrant it. Put an “A” score just because you like the project’s approach or are enthusiastic about its relevance. Change scores on sub-criteria in order to have the “desired” score at criterion level.

The JUSTIFICATION OF SCORES must appear clearly in the BCS (and later in the MR). For “A”, “C” and “D” scores the justification (and corresponding recommendations or best practices description) are essential and must appear both in the BCS and the MR. For RE-MONITORED projects, be careful to explain any striking evolution of scores in relation to the previous monitoring. You do not have to agree with the previous monitor but you are expected to apply the criteria with objectivity.

BACKGROUND CONCLUSION SHEET (BCS)

1. RELEVANCE AND QUALITY OF PROJECT DESIGN The relevance and quality of project design are defined as the “appropriateness of project’s objectives to the real problems, needs and priorities of its target groups/beneficiaries and the quality of the design through which these objectives are to be reached”.

SUB CRITERION 1.1 - WHAT IS THE PRESENT LEVEL OF RELEVANCE OF THE PROJECT? 1.1.1 ARE THE PROJECT PURPOSE AND OVERALL OBJECTIVES CONSISTENT WITH, AND SUPPORTIVE OF PARTNER GOVERNMENT POLICIES? Check if there are specific policy documents related to the project action. They are often already mentioned in the Financial Agreement (Annex II) or Project Proposal (in case of contracts). Most important documents are sector policy documents and Poverty Reduction Strategy papers/PRSP). Be specific about government policies relevant to the project 1.1.2 DOES IT SUPPORT EC DEVELOPMENT AND COOPERATION STRATEGIES IN THE CONCERNED FIELD (SPECIAL CONSIDERATION GIVEN TO CSP/NIP, PARIS DECLARATION, EU EFFECTIVENESS AGENDA)? The Country Strategy Paper and National Indicative Programme are normally available. Check if the project fits clearly in the related period documents. Paris declaration deals with donor’s coordination and is a basic global reference. Check if the project responds broadly to its main commitments in terms of the 12 indicators (see box).

BACKGROUND CONCLUSION SHEET (BCS) (On going projects)

EU effectiveness agenda refers to commitments made in 2005 to increase effectiveness. (http://ec.europa.eu/development/policiesgen_ en.cfm). 12 INDICATORS OF THE PARIS DECLARATION OWNERSHIP 1 Partners have operational development strategies — Number of countries with national development strategies (including PRSs) that have clear strategic priorities linked to a medium-term expenditure framework and reflected in annual budgets. ALIGNMENT 2 Reliable country systems — Number of partner countries that have procurement and public financial management systems that either (a) adhere to broadly accepted good practices or (b) have a reform programme in place to achieve these. 3 Aid flows are aligned on national priorities — Percent of aid flows to the government sector that is reported on partners’ national budgets. 4 Strengthen capacity by co-ordinated support — Percent of donor capacity-development support provided through coordinatedprogrammes consistent with partners’ national development strategies. 5 Use of country public financial management systems — Percent of donors and of aid flows that use public financial management systems in partner countries, which either (a) adhere to broadly accepted good practices or (b) have a reform programme in place to achieve these. Use of country procurement systems — Percent of donors and of aid flows that use partner country procurement systems which either (a) adhere to broadly accepted good practices or (b) have a reform programme in place to achieve these. 6 Strengthen capacity by avoiding parallel implementation structures — Number of parallel project implementation units (PIUs) per country. 7 Aid is more predictable — Percent of aid disbursements released according to agreed schedules in annual or multiyear frameworks. 8 Aid is untied — Percent of bilateral aid that is untied. HARMONI SATION 9 Use of common arrangements or procedures — Percent of aid provided as programme-based approaches. 10 Encourage shared analysis — Percent of (a) field missions and/or (b) country analytic work, including diagnostic reviews that are joint. MANAGING FOR RE SULTS 11 Results-oriented frameworks — Number of countries with transparent and monitorable performance assessment frameworks to assess progress against (a) the national development strategies and (b) sector programmes. MUTUAL ACCOUNTABILITY 12 Mutual accountability — Number of partner countries that undertake mutual assessments of progress in implementing agreed commitments on aid effectiveness including those in this Declaration.

BACKGROUND CONCLUSION SHEET (BCS)

1.1.3 IS THE PROJECT, AS DESIGNED, STILL LIKELY TO ADDRESS THE NEEDS AND BRING RELEVANT BENEFITS TO THE TARGET GROUPS?

Comment whether the target group is well defined and the project purpose responds to their needs as compared to the initial situation when project formulation was made. Be specific and avoid complacency (relevance ratings are often too high because of superficial assessment).

Summary Sub criterion 1.1

20%

1.1.2 - Consistency with CSP/EU agenda: be specific about EU strategies (CSP, cooperation instruments etc); mention if the project as designed supports or contradicts alignment and harmonization.

1.2.1 DOES A LOGFRAME OR SIMILAR TOOL EXIST? IF YES, WHAT IS ITS PRESENT QUALITY (DOES IT CLEARLY SHOW HOW ACTIVITIES WILL ACHIEVE RESULTS AND IMPACT)? IF NOT, WHY NOT? Discuss here the overall quality of the LF to translate project strategy and be used as management tool. Check if there is a clear logic of intervention with the four basic levels (inputs, outputs, outcome, impact) and a clear vertical logic. 1.2.2 ARE THE OO, PP AND RESULTS CLEAR AND LOGICAL, AND DO THEY ADDRESS CLEARLY IDENTIFIED NEEDS?

1.1.1- Consistency with govt. policies: be specific about government policies relevant to the project.

SUB CRITERION 1.2 - AS PRESENTLY DESIGNED, IS THE INTERVENTION LOGIC HOLDING TRUE?

20%

Is the strategy as expressed in LFM, clearly addressing the problems that the project pretends to solve? Are the results likely to bring relevant solutions in terms of capacities etc to achieve the PP? This is where strategy can be discussed from a technical point of view (not only LFM). Check if the levels of impact, outcome and outputs (whatever the terminology) make sense and address the problems of the target group 1.2.3 IS THE PP ACHIEVABLE IN THE PROJECT FRAMEWORK?

1.1.3 - Relevance to solve problems of target group: discuss whether the benefits supposed to be produced should are relevant to contribute to solving the problems addressed.

60%

Check if the expected outcome/project purpose as stated, can be reasonably achieved through the outputs (specific outputs/outcome vertical logic) and within the time frame of the project. he timeframe, given the means put to achieve outputs and results, should be sufficient to achieve the PP. Be specific.

BACKGROUND CONCLUSION SHEET (BCS)

1.2.4 ARE THERE SUITABLE AND INFORMATIVE OVIS/TARGETS FOR EXAMPLE, ARE THEY SPECIFIC, MEASURABLE, ACHIEVABLE, REALISTIC AND TIME-BOUND (SMART)? Discuss OVIs from the point of view of usefulness to monitor the progress towards results/outcomes and PP; in following pages you can be more specific (see 2.3.4, 3.1.2).This is a general appreciation. Do not enter into details but concentrate on the fact that OVIs should allow the project and stakeholders to measure progress towards outcome and impact 1.2.5 ARE THE ACTIVITIES, OUTPUTS AND OUTCOMES PLANNED APPROPRIATE TO ACHIEVE THE PP? This is vertical logic; be specific about issues that can really affect performance. Put some technical content, not only formal discussion of LF, but may be answered in one phrase. 1.2.6 ARE THE RISKS AND ASSUMPTIONS HOLDING TRUE? ARE RISK MANAGEMENT ARRANGEMENTS IN PLACE? Check if unverified risks and assumptions affect the relevance of the intervention logic and if the project has means to respond to identified risks. Be specific about issues. 1.2 7 IS SUSTAINABILITY AN INTEGRAL PART OF THE DESIGN I.E. IS THERE A PHASE OUT/HAND OVER STRATEGY? Check in very general terms if the project is institutionally and socially well-embedded and if the sustainability of outputs and outcomes will require a specific exit strategy, see if it is included in the design. Relates to questions under sustainability: the design should pre-empt many problems. You can develop further details under potential sustainability (5)

Summary Sub Criterion 2.2

1.2.1 - Quality of logframe: Discuss the overall quality of the LF to translate project strategy

25%

1.2.2/1.2.3 - Feasibility: is the strategy as expressed in LF, clearly addressing the problems that the project pretends to solve?

20%

1.2.4 - Quality of OVIs: usefulness to monitor the progress towards results/outcomes and PP

10%

1.2.5 - Technical quality and consistency of project strategy: 30% be specific about issues that can really affect performance. Put some technical content, not only formal discussion of LF

1.2.6 - Quality of assumptions and risk assessment:

7,5%

Be specific about issues.

1.2.7 - Existence of sustainability strategy in design: 7,5% Relates to questions under sustainability (section 5)

BACKGROUND CONCLUSION SHEET (BCS)

SUB CRITERION 1.3 IS THE CURRENT DESIGN SUFFICIENTLY SUPPORTED BY ALL STAKEHOLDERS?

STAKEHOLDERS ARE THE ORGANIZATIONS AND INSTITUTIONS INVOLVED IN PROJECT IMPLEMENTATION

1.3.1 HAVE KEY STAKEHOLDERS BEEN INVOLVED IN THE DESIGN PROCESS? Stakeholders involve institutions, social and civil society organizations, organizations of the beneficiaries. Explain whether the design process started from identified needs and was inclusive.This information can be obtained through interviews 1.3.2 ARE COORDINATION, MANAGEMENT AND FINANCING ARRANGEMENTS CLEARLY DEFINED AND DO THEY SUPPORT INSTITUTIONAL STRENGTHENING AND LOCAL OWNERSHIP? This is a major aspect of design: be specific about the quality in the choice of partners, the definition of their roles, the identification of capacity building needs, the coordination mechanisms. Who manages the funds, what is the flow of resources. Check in the contractual documents and through interviews if the respective roles of the different actors are clearly defined, the flow and management of resources established, and if they give from the start, responsibilities to the stakeholders that will have to ensure sustainability. If the stakeholdersÂ´ capacity was deemed insufficient, is there a capacity building strategy or rather a substitution strategy? 1.3.3 IS THE SUSTAINABILITY STRATEGY (HANDING OVER STRATEGY TO PARTNERS) FULLY UNDERSTOOD BY THE PARTNERS?

Discuss here whether there is a clear definition of the partners who will manage the outcomes of the project, including a capacity building strategy. Check in the contractual documents and through interviews if the partners are aware of their current and future responsibilities 1.3.4 ARE THE OO AND PP CLEARLY UNDERSTOOD BY THE PROJECT PARTNERS? You should assess whether the main stakeholders have sufficient ownership of the strategy and design. Do they know the contractual documents and logframe? Do they use them or at least understand what they are all about? 1.3.5 IS THE TIMESCALE AND/OR RANGE OF ACTIVITIES REALISTIC WITH REGARD TO THE STAKEHOLDERS CAPACITIES? Looking from the capacity building point of view, develop on question 1.2.3. In other words, is it reasonably feasible for the partners to assume the responsibility of the process, outputs and outcomes by the planned end of the project? 1.3.6 IF APPLICABLE: HOW WELL HAS THE PROJECT DESIGN BEEN ADAPTED TO MAKE IT MORE RELEVANT? WAS IT STRAIGHTFORWARD TO DO CONTRACTUALLY? Check if there is a need to adapt the strategy because elements of the context and problematic have changed, or the situation of beneficiaries and stakeholders has been modified. If adaptations were made, did they improve the relevance and quality? Are there administrative obstacles to adapt the strategy?

BACKGROUND CONCLUSION SHEET (BCS)

Summary Sub Criterion 1.3

SUB CRITERION 1.4 - IS THE CURRENT DESIGN SUFFICIENTLY TAKING CROSSCUTTING ISSUES INTO ACCOUNT?

1.3.1 - Stakeholders Participation in design:

Explain whether the design process started from identified needs and was inclusive.

10%

1.3.2 - Quality of implementation arrangements:

Be specific about the quality in the choice 35% of partners, the definition of their roles, the identification of capacity building needs, the coordination mechanisms.

1.3.3/ 1.3.4 - Ownership of design by partners:

Assess whether the main stakeholders have 15% sufficient ownership of the strategy and design.

1.3.5 - Partners capacity needs addressed:

From the capacity building point of view (related to question 1.2.3.)

30%

1.3.6 - Adaptation of design during implementation so far

Check the need to adapt the strategy because the context have changed.

10%

1.4.1 HAVE THE RELEVANT CROSS CUTTING ISSUES (ENVIRONMENT, GENDER, HUMAN RIGHTS AND GOVERNANCE, DONOR COORDINATION OR OTHERS) BEEN ADEQUATELY MAINSTREAMED IN THE PROJECT DESIGN? Indicate here which cross cutting aspects are relevant to the project and whether they have been mainstreamed adequately. There is space to develop under Section 7 of the BCS. At this level just check if those issues are specifically included/mentioned in the project design with corresponding activities and indicators. Summary Sub Criterion 1.4 1.4.1 - Cosideration of relevant cross cutting issues in the design:

cross cutting aspects relevant to the project and whether they are 100% mainstreamed adequately (relate to section 7).

BACKGROUND CONCLUSION SHEET (BCS)

vague ( "improve the OVIs"). Be specific! (What, who, when, how?) Recommendations should be summarized in the MR, however avoid putting too many! Recommendations must be ordered by actor (project management, EC Delegation, etc) and numbered in order of priority.

BACKGROUND CONCLUSION SHEET (BCS)

TEMPLATE BCS –SECTION 1

BACK GROU ND CONC LUSIO N SHEET (BCS) (On going projec ts)

BACKGROUND CONCLUSION SHEET (BCS)

2. EFFICIENCY

(OR LOWER THAN PLANNED), FROM ALL PARTIES INVOLVED?

Efficiency is mainly about concrete aspects of the project: the use of inputs (means) to implement activities that allow delivering concrete outputs.

Refer to the initial budget and latest report. Are all resources available according to budget, and from all sources (EC, partners, and beneficiaries)? Relevant information refers to issues that may arise if actual cost of inputs did not match the budget, or the contrary. If there are aspects of importance explain possible effect on efficiency.

Outputs are the concrete goods and services delivered through the activities of the project. They may be different from what is called “results” in the logframe if those results are not tangible goods and services. In that case, you have to distinguish the concrete outputs (possibly put as indicators) from the outcomes derived from the delivery of these products and services. For instance, for capacity building services, the training of the target group is an output obtained through specific activities (seminars, workshop, on the job training) while the capacities developed by the target group are more of an outcome. The distinction is not always so easy.

SUB CRITERION 2.1 - HOW WELL IS THE AVAILABILITY/USAGE OF MEANS/INPUTS MANAGED? This level is to discuss ONLY INPUTS

2.1.3 ARE INPUTS MONITORED REGULARLY TO ENCOURAGE COSTEFFECTIVE IMPLEMENTATION OF ACTIVITIES? BY WHO ARE THEY MONITORED? Ask for the most recent budget expenditure table and check how the project monitors expenses. Is there an efficient and transparent accounting system? Is the project capable of producing up to date financial situations? 2.1.4 ARE PROJECT RESOURCES MANAGED IN A TRANSPARENT AND ACCOUNTABLE MANNER?

2.1.1 TO WHAT DEGREE ARE INPUTS PROVIDED/AVAILABLE ON TIME TO IMPLEMENT ACTIVITIES FROM ALL PARTIES INVOLVED?

A good indicator is to check for audit reports and tendering of equipment etc. You are not supposed to audit, just have a broad overview whether the budget is managed according to general procedures.

You must refer to availability of the three types of inputs: financial, human and physical (equipments and infrastructure). Did the project encounter delays or difficulties that affected its implementation?

2.1.5 ARE ALL CONTRACTUAL PROCEDURES CLEARLY UNDERSTOOD AND DO THEY FACILITATE THE IMPLEMENTATION OF THE PROJECT?

Give specific information on how financial, HR and other inputs were made available; if there were delays and issues, explain possible effect on efficiency. 2.1.2 TO WHAT DEGREE ARE INPUTS PROVIDED / AVAILABLE AT PLANNED COST BACKGROUND CONCLUSION SHEET (BCS) (On going projects)

Check if the project management has difficulties understanding and applying EC procedures, or adjusting them to other procedures they also have to follow. This is to be checked both at project and ECD/ NAO level. Contractual procedures for tendering and other purposes may often create problems

BACKGROUND CONCLUSION SHEET (BCS)

if project staff is not properly trained; check whether this has impacted on efficiency; be specific.

2.2.2 IS AN ACTIVITY SCHEDULE (OR WORK PLAN) AND RESOURCE SCHEDULE AVAILABLE AND USED BY THE PROJECT MANAGEMENT AND OTHER RELEVANT PARTIES?

Summary Sub Criterion 2.1.

2.1.1/2.1.2 - Availability of financial inputs and human/physical resources: 50%

specific information on how financial, HR and other inputs were made available

2.1.3/2.1.4 - Quality of financial and physical management and control:

Is there an efficient and transparent accounting and financial information system?

30%

2.1.5 - Management of contractual procedures:

Check whether different procedures has impacted on efficiency; be specific.

management tool if: plans, reports, internal monitoring are based on the intervention logic.

20%

Check whether a work plan exists and is actually used for planning and day to day management. An initial plan may exist but not be actually used. Appreciation must take into consideration the complexity of the project. 2.2.3 TO WHAT EXTENT ARE ACTIVITIES IMPLEMENTED AS SCHEDULED? IF THERE ARE DELAYS HOW CAN THEY BE RECTIFIED? In this part you need to be specific. Provide an overview of advancement of main activities as per logframe and planning. Explain specifically where delays occur and why. Delays are significant if they affect the delivery of outputs. 2.2.4 ARE FUNDS COMMITTED AND SPENT IN LINE WITH THE IMPLEMENTATION TIMESCALE? IF NOT, WHY NOT?

SUB CRITERION 2.2 - HOW WELL IS THE IMPLEMENTATION OF ACTIVITIES MANAGED?

This level is to discuss ONLY ACTIVITIES.

2.2.1 IS THE LOGFRAME OR SIMILAR TOOL USED AS A MANAGEMENT TOOL? IF NOT, WHY NOT? The logframe should be used for reporting and day to day management. If not, is there an acceptable substitute? The logframe is used as

Check whether there is a serious gap between what should have been spent at this level of implementation, and what has been really spent. Discuss the implications for the future of the project. Do not enter into too much detail. Compare broadly the % of expenditure with the % of implementation time. Do not forget general commitment requirements (take into account the D+3 rule fixing the deadline for contracting project funds, normally specified in Financing Agreement) and other specifications.

BACKGROUND CONCLUSION SHEET (BCS)

2.2.5 HOW WELL ARE ACTIVITIES MONITORED BY THE PROJECT AND ARE CORRECTIVE MEASURES TAKEN IF REQUIRED? Check in the reporting and with project management. Check how the project management is aware of advancement of activities. If there are delays and issues explain if they are capable of addressing them. 2.2.6 IF APPROPRIATE, HOW FLEXIBLE IS THE PROJECT IN ADAPTING TO CHANGING NEEDS? This questions relates to 1.3.6. If important changes have occurred (and they should have been mentioned in relevance) did the project adapt or not? If not what were the reasons? 2.2.7 IF APPROPRIATE HOW DOES THE PROJECT CO-ORDINATE WITH OTHER SIMILAR INTERVENTIONS TO ENCOURAGE SYNERGY AND AVOID OVERLAPS? Check if possible what other interventions are being implemented in the same sector/ area and ask the project management how they link with them. Ask about coordination measures and mechanisms.

Summary Sub Criterion 2.2. 2.2.1 /2.2.2 - Quality of planning of activities:

Check whether LF and work plans are 10% actually used for planning and day to day management.

2.2.3/2.2.4 - Progress against planning of activities (detailed):

Provide an overview of advancement of main activities as per logframe and 50% planning, specifying where delays occur and why.

2.2.5

Financial execution vs activities:

If there are delays and issues explain if management is capable of addressing 15% them.

2.2.6 - Quality of monitoring of activities and adaptive management:

15%

Relate with question 1.3.6. 2.2.7 - Coordination with other interventions:

Report about coordination mechanisms 10% with similar interventions in the area/sector.

SUB CRITERION 2.3 - HOW WELL ARE OUTPUTS ACHIEVED?

IMPORTANT: before discussing this point make sure of what are the “outputs” of your project: they must be the tangible services, goods, infrastructures, delivered through the activities. 2.3.1 HAVE ALL PLANNED OUTPUTS BEEN DELIVERED TO DATE? AND IN A LOGIC SEQUENCE? Compare the delivery of outputs (services and goods or infrastructure) with the planning. The logic sequence applies when one output is

BACKGROUND CONCLUSION SHEET (BCS)

necessary to deliver another. Be clear about what the expected outputs are! Outputs are concrete products of project activities: goods, services delivered. The LF might be confusing. Give specific information on how they are being achieved

Summary Sub Criterion 2.3.

2.3.2 WHAT IS THE QUALITY OF OUTPUTS TO DATE?

2.3.3 - 2.3.4 - Contribution to results

The quality of outputs may be easy to discuss in some cases but in others is not. Are they of good quality considering existing standards and the needs of the target groups? Please be careful that the discussion over the achievement of OVIs at results (outcomes) level is now in effectiveness. 2.3.3 ARE THE OUTPUTS ACHIEVED LIKELY TO CONTRIBUTE TO THE INTENDED RESULTS? Results (outcomes) are the effects derived from outputs: for instance, increased capacity from training. So that the timeliness, outreach and quality of outputs of course are determinant in the achievement of outcomes. 2.3.4 ARE THEY CORRECTLY REFLECTED THROUGH THE OVIS/TARGETS? These two questions may create lots of confusion with sub criterion 3.1. We suggest limiting the discussion to “physical” coverage of outputs: quantity, outreach, and quality as compared to needs. If delivery of outputs so far seems likely to affect the achievement of results it may be mentioned here but do not discuss qualitative OVIs at results level. You can develop from 1.2 .4 if there are issues with OVIs at output level.

2.3.1. - 2.3.2 - Progress of outputs compared to planned:

Give specific information on how 60% outputs they are being achieved.

(outcomes) and capacity to assess progress:

40%

Describe and comment on the effects derived from outputs.

SUB CRITERION 2.4 - HOW WELL IS THE PARTNER CONTRIBUTION / INVOLVEMENT WORKING? 2.4.1 DO THE INTER-INSTITUTIONAL STRUCTURES E.G. STEERING COMMITTEES, MONITORING SYSTEMS, ALLOW EFFICIENT PROJECT IMPLEMENTATION? Comment on relevant aspects in specific form: is there a steering committee or a similar structure, does it meet and fulfil its supposed role? Review the functioning of these structures (regularity, participation, role in steering or supporting implementation). 2.4.2 HAVE ALL PARTNERS BEEN ABLE TO PROVIDE THEIR FINANCIAL AND/OR HR CONTRIBUTIONS? Is there a partner creating problems by lack of contribution (financial, HR, management)? This may already have been mentioned in 2.1. Mention only significant aspects.

BACKGROUND CONCLUSION SHEET (BCS)

2.4.3. IS THE COMMUNICATION BETWEEN THE PARTNER COUNTRY RESPONSIBLES, THE EC DELEGATION AND THE PROJECT MANAGEMENT SATISFACTORY? Communication refers especially to the decision chain on planning, reporting, funding. There may be issues linked to the lack of communication between partners. Sometimes the communication chain is inefficient. If some issues arise, be specific (as it can be sensitive). When some actors complain about it report it clearly without assigning blame.

Recommendations should address relevant issues identified under the specific questions. It is imperative to have recommendations for issues justifying a "c" or "d" Recommendation must be relevant and consistent with the most important remarks. They must be practical and applicable. Recommendations should have added value: avoid being self evident or too vague ( "improve the OVIs"). Be specific! (What, who, when, how?) Recommendations should be summarized in the MR: avoid putting too many!

Summary Sub Criterion 2.4.

Recommendations must be ordered by actor (project management, EC Delegation, etc) and numbered in order of priority.

2.4.1.- Quality of performance of supervisory/coordination structures:

Comment on relevant aspects (ex.. steering committee and its supposed role)

30%

2.4.2. - Contribution of partners:

Are there problems by lack of contribution?

40%

2.4.3. - Quality of communication/feedback:

Comment on possible lack communication between partners.

KEY LESSONS LEARNED AND/OR ACTION(S) RECOMMENDED, BY WHOM IN ORDER OF PRIORITY

30%

BACKGROUND CONCLUSION SHEET (BCS)

TEMPLATE BCS –SECTION 2

BACK GROU ND CONC LUSIO N SHEET (BCS) (On going projec ts)

BACKGROUND CONCLUSION SHEET (BCS)

3. EFFECTIVENESS Effectiveness describes how the project results (that is, the outcomes or effects of the outputs in terms of benefits to the target groups) have contributed so far to achieve the project purpose. It is very important at this level to have clear understanding of what is meant by “results”. Results are NOT tangible goods and services (those are outputs) but the consequence of the delivery of outputs: they are OUTCOMES, normally defined as what the target group does with the outputs: it is generally expected that outcomes are characterized as benefits or increases in capacities of the target groups enabling them to achieve the intervention purpose. Be sure that you distinguish outputs, outcomes and project purpose whatever the terminology used in the project intervention logic. It is likely that the project has no adequate OVIs for results beyond quantitative targets (already discussed in 2.3). In that case you have to derive a “proxy” indicator for your discussion. For example, if the project intends to deliver an infrastructure as one of its results, and it has only physical indicators, you should introduce a discussion on

ADDRESS ONLY RESULTS OR OUTCOMES (NOT OUTPUTS!). BE CLEAR ABOUT WHAT THE OUTCOMES AND PP ARE. what the target group is likely to be able to do with this infrastructure.

SUB CRITERION 3.1 - HOW WELL IS THE PROJECT ACHIEVING ITS PLANNED RESULTS? 3.1.1 HAVE THE PLANNED RESULTS TO DATE BEEN ACHIEVED? Discuss on the basis of the OVIs at results level (or a “proxy” one) if the outputs have been achieved (or are likely to be achieved) in a manner that guarantees the outcomes. Give

BACKGROUND CONCLUSION SHEET (BCS) (On going projects)

specific information on how outcomes are being achieved; refer to the LF or initial planning. 3.1.2 ARE THE OVIS/TARGETS FOR THE RESULTS/OUTCOMES AND PP APPROPRIATE AND ARE THEY BEING REPORTED AGAINST? Remember that here the focus is on the outcome, not the output. It is very common that OVIs at outcome level are not useful/not monitored; be specific about shortcomings affecting internal monitoring. Also discuss the accuracy of targets (even if they have adequate OVIs). 3.1.3 WHAT IS THE QUALITY OF THE AVAILABLE INTERMEDIARY OUTCOMES? Quality of outcomes needs a technical discussion; be specific about issues. 3.1.4 HAVE ALL PLANNED TARGET GROUPS ACCESS TO / USING PROJECT RESULTS AVAILABLE SO FAR? Outputs may be delivered but target groups cannot use them. This is a good example of difference between output and outcome. The actual

outreach of project compared to what was planned should be assessed; if it is less than planned, or if some target groups are left out, is important to discuss the issue. The reasons for reduced outreach or exclusion of groups should be identified. 3.1.5 ARE THERE ANY FACTORS WHICH PREVENT TARGET GROUPS ACCESSING THE RESULTS/SERVICES? Discuss here any financial, social, political, human factor that may prevent target groups of accessing benefits. If you discover important deficiencies you might review your appreciation of quality of design!

BACKGROUND CONCLUSION SHEET (BCS)

Summary Sub Criterion 3.1.

Progress towards results (outcomes) against targets:

3.1.1 Give specific information on how outcomes are being achieved; 3.1.2 be specific about shortcomings 60% affecting internal monitoring 3.1.3 be specific about quality of outcomes.

Access of target groups to results:

3.1.4-3.1.5 - assess the project’s actual outreach compared to what was 40% planned, explaining any reduction or exclusion of beneficiary groups.

The EC PCM guidelines insist on only one Project Purpose (also called Specific Objective). Now many projects’ intervention logics still use several “specific objectives” but most of the time these are rather results/outcomes. The adjustment may be easy, if the overall objective is actually the project purpose, then the specific objectives may be treated as results. However, more often than not the vertical logic is deficient and there is a mix of results, outputs etc. In this case you have to introduce a “proxy” Project Purpose that may already be there under another terminology, or has to be derived revising the vertical logic. Since this is often difficult to do, and also because there is generally a lack of adequate OVIs or they are not monitored, the BCS does not make the question of achievement of PP straightforward. However, in case you can straighten these things; discuss the achievement of the PP straightforwardly.

3.2.1. TO WHAT EXTENT IS THE PP BEING ACHIEVED SO FAR? WHAT IS THE LIKELINESS OF IT BEING ACHIEVED IN THE PROJECT TIMEFRAME? SUBCRITERION 3.2 - AS PRESENTLY IMPLEMENTED WHAT IS THE LIKELIHOOD OF THE PP TO BE ACHIEVED?

THE PROJECT PURPOSE (sometimes called the Objective or the Specific Objective, if there is only one) is the central development outcome expected to be delivered at the end of the project – more specifically the main expected benefits to the target group(s).

By Project Purpose we understand the central, superior outcome to be achieved by the intervention (in planning terms this must be an answer to the central problem identified). Whatever the terminology used in the project intervention logic, this is the highest-level outcome that the project may pretend to achieve by itself (as distinguished from overall objectives).

Make sure that progress of the project towards achieving the PP is clearly discussed.

3.2.2 TO WHAT EXTENT HAS THE PROJECT ADAPTED OR IS ABLE TO ADAPT TO CHANGING EXTERNAL CONDITIONS (RISKS AND ASSUMPTIONS) IN ORDER TO ENSURE BENEFITS FOR THE TARGET GROUPS? Here refer to assumptions and risks at results level. They are supposed to indicate the main external conditions that would allow the results to ensure achievement of the PP. If the logframe had no explicit assumptions and risks, discuss which are the most likely if relevant.

BACKGROUND CONCLUSION SHEET (BCS)

3.2.3 IF ANY UNPLANNED NEGATIVE EFFECTS ON TARGET GROUPS OCCURRED, OR ARE LIKELY TO OCCUR THROUGH THE PROJECT, TO WHAT EXTENT DID THE PROJECT MANAGEMENT TAKE APPROPRIATE MEASURES? Many projects can have "side effects", unplanned negative or positive effects not initially planned for the intended target group or for other groups. Unplanned negative effects are situations derived from the intervention, that may prevent (some) target groups from using the benefits, or may have negative effects in other aspects. 3.2.4 TO WHAT EXTENT ARE UNPLANNED POSITIVE EFFECTS CONTRIBUTING TO RESULTS PRODUCED / SERVICES PROVIDED? Unplanned positive effects are additional benefits derived from the intervention, that were not foreseen. If they have no bearing on the performance of the project they might be left aside.

Summary Sub Criterion 3.2.

Progress towards project purpose and likeliness of achieving it:

3.2.1. Discuss progress of the project towards achieving the PP.

70%

Adaptive risk management :

3.2.2. Discuss risks and assumptions at results/outcomes level. Incidence of unplanned effects towards achieving the PP (a for positive effect, b for no unplanned effect c or d for negative effect):

3.2.3 -3.2.4. Comment if any "side effects" were registered or mark N/A.

15%

BACKGROUND CONCLUSION SHEET (BCS)

KEY LESSONS LEARNED AND/OR ACTION(S) RECOMMENDED, BY WHOM IN ORDER OF PRIORITY. Recommendations should address relevant issues identified under the specific questions. It is imperative to have recommendations for issues justifying a "c" or "d" Recommendation must be relevant and consistent with the most important remarks. They must be practical and applicable. Recommendations should have added value: avoid being self evident or too vague ( "improve the OVIs"). Be specific! (What, who, when, how?) Recommendations should be summarized in the MR: avoid putting too many! Recommendations must be ordered by actor (project management, EC Delegation, etc) and numbered in order of priority.

BACKGROUND CONCLUSION SHEET (BCS)

TEMPLATE BCS –SECTION 3

BACK GROU ND CONC LUSIO N SHEET (BCS) (On going projec ts)

BACKGROUND CONCLUSION SHEET (BCS)

4. IMPACT PROSPECTS Impact is defined as the contribution of the project to overall objective(s). This contribution may be direct (as planned in the intervention logic) or indirect (not foreseen but due to other factors – such as the activities of other donors, changes in economic factors, new government policy). Impacts are long term changes related to the intervention. The discussion of impacts is generally qualitative in the absence of monitoring of OVIs at that level. It is also generally impossible to distinguish the contribution of the project from some other factors. If the project has no adequate OVIs at overall objective level, you might try to refer to significant OVIs in the national policy or to Millenium Development Goals (MDG).

4.1.3 ARE THE CURRENT OVIS/TARGETS REALISTIC AND ARE THEY LIKELY TO BE MET? This relates to relevance and quality of design (sub criterion 1.2 treated feasibility of PP). If the indicators at OO level are unrealistic, say clearly so because then they are useless to evaluate impact. If they are realistic, discuss them even in qualitative terms. 4.1.4 ARE ANY EXTERNAL FACTORS LIKELY TO JEOPARDISE THE PROJECT’S DIRECT IMPACT? Now this refers to assumptions and risks at PP level. Some assumptions if not met, may impede the contribution to OO.

Summary Sub Criterion 4.1. SUB CRITERION 4.1 - WHAT ARE THE DIRECT IMPACT PROSPECTS OF THE PROJECT AT OVERALL OBJECTIVES LEVEL? 4.1.1 ARE ANY IMPACTS OF THE PROJECT ALREADY APPARENT? Limit the discussion to impacts derived from the achievement of the project purpose and outcomes. They must be at least outlined in the project intervention logic. Apparent means that they can be observed, even if not measured. This is more than often a very qualitative and imprecise discussion; try to be as objective and specific as possible.

4.1.2 WHAT IMPACTS OF THE PROJECT APPEAR ACHIEVABLE? Here discuss if some direct (planned) impacts are to be expected later on, given the current progress of the project, or are probable but not readily apparent.

BACKGROUND CONCLUSION SHEET (BCS) (On going projects)

Actual and likely progress towards OO:

4.1.1 -4.1.2 - try to be as objective and specific as possible, avoiding too qualitative and imprecise discussions.

60%

Adequateness of existing targets at OO level:

4.1.3 - discuss whether the OVIs allow the project to monitor its impacts.

20%

Assessment of risks affecting impact:

4.1.4 - Refer to assumptions/risks at PP 20% level or any non specified external factor.

BACKGROUND CONCLUSION SHEET (BCS)

SUB CRITERION 4.2. - TO WHAT EXTENT DOES /WILL THE PROJECT HAVE ANY INDIRECT POSITIVE AND/OR NEGATIVE IMPACTS? (I.E. SOCIAL, CULTURAL, GENDER AND ECONOMIC) Indirect impacts are impacts that are not a direct result of the project strategy, but occur away from the direct source of impact or as a result of a complex pathway 4.2.2 HAVE THERE BEEN/ WILL THERE BE ANY UNPLANNED POSITIVE IMPACTS ON THE PLANNED TARGET GROUPS OR OTHER NON-TARGETED COMMUNITIES ARISING FROM THE PROJECT? HOW DID THIS AFFECT THE IMPACT? Unplanned positive impacts are distinct from unplanned positive effects at outcome level. They are broad spin-offs from the project impact either to more target groups (then of the same nature as direct impacts) or in other important aspects: for instance, important policy spin-offs achieved thanks to the intervention, but were not specifically planned. 4.2.2 DID THE PROJECT TAKE TIMELY MEASURES FOR MITIGATING THE UNPLANNED NEGATIVE IMPACTS? WHAT WAS THE RESULT? Unplanned negative impacts are distinct from unplanned negative effects at outcome level. To be discussed only if such indirect impacts have been detected or reported. 4.2.3 DOES DONOR COHERENCE, COMPLEMENTARITIES AND COORDINATION EXIST AND HAVE ANY INDIRECT IMPACT ON THE PROJECT? This refers to overall donor coordination beyond the inter-institutional structures of the project. This question is also related to 1.4. In this context it is to be discussed if it has a

relation to potential indirect impacts. If not possible to be discussed specifically then it is not relevant.

Summary Sub Criterion 4.2. Incidence of unplanned effects towards achieving the OO (a for positive effect, b for no unplanned effect c or d for negative effect):

4.2.1 . / 4.2.2 - to be discussed only if 70% such indirect impacts have been detected or reported. Incidence of donor coherence and coordination on project impact:

4.2.3 to be discussed if it has a 30% relation to potential indirect impacts.

KEY LESSONS LEARNED AND/OR ACTION(S) RECOMMENDED, BY WHOM IN ORDER OF PRIORITY Recommendations should address relevant issues identified under the specific questions. It is imperative to have recommendations for issues justifying a "c" or "d" Recommendation must be relevant and consistent with the most important remarks. They must be practical and applicable. Recommendations should have added value: avoid being self evident or too vague ( "improve the OVIs"). Be specific! (What, who, when, how?) Recommendations should be summarized in the MR: avoid putting too many! Recommendations must be ordered by actor (project management, EC Delegation, etc) and numbered in order of priority.

BACKGROUND CONCLUSION SHEET (BCS)

TEMPLATE BCS –SECTION 4

BACK GROU ND CONC LUSIO N SHEET (BCS) (On going projec ts)

BACKGROUND CONCLUSION SHEET (BCS)

5. POTENTIAL SUSTAINABILITY Potential sustainability has been defined as the likelihood of a continuation in the stream of outputs and outcomes/benefits produced by the project after the period of external support has ended. The BCS questionnaire has been streamlined to the four levels that have been found to be significant over the whole range of projects: Financial/economic sustainability;

target groups have to pay for services or benefits (either as fees to an institution or as operational costs for themselves) 5.1.3 CAN THE BENEFITS BE MAINTAINED IF ECONOMIC FACTORS CHANGE (E.G. COMMODITY PRICES, EXCHANGE RATE)? This relates to important assumptions in the economic dimension. Refer to available macroeconomic overview of the country/sector and interviews with stakeholders

Social/institutional sustainability; Policy environment. Capacity building

SUB CRITERION 5.1 - FINANCIAL / ECONOMIC VIABILITY Financial and economic sustainability refers to internal as well as external elements. If it is highly problematic you might want to review your opinion about relevance and quality of design.

5.1.1 IF THE SERVICES/RESULTS HAVE TO BE SUPPORTED INSTITUTIONALLY, ARE FUNDS LIKELY TO BE MADE AVAILABLE? IF SO BY WHOM? Two scenarios exist: either the action will rely on further external financing, or will have to be sustained by other sources (or self sustained). It is important to specify if this has been explicitly planned or not, because “assumed” financial support has not the same value as guaranteed support. 5.1.2 ARE THE SERVICES/RESULTS AFFORDABLE FOR THE TARGET GROUPS AT THE COMPLETION OF PROJECT? The term "target group" has to be taken in a general sense: who will ensure the running costs of the services or goods? In case the BACKGROUND CONCLUSION SHEET (BCS) (On going projects)

5.1.4 ARE THE BENEFICIARIES AND/OR RELEVANT AUTHORITIES/INSTITUTIONS ABLE TO AFFORD MAINTENANCE OR REPLACEMENT OF THE TECHNOLOGIES/SERVICES INTRODUCED BY THE PROJECT? After discussing who will shoulder running costs post project, if any (5.1.2) discuss whether there are financial provisions for maintenance whenever relevant. Sustainability is more probable if relevant institutions already assume a good part of maintenance costs. High cost technologies are likely to be the less sustainable. 5.1.4 IS THERE A FINANCIAL/ ECONOMIC PHASE-OUT STRATEGY; IF SO HOW LIKELY IS IT TO BE IMPLEMENTED? Develop on the general observations made under 1.2.7 There must be specific provisions for transfer and future funding.

BACKGROUND CONCLUSION SHEET (BCS)

Summary Sub Criterion 5.1.

5.2.1 HOW FAR IS THE PROJECT EMBEDDED IN LOCAL STRUCTURES?

Availability of donor/institutional funds after phase out 20%

5.1.1 - identify possible scenarios Financial viability for target group:

5.1.2 - who will ensure the running 20% costs of the services or goods? Economic risks for viability:

5.1.3 - relate to assumptions in the 20% economic dimension. Maintenance and replacement prospects

discuss whether there are financial 20% provisions for maintenance, whenever relevant. Description of economic/financial phase out strategy if any:

5.1.5 - develop on the general 20% observations made under 1.2.7

SUB CRITERION 5.2 - WHAT IS THE LEVEL OF OWNERSHIP OF THE PROJECT BY TARGET GROUPS AND WILL IT CONTINUE AFTER THE END OF EXTERNAL SUPPORT? This level is about social and institutional sustainability at the level of the target groups or final beneficiaries, their organizations and structures. It refers to the basic levels of structures involved in the project (the ones nearest to the beneficiaries). It is likely to be one of the most determinant factors in sustainability and project performance in general. If it is highly problematic you might want to review your opinion about relevance and quality of design.

A project is likely to be more sustainable if implemented through pre-existing, permanent local structures. Independent, ad-hoc structures are the least sustainable. Be specific about target groups: is the project embedded in / linked to their organizational structures. 5.2.2 TO WHAT EXTENT HAVE TARGET GROUPS AND POSSIBLY OTHER RELEVANT INTEREST GROUPS / STAKEHOLDERS BEEN INVOLVED IN THE PLANNING / IMPLEMENTATION PROCESS? This is already a question under 1.3 so that you need to maintain consistency. Now comment about participation in implementation and monitoring. Discuss whether the planning of activities is involving consultations with target groups and interest groups (degrees of participation) 5.2.3 TO WHAT EXTENT ARE RELEVANT TARGET GROUPS ACTIVELY INVOLVED IN DECISION-MAKING CONCERNING PROJECT ORIENTATION AND IMPLEMENTATION? For participation in decision-making target groups should be represented with a real voice in steering committees and other coordination structures. There are different levels of participation from very basic (just providing feedback) through consultative to full participation in decision-making. 5.2.4 WHAT IS THE LIKELIHOOD THAT TARGET GROUPS WILL CONTINUE TO MAKE USE OF RELEVANT RESULTS? This is the final test of relevance. There may be cases where one can doubt that the outcomes will be used after the project closes: explain any doubt.

BACKGROUND CONCLUSION SHEET (BCS)

5.2.5 DO THE TARGET GROUPS HAVE ANY PLANS TO CONTINUE DELIVERING THE STREAM OF BENEFITS AND IF SO ARE THEY LIKELY TO MATERIALISE? This must be linked to initial design. If local structures/institutions have a key role in implementation, then the conditions for sustainability should have been addressed from the start. Discuss if the target groups and their organizations will have a defined role to play after the project. Summary Sub Criterion 5.2. Involvement/ ownership at target groups level: 5.2.1, 5.2.2 , 5.2.3 - be specific about target groups: is the project embedded in / linked to 50% their organizational structures.

Arrangements for the transfer of responsibilities to target groups: 5.2.4, 5.2.5 - Discuss if the target groups and 50% their organizations will have a definite role to play after the project

SUB CRITERION 5.3. - WHAT IS THE LEVEL OF POLICY SUPPORT PROVIDED AND THE DEGREE OF INTERACTION BETWEEN PROJECT AND POLICY LEVEL? This is about “institutional” sustainability. The success of any project depends to a variable extent, on the existence of an “enabling” policy environment. Laws, policies and their enforcement can be positive external conditions

5.3.1 WHAT SUPPORT HAS BEEN PROVIDED FROM THE RELEVANT NATIONAL, SECTORAL AND BUDGETARY POLICIES?

This is not a question about theory (like in relevance) but about practice. It refers to what institutions supposed to enforce policies have done to help or hinder the performance of the project. Are current policies and regulations providing an ENABLING ENVIRONMENT for the sustainability of benefits? 5.3.2 DO CHANGES IN POLICIES AND PRIORITIES AFFECT THE PROJECT AND HOW WELL IS IT ADAPTING IN TERMS OF LONG-TERM NEEDS FOR SUPPORT? Projects heavily dependent on the policy environment should have included assumptions included in their LF . Changes in policies can be so important that they affect the whole relevance or feasibility, in that case review consistency with your observations on relevance. Discuss whether changes in the policy/regulatory environment have or may affect sustainability negatively or positively. 5.3.3 IS ANY PUBLIC AND PRIVATE SECTOR POLICY SUPPORT LIKELY TO CONTINUE AFTER THE PROJECT HAS FINISHED? This is about verifiable, planned or already effective policy support from relevant institutions. Support goes beyond an enabling environment: is there an active institutional involvement? Summary Sub Criterion 5.3. Present and future policy support at national and local levels: 5.3.1., 5.3.2, 5.3.3 – comment about policies providing an enabling environment for the sustainability of benefits and 100% whether changes in the policy/regulatory environment may affect sustainability.

BACKGROUND CONCLUSION SHEET (BCS)

SUB CRITERION 5.4 - HOW WELL IS THE PROJECT CONTRIBUTING TO INSTITUTIONAL AND MANAGEMENT CAPACITY?

5.4.3 WILL ADEQUATE LEVELS OF SUITABLE QUALIFIED HR BE AVAILABLE TO CONTINUE TO DELIVER THE PROJECT’S STREAM OF BENEFITS?

Many projects are all about capacity building, and most have important components geared to increasing capacities of target groups. Institutional and management capacity building are supposed to enable the target groups’ and stakeholders’ structures and institutions to deliver the stream of outputs and outcomes initiated by the intervention.

Discuss whether HR linked to the delivery of project benefits will be maintained in the structures that will continue after the project If the partner structures do not have the necessary or sufficiently trained HR, are these hired and trained through the intervention? Are they likely to remain working in the structures?

The questions here are similar to 5.2 but relate more to formal institutions suppose to provide services to the target groups or ensure governance.

5.4.1 HOW FAR IS THE PROJECT EMBEDDED IN INSTITUTIONAL STRUCTURES THAT ARE LIKELY TO SURVIVE BEYOND THE LIFE OF THE PROJECT? Here the question refers to the management level of the project. Is the project operating within existing relevant institutional structures? This relates also to question 1.3.2. 5.4.2 ARE PROJECT PARTNERS BEING PROPERLY DEVELOPED (TECHNICALLY, FINANCIALLY, AND MANAGERIALLY) FOR CONTINUING TO DELIVER THE PROJECT’S BENEFITS/SERVICES? Project partners include the structures and institutions at different levels, supposed to provide all the services necessary for the continued flow of benefits. Many projects do take institutional capacities for granted and then have to adjust along the way to strengthen the key partners. If it was considered or not in the design, this should have been already mentioned under 1.3.2.

5.4.4 ARE THERE GOOD RELATIONS WITH NEW OR EXISTING INSTITUTIONS AND ARE THERE PLANS TO CONTINUE WITH SOME OR ALL OF THE PROJECT’S ACTIVITIES? If this is not good you might again review your appreciation of relevance and quality of design. This is not supposed to be thought off at the end of the project… Discuss whether the institutional structures that are stakeholders of the project have a phase out /handover strategy Summary Sub Criterion 5.4. Present and future institutional ownership:

5.4.1 - Comment about institutional structures.

relevant 30%

Capacity building of partners for sustainability:

5.4.2, 5.4.3 - discuss whether HR will be 60% maintained in the structures that will continue after the project Networking of partners and institutions for further support:

5.4.4 - discuss whether the institutional 10% structures have a phase out strategy

BACKGROUND CONCLUSION SHEET (BCS)

TEMPLATE BCS –SECTION 5

BACK GRO UND CONC LUSIO N SHEE T (BCS) (On going proje cts)

BACKGROUND CONCLUSION SHEET (BCS)

6. HORIZONTAL ISSUES This part is new in the BCS and we reproduce the indications of the new BCS reader with some additions. This new part must be reviewed in very synthetic form (monitors will not have the time to develop it unless some points are of particular importance for specific project). You should check whether there is any documentation on your project from the QSG process; normally if they are available Eptisa office should provide them. Most projects formulated before EDF 10 did not go through the QSG; nor do NGO grant contracts.

QUALITY SUPPORT GROUP (QSG) AND ROM, LESSONS LEARNT Because of the positive developments in EU DEVCO whereby ROM and the QSG processes are becoming more closely linked, there are areas where the EU would like to be able to collect more data. ROM can provide two types of information: to inform QSG decisions with information on previous experience of similar interventions and also to be able to validate the QSG process over time. To understand more about the QSG process please see extract in box from: http://ec.europa.eu/europeaid/how/ensuringquality/qsg/index_en.htm

WHETHER THE COMMENTS PARTICULARLY REGARDING STAKEHOLDERS AND NEEDS ANALYSIS, INSTITUTIONAL CAPACITY ASSESSMENT OF THE IMPLEMENTING PARTNER AND RISKS AND ASSUMPTIONS MADE IN THE CHECKLIST AND MINUTES AGAINST THE QUALITY AND CONTENT OF THE ACTION FICHE WERE APPROPRIATE / RELEVANT? To address this issue a copy of the QSG Action Fiche and the checklist needs to be requested from the delegation as part of the project

BACKGROUND CONCLUSION SHEET (BCS) (On going projects)

documentation package. Please note that it will not be available for all projects WHETHER THE COMMENTS WERE TAKEN INTO CONSIDERATION AND INCLUDED IN THE PROJECT DOCUMENTS, E.G. TAPS, FINANCING AGREEMENTS, LFM, INCEPTION REPORTS, ETC. AND IF SO, DID THEY IMPROVE PROJECT IMPLEMENTATION? If you manage to get a copy of the QSG Action fiche and check list, see if most relevant issues were addressed in project documents IF THE COMMENTS WERE NOT TAKEN INTO CONSIDERATION, WAS THERE ANY CONSEQUENCE DURING THE IMPLEMENTATION OF THE PROJECT? IF SO, PLEASE DESCRIBE IN THE FREE TEXT BOX. Check whether any comments were relevant in adaptation of the project strategy HAS ANY MONITORING OR EVALUATION (INCLUDING ROM) RESULTED IN IMPROVEMENTS IN THE PROJECT? PLEASE COMMENT This replaces the repetitious reference to previous monitoring in the older version of the BCS and also broadens the issue to cover any other type of internal or external monitoring or evaluation. PLEASE COMMENT ON LESSONS LEARNT, IF ANY. This question monitors may not feel able to comment on. Nevertheless if there is something to draw attention to, be it positive or negative then it should be commented upon here. Using a separate text box simplifies the process of collecting all these comments for future analysis.

BACKGROUND CONCLUSION SHEET (BCS)

Summary QSG involvement QSG is the EC system of quality assurance at identification and formulation stage. Check whether you have QSG information for the project: ID fiche, Action fiche and Checklists You are expected to check whether the important observations and assumptions commented in the QSG process were taken into account.

performing. As the ROM monitors see them in action in every project they monitor they can provide a rich source of information about the role and function of PIUs. In future MR the percentage of the budget allocated to TA will be noted when available.

ROM

To understand the issue and clarify the terminology please see:

Include some relevant comments in the box. If no QSG data are available quote "NO QSG data AVAILABLE

the TA/PIU Strategy and workplan: http://ec.europa.eu/europeaid/what/deliv ering-aid/aid-effectiveness/index_en.htm

This also monitoring.

applies

for

If no previous ROM, quote "NO PREVIOUS ROM DATA AVAILABLE"

REVIEW OF TECHNICAL COOPERATION (TC) QUALITY CRITERIA These questions apply to projects with a specific Technical Cooperation/Technical Assistance component. You will need at least the ToR of the TC/TA contract. You are supposed to give a general overview of the issue, not to get into details. The role of PIUs (Project Implementation Units) is under review and Unit E5 has presented a strategy to “Reform Technical Cooperation and Project Implementation Units” in the light of both the European Court Of Auditor’s 2007 Report on the Performance of EC Technical Assistance for Capacity Development and the EC’s commitment to the Paris Declaration on aid delivery. Specifically the intention is to make the PIUs far less eurocentric in order to comply with EU Target 2 “provide all capacity building through coordinated programmes with an increasing use of multi-donor arrangements” and EU Target 3 “avoid the establishment of new PIU’s altogether”. Thus there is an interest in collecting information on how PIUs are

ParisDeclaration http://www.oecd.org/document/18/0,334 3,en_2649_3236398_35401554_1_1_1_1, 00.html Extracts are included in the following boxes.

DEMAND DRIVEN TC/TA AND OWNERSHIP: (1) DO LOCAL PARTNERS EFFECTIVELY LEAD IN THE PLANNING AND MANAGEMENT OF TC SUPPORT (I.E. BEYOND FORMAL ENDORSEMENT)? (2) DO LOCAL PARTNERS PROVIDE THE REQUIRED INPUTS ENABLING TC TO PERFORM (E.G. HUMAN AND/OR PHYSICAL INPUTS)? See if local partners are in the driver’s seat or the project is TC driven; check if TC receives the inputs allowing them to operate. ADAPTATION TO THE CONTEXT AND EXISTING CAPACITY: (1) IS THE TC SUPPORT LIKELY TO ACHIEVE ITS OBJECTIVES WITHOUT CRITICAL CONSTRAINTS? (2) IS THE TC SUPPORT ADEQUATE VIS-À-VIS THE CAPACITY OF THE LOCAL PARTNER? These are self explaining questions; Comment on the degree of integration of the TC/TA and arrangements for sustainability.

BACKGROUND CONCLUSION SHEET (BCS)

HARMONISATION OF TC: (1) IS THE TC SUPPORT TAKING INTO ACCOUNT TC FROM OTHER DONORS IN THE SAME SECTOR? (ASK THE EC DELEGATION TO ACQUIRE THIS DATA). (2) IS THE PRESENT TC SUPPORT FROM THE EC DELIVERED JOINTLY WITH OTHER DONORS? (IF NO OTHER DONORS ARE ACTIVE IN THE SECTOR THEN MARK N/A). (3) IF THE TC SUPPORT FROM THE EC IS NOT HARMONISED WITH OTHER DONORS, ARE RELEVANT STEPS BEING TAKEN FOR A MORE HARMONIZED TC IN THE FUTURE? THESE QUESTIONS ARE ONLY RELEVANT IF OTHER DONORS ARE ENGAGED IN THE SAME SECTOR/AREA.RESULT ORIENTED TC/TA: (1) ARE THE TC DELIVERABLES (OUTPUTS AND OUTCOMES) CLEARLY SPECIFIED (E.G. IN THE TA TOR) AND REGULARLY ASSESSED (E.G. THROUGH A JOINT PERFORMANCE DIALOGUE/ASSESSMENT)? (2) ARE THERE INNOVATIVE FORMS OF TC SUPPORT BEING USED (E.G. PEER REVIEW, SOUTH/SOUTH AND/OR PUBLIC/PUBLIC COOPERATION, TWINING, NETWORKING, ETC.)?

PLEASE COMMENT ON LESSONS LEARNT, IF ANY.

Without ToR of the TC you will not be able to answer this.

As the ROM monitors see them in action in every project they monitor, they can provide a rich source of information about the role and function of PIUs. In the MR the percentage of the budget allocated to TA will be noted if available.

PROJECT IMPLEMENTATION ARRANGEMENT (PIA): (1) ARE PARTNER COUNTRY STAFF EFFECTIVELY MANAGING THE PROGRAMME (AND NOT TA)? (2) DO THE EXPERTS TAKE INSTRUCTIONS FROM AND REPORT TO THE PARTNER (AND NOT THE EC)? See the definition of a parallel PIA in the reltated box (from Reforming Technical Cooperation and Project Implementation Units for External Aid provided by the European Commission: A Backbone Strategy JULY 2008)

As under QSG this element does not have a yes, no or n/a.

Summary Technical Cooperation quality These questions apply to projects with a specific Technical Cooperation/Technical Assistance component. You would need at least the ToR of the TC/TA contract. You are supposed to give a general overview of the issue, not to go into details . The role of PIUs is under review and Unit E5 has presented a strategy to “Reform Technical Cooperation and Project Implementation Units” in the light of both the European Court Of Auditor’s 2007 Report on the Performance of EC Technical Assistance for Capacity Development and the EC’s commitment to the Paris Declaration on aid delivery.

Specifically the intention is to make the PIUs far less euro-centric in order to comply with EU Target 2 “provide all capacity building through coordinated programmes with an increasing use of multi-donor arrangements” and EU Target 3 “avoid the establishment of new PIU’s altogether”. Thus there is an interest in collecting information on how PIUs are performing.

Parallel PIU: A PIU is parallel when it is created and operates outside existing country institutional and administrative structures at the behest of a donor. In practice, there is a continuum between parallel and integrated PIUs.

BACKGROUND CONCLUSION SHEET (BCS)

Assessing the quality of external assistance

Office Quality Support Groups (oQSGs) perform peer reviews during the identification and formulation of external assistance measures managed by EuropeAid. They are part of the broader quality-assurance mechanism through which the European Commission improves the quality of development co-operation policies and practices. The Commission reformed the management of external assistance, via its Communication of 16 May 2000, to improve radically the speed, quality and profile of EU external assistance. Particular focus was placed on the ex-ante aspect (programming, identification and formulation) as well as on outputs, impact and outcomes of external aid measures. Against this background, EuropeAid put in place its oQSG mechanism to help improve the design of external aid measures at the identification and formulation stage, building on in-house expertise, as well as on best practice from previous and ongoing measures. Focus of work The oQSGs are involved in the preparatory stages of the project cycle, when external aid measures are designed. Good preparation is vital if a planned measure is to be implemented successfully. The primary focus of these concerted bodies is to support the capacity of task managers to identify and formulate highquality measures; undertake systematic assessments of the design quality; identify improvements that need to be made. They also seek to ensure reporting on and dissemination of conclusions and recommendations, transfer of good practices and statistical processing of the results of the oQSG-proceedings. The functioning and logistics of oQSG meetings are handled by an independent secretariat in EuropeAid. The core members of each oQSG are EuropeAid staff based in Brussels. They are responsible for the geographical aspects of the action, its quality from a thematic perspective and its conformity with contractual and financial rules and practices. Colleagues responsible for the programming phase from other DirectoratesGeneral (External Relations and Development) and other DGs can also participate in oQSG meetings, as well as colleagues from the European Union delegations to partner countries. oQSGs come into play early in the preparatory process where they can improve quality by providing guidance. They intervene at the identification and formulation stages to assess the quality achieved. With management responsibilities having been devolved to the delegations in partner countries, the oQSGs also create an efficient exchange of information between Brussels and the delegations on the preparation of the planned measures. This provides useful backup for the delegations. Based on the same criteria as the monitoring activities and evaluation programmes, this approach guarantees consistency and transparency throughout the project cycle. How many oQSGs? There are five operational oQSGs, one for each of EuropeAid's four geographical Directorates and another one dealing with measures funded from Thematic Budget Lines. The main role of the oQSG secretariat is to provide the methodological approaches and to promote cross-fertilisation between the geographical and thematic areas. Meetings are chaired either by the director of the geographical directorate concerned or by the director of the thematic programmes directorate. The chairperson takes a decision on each measure reviewed, based on the issues raised and advice provided by other group members and participants. The primary responsibility of each member is to provide useful advice which will help to ensure that the proposed action is of high quality. The oQSGs intervene at two points in the process: at the end of the identification stage, when the oQSGs support quality by giving guidance for further preparatory work at the end of the formulation stage, when the oQSGs assess the quality of the key documents before a financing decision is made In both steps, quality is assessed by means of checklists that are completed by experts before an oQSG meeting. These checklists are based on the OECD Development Assistance Committeeâ&#x20AC;&#x2122;s criteria and help to assess quality in a structured and systematic way. Proposed measures found to meet these requirements at the end of the preparatory process are considered to be of good quality. These checklists take into account the three types of aid approach (project, sector, or budget).

BACKGROUND CONCLUSION SHEET (BCS)

Guiding principles on TC/TA A set of principles will guide the future provision of EC-funded Technical Cooperation, and the design of Project Implementation Arrangements. These principles will apply to all TC operations (notwithstanding the management mode, including decentralized and centralized management). 1. Focus on capacity development – TC is provided with the primary aim of supporting internal country processes to promote capacity development at individual, organisational and countrywide levels. Where relevant, TC can be called upon to play other roles9, such as offering advice, providing support for the implementation and facilitation/preparation of EC cooperation. 2. Demand-led approach where TC is not provided by default – The provision of TC must be based on the demand and requirements of the partner country. Costs and available options should be transparent. Appropriate dialogue and support may be needed in order to enable clear formulation of the demand for TC. 3. Adopting a results-orientation – TC design will ensure that TC inputs/activities are linked to targeted outputs which in turn lead to sustainable development outcomes. Appropriate indicators will be agreed on in advance to monitor the implementation of TC. 4. Country-owned and managed TC process – Country partner ownership is the key underlying principle for the organisation of EC-funded TC. From the identification to the implementation phase, partner countries will be actively involved in the design of PIAs and TC-supported programmes, including the procurement of TC services and the management, review and accounting of TC results. 5. Taking account of country and sector-specific requirements – TC support will build on a thorough understanding of the political, socio-cultural, sectoral and institutional context. Blueprint approaches should be avoided. 6. Working through harmonised and aligned action – TC support will be closely coordinated with other donors and aligned to country strategies and programmes through the increased use of pooling arrangements or other harmonised approaches, such as delegated cooperation. 7. Avoiding the use of parallel PIUs and promoting effective Project Implementation Arrangements – The use of parallel PIUs will be avoided as far as possible in favour of effective implementation arrangements that are fully integrated and accountable to national structures10. 8. Considering different and innovative options for the provision of Technical Cooperation – The design of TC support will consider alternatives to the use of international long- and shortterm consultants. These alternatives include the use of national and regional resources, twinning arrangements and knowledge transfer beyond standard training approaches.

BACKGROUND CONCLUSION SHEET (BCS)

Project Implementation Units When providing development assistance in a country, some donors establish Project Implementation Units. (They are also commonly referred to as project management units, project management consultants, project management offices, project coordination offices, etc.) These are dedicated management units designed to support the implementation and administration of projects or programmes. PIUs typically share the following key features: •

PIUs are TYPICALLY required to perform subsidiary (rather than principal) tasks with regard to the implementation of a project or programme: monitoring and reporting on technical and/or financial progress, accounting, procurement of works, goods and services, drawing-up terms of reference, contract supervision, detailed design or equipment specification.

•

PIUs are often established at the request of a donor following the inception of a project or programme.

•

The staff of PIUs vary considerably in size and composition. Staff size can vary from 1 to as many as 200 but most count less than 10 professional staff. Although a significant number of PIUs make use of government staff, most PIUs rely on staff recruited outside the civil service (e.g. long-term local consultants).

A distinction is made here between a PIU and technical advice provided directly to national administrations. Parallel PIU: A PIU is parallel when it is created and operates outside existing country institutional and administrative structures at the behest of a donor. In practice, there is a continuum between parallel and integrated PIUs. The criteria below have been designed to help donors and partner authorities draw a line within this continuum and identify with greater certainty parallel PIUs. Donors are invited to review all their development activities with a view to determining how many PIUs are parallel. For the purpose of this survey, PIUs are said to be parallel when there are three or more ‘Yes’ to the four questions below (anything less counts as integrated): 1. Are the PIUs accountable to the external funding agencies/donors rather than to the country implementing agencies (ministries, departments, agencies etc)? (Y/N). 2. Are the terms of reference for externally appointed staff determined by the donor (rather than by the country implementing agencies)? (Y/N). 3. Is most of the professional staff appointed by the donor (rather than the country implementing agencies)? (Y/N). 4. Is the salary structure of national staff (including benefits) higher than those of civil service personnel? (Y/N).

BACKGROUND CONCLUSION SHEET (BCS)

EC VISIBILITY EC visibility is often dealt with very superficially during monitoring and visibility is deemed to be good if there is a board announcing that the project is funded by the EC. Although this is a valid observation EC visibility also relates to a wider understanding of what the role and function of the EC is. It is the level of this understanding that this section endeavors to get some idea of.

DOES THE PROJECT CONTRIBUTE TO PROMOTING EC VISIBILITY (E.G. DOES IT COMPLY WITH THE EC GUIDELINES?) PLEASE COMMENT Check that the project has a visibility plan and whether it is reported on. Comment briefly on that and your personal observations. See below for extract of the EC Visibility Guidelines: http://ec.europa.eu/europeaid/work/visibility/ index_en.htm PLEASE COMMENT ON LESSONS LEARNT, IF ANY. As under QSG and TC quality, this element does not have a yes, no or n/a.

Appropriate Visibility is defined by the EC as: In all actions (projects, programmes, and so on) communication activities should be properly planned. Communication should focus on development with the EU as partner and on the achievements and impact of the action, not on administrative or procedural milestones. FOR MORE DETAILS CHECK VISIBILITY GUIDELINES (see next box).

BACKGROUND CONCLUSION SHEET (BCS)

Appropriate Visibility In all actions (projects, programmes, and so on) communication activities should be properly planned. Communication should focus on development with the EU as partner and on the achievements and impact of the action, not on administrative or procedural milestones. In order to maximise the impact of communication efforts: Activities need to be timely Information used must be accurate Activities should be co-ordinated closely with the Commission The right audience(s) should be targeted Messages should interest the target audience(s) Activities should be appropriate in terms of resources spent and expected impact However, there should also be room in any plan to seize a good opportunity. A good communication reflex and the ability to exploit unexpected opportunities to the benefit of the action will often be as important as more formal efforts and may often be free of cost. Where such opportunities arise, they should be exploited. The communication and visibility plan for any action should seek to maximise synergies with the overall visibility strategy of the EC Delegation (or the relevant Commission department) and, in the case of the UN, that of the relevant UN Organisation. Key Audiences and Target Groups In order to communicate effectively, target audiences should be clearly identified in communication and visibility plans. Audiences will include opinion formers and influential figures, as well as those beyond government and media who have a stake in the action, or are affected by it. Contractors, implementing partners and international organisations should focus their communication and visibility plan on audiences in the beneficiary country, who are the ones for whom the impact of the action will be most apparent and most immediately relevant. In as much as the financing is available, and the urgency, nature, and context of the action will allow, international organisations should include in their communication and visibility plan audiences within the EU, specifically opinion-formers and "elite" audiences involved in the field of international co-operation. Communication and Visibility Plans A budgeted communication and visibility plan should be included in the work plan of any EU-funded or cofunded action, highlighting the external communication activities that need to take place at key stages in the life of the action. The level of detail should be commensurate with the nature, extent and cost of the communication activities envisaged. Contractors, implementing partners or international organisations should produce a communication and visibility plan which will have to be agreed with the Commission (Delegation) staff managing the action, in close co-ordination with the Commission (Delegation) staff responsible for communication activities. The visibility matrix is a simple tool that has been designed to offer basic guidance on the main elements of communication activities at various stages of the project cycle.

BACKGROUND CONCLUSION SHEET (BCS)

TEMPLATE BCS –SECTION 6

BACK GRO UND CONC LUSI ON SHEE T (BCS) (On going proje cts)

BACKGROUND CONCLUSION SHEET (BCS)

7. CROSS CUTTING ISSUES This is already briefly considered in sub criterion 1.4. This part gives more details on those issues. Cross cutting issues are not rated. The questions are to be answered by yes, no or n/a. In any case the answer has to be explained.

7.1) HAVE PRACTICAL AND STRATEGIC GENDER INTERESTS BEEN ADEQUATELY CONSIDERED IN THE PROJECT STRATEGY? HAS THE PROJECT BEEN PLANNED ON THE BASIS OF A GENDER-DIFFERENTIATED BENEFICIARIES’ ANALYSIS? This should be apparent in the intervention logic (specific activities and/or targets and indicators). Token reference to gender does not mean that there is gender mainstreaming. TO WHAT EXTENT WILL / COULD THE GENDER SENSITIVE APPROACH LEAD TO AN IMPROVED IMPACT OF THE PROJECT? Is the project targeting groups with important gender aspects? Are there specific gender related needs and issues in the target population? Can unplanned negative effects be expected if gender is not considered? WHAT IS THE LIKELINESS OF INCREASED GENDER EQUALITY BEYOND PROJECT END? Is the project actively promoting gender participation in the process (especially on decision making levels) and to what level?

BACKGROUND CONCLUSION SHEET (BCS) (On going projects)

ACCORDING TO THE OECD GENDER POLICY MARKER HOW WOULD YOU CLASSIFY THIS PROJECT? According to OECD, an activity should be classified as gender equality focused (score Principal or Significant) if it is intended to advance gender equality and women’s empowerment or reduce discrimination and inequalities based on sex.

GENDER MARKER

The gender equality marker, as other markers, can take three values: “0” for not targeted, “1” for significant and “2” for principal. A principal mark is given where the intervention would not have been undertaken without a gender equality objective. a gender mainstreaming project would be marked with gender equality as a principal objective.

BACKGROUND CONCLUSION SHEET (BCS)

Mainstreaming gender equality through the project approach (Extract from EC Toolkit) Highlighting gender equality criteria at project implementation stage : The gender equality issues to be addressed at this stage are summarised below, according to the quality attributes and criteria of the PCM Quality Frame: Assess whether changes in the political, legal, economic or social context of the project have affected the assumptions about gender roles and relationships made at the beginning of the project. Phenomena such as migration, civil unrest or the HIV/AIDS pandemic can all affect existing roles and relationships and may require adjustments to the project. Assess whether project results are being delivered in a gender-equitable manner as was originally planned, and make adjustments if that is not the case. Assess whether information on project achievements/results is disaggregated by sex, and whether action is taken to redress inequalities and shortfalls Assess whether capacity is being built within the project structure and among stakeholders to ensure that gender equality achievements can be maintained after the project ends.

7.2) IS THE PROJECT ENVIRONMENTAL NEEDS?

RESPECTING

• HAVE ENVIRONMENTAL CONSTRAINTS AND OPPORTUNITIES BEEN CONSIDERED ADEQUATELY IN THE PROJECT DESIGN? • ARE GOOD ENVIRONMENTAL PRACTICES FOLLOWED IN PROJECT IMPLEMENTATION (IN RELATION TO USE OF WATER AND ENERGY AND MATERIALS, PRODUCTION OF WASTES, ETC)? DOES THE PROJECT RESPECT TRADITIONAL, SUCCESSFUL ENVIRONMENTAL PRACTICES?

• WHAT CAPACITIES EXIST (WITHIN PROJECT, PROJECT PARTNERS AND PROJECT CONTEXT) TO DEAL WITH CRITICAL RISKS THAT COULD AFFECT PROJECT EFFECTIVENESS SUCH AS CLIMATE RISKS OR RISKS OF NATURAL DISASTERS (IN THE CASE OF PROJECTS IN SENSITIVE GEOGRAPHICAL AREAS / NATURAL DISASTERS HOTSPOTS)? • HAS ENVIRONMENTAL DAMAGE BEEN CAUSED OR LIKELY TO BE CAUSED BY THE PROJECT? WHAT KIND OF ENVIRONMENTAL IMPACT MITIGATION MEASURES HAS BEEN TAKEN? • IS THE ACHIEVEMENT OF PROJECT RESULTS AND OBJECTIVES LIKELY TO GENERATE INCREASED PRESSURE ON FRAGILE ECOSYSTEMS (NATURAL FORESTS, WETLANDS, CORAL REEFS, MANGROVES) AND SCARCE NATURAL RESOURCES (E.G. SURFACE AND GROUNDWATER, TIMBER, SOIL)? Besides these rather obvious, straightforward questions consider general mainstreaming of environment in the project whenever relevant. According to the EU Environmental Integration Handbook the following are the evaluation criteria and related environmental issues: Did the project address the environmental issues identified in the Problem Analysis? Does it incorporate environmental objectives addressing its potential environmental impacts? Have adequate resources or efforts been invested in mitigating/optimising impacts? Did the project comply with its environmental requirements and objectives? What are the project’s contributions to sustainable development? Are project outcomes threatened by environmental degradation (or resource impoverishment) or disasters?

BACKGROUND CONCLUSION SHEET (BCS)

Environmental mainstreaming According to the EU Environmental Integration Handbook the following are the evaluation criteria and related environmental issues Did the project address the environmental issues identified in the Problem Analysis? If not, are there good reasons for giving priority to other problems? Does it incorporate environmental objectives addressing its potential environmental impacts? Has the project made efficient use of environmentally damaging means? Have adequate resources or efforts been invested in mitigating/optimising impacts? Has the project been handicapped by under-estimated environmental constraints? Did the project comply with its environmental requirements and objectives? Did the environmental protection measures produce the expected results? What are the project’s contributions to sustainable development? Could these have been improved? What are the social and environmental effects external to project objectives? What have the impacts been compared to those predicted by the EIA? Are project outcomes threatened by environmental degradation (or resource impoverishment) or disasters?

7.3) HAS (GOOD) GOVERNANCE BEEN MAINSTREAMED IN THE PROJECT? DOES IT TAKE INTO CONSIDERATION THE DIFFERENTIAL IMPACT OF POVERTY BY DISADVANTAGED GROUPS? This is of particular relevance for all economic development projects but not only those.

IS THE PROJECT DESIGNED IN SUCH A WAY THAT IT TAKES INTO ACCOUNT POTENTIAL CONFLICT ? Is there possibility of conflict around the questions of access to the benefits generated by the project? If the project is implemented in a conflict-affected or conflict-prone context is that tackled through the intervention? IS REGULAR, TRANSPARENT, FINANCIAL REPORTING BUILT INTO THE PROJECT? ARE ITS RESULTS WIDELY CIRCULATED AND UNDERSTANDABLE? ARE THERE EFFECTIVE ANTI-CORRUPTION MONITORING TOOLS IN PLACE? Consider the transparency aspects of the project whenever relevant. The draft handbook on promoting good governance in ec development and cooperation indicates the importance of assessing the fulfilment of the guiding principles and the core concerns of good governance (see principles in following Box).

7.4) DOES THE PROJECT ACTIVELY CONTRIBUTE TO THE PROMOTION OF HUMAN RIGHTS? HAS THERE BEEN AN ANALYSIS OF “WINNERS AND LOSERS” REGARDING POSSIBLE “DISCRIMINATION” OF TARGET GROUPS BY THE PROGRAMME/PROJECT? WILL THE PROJECT HELP TO ENSURE RESPECT FOR ANY RELEVANT HUMAN RIGHTS AND NOT CAUSE THEM TO BE REDUCED IN ANY WAY? DO ANY INTERESTED PARTIES AND OBSERVERS RAISE THEIR CONCERNS?

BACKGROUND CONCLUSION SHEET (BCS)

Good Governance checklist 1. Participation and ownership • Have all relevant stakeholders been identified and given the opportunity to participate? • Has there been a balanced and representative participation of stakeholders? • Have the costs of participation for stakeholders been evaluated? • Have expectations and concerns, voiced in consultation with the stakeholders, been taken into consideration in the project? • Did the relevant stakeholders perceive the project as important? 2. Equity • Did the project provide net gains to poor people that responded to their priority needs? • Did the project enhance strategic and immediate gender equity in the country? • Were the benefits of the project free from discrimination? 3. Organisational capacity • Was the mandate of the concerned organisations compatible with the role expected from them in the project? Did they have the capacities (financial, human, structural…) to undertake the specific tasks in question and Guarantee responsibility for them? • Were necessary internal and external incentives in place for them to be able to play their role adequately? 4. Transparency and accountability • Was there a flow of information (in an adequate format) to the relevant constituencies about the project? • was there clarity in the project proposal regarding responsibilities, roles and organisation among the stakeholders? • Were targets and results of the project clearly identified?? • Did those dealing with resources have well-defined and consequent obligations (e.g. Reporting on the use of those resources)? Were they required to demonstrate fulfilment of those obligations and did they face sanctions for failure? 5. Anti-corruption Were all partners committed to implementing the project with no corruption as a goal? Was there enough information On their identity, resources, practices, and track record to give credibility to this intention? Were there effective anti-corruption monitoring tools in place? Was regular, transparent, financial reporting built into the project? Are the results widely circulated and understandable? 6. Conflict prevention and security • Was the project designed in such a way that it took into account conflict potential? • were the project’s benefits delivered in such a way so as not to be appropriated by any existing parties in conflict? • did the project help increase rival groups’ confidence through more openness and communication and through the encouragement of balanced, local ownership of the project?

BACKGROUND CONCLUSION SHEET (BCS)

8. LIST OF PERSONS INTERVIEWED /DOCUMENTS ANALYSED The purpose of this sheet is to have a record of the main persons interviewed and documents analysed in order to substantiate the monitorâ&#x20AC;&#x2122;s findings should the need occur. As in the field meetings may often attract the whole village or all the women in a micro credit scheme than it is not appropriate to list everyone but just the type of beneficiary group. In cases were comments are off the record or individuals do not wish to be indentified the monitor should decide himself/herself whether to and how to reflect the contact in this sheet. Personal contacts (telephone numbers/ e-mail addresses should be included whenever possible.

BACKGROUND CONCLUSION SHEET (BCS) (On going projects)

BACKGROUND CONCLUSION SHEET (BCS)

TEMPLATE BCS –SECTION 7

BACK GROU ND CONC LUSIO N SHEE T (BCS) (On going proje cts)

BACKGROUND CONCLUSION SHEET (BCS)

100

DE-BRIEFING FICHES

THE OBSERVATIONS IN THE DEBRIEFING FICHE MUST BE SHORT AND CLEAR

DE-BRIEFING FICHES

WHY DE-BRIEFING FICHES

WHEN TO DO IT

De-briefing is an important stage of the mission. It has several objectives: To inform the Delegation implementation of the mission

the

To share with the Delegation the provisional observations of the monitors To check factual accuracy of observations To correct information.

any

important

101

gap

At Eptisa we have observed that well structured presentations at de-briefing allow for better feedback from the Delegation, and increase the credibility of the monitoring exercise. We have developed the simplest fiche for collecting preliminary conclusions from each monitor before final de-briefing to allow for a synthetic presentation with a minimum input.

De-briefing fiches have to be ready for each project the day before de-briefing. It is the only product that we expect each monitor to deliver on that date, whatever the logistic difficulties. It is best sent by e-mail to the mission leader as soon as possible. The mission leader is then responsible for collating all de-briefing notes in the PowerPoint presentation. This is done by copying and pasting the individual Excel tables into the PowerPoint format (please always seize the whole table and use regular paste to avoid formatting problems).

HOW TO DO IT The debriefing fiche template is a simple table in a Power Point format. The information to be introduced includes the project’s name, the scores, a sentence explaining your (preliminary) observations and the main (preliminary) recommendations.

In this way you just have to cut and paste either way. Just write “a, b, c or d” in the corresponding cell of the table.

Navigate TROUGH the manual

DE-BRIEFING FICHES

The best way to do it without investing extra time is to write, before the debriefing, the introductory sentence explaining the score for each of the 5 ROM criteria, that might later be used in your MR. You should be able to summarize your appreciation for each criteria in one sentence.

DE-BRIEFING FICHES

DEBRIEFING FICHE

Project:

ROM criteria

Type of ROM

Preliminary observations

(ongoing, ex post, SPSP)

Preliminary rating

Relevance and quality of design

Efficiency

Effectiveness

Impact

Sustainability

Main recommendations

102

MONITORING REPORT (MR)

WHAT IS THE MONITORING REPORT The Monitoring report is the main product presenting the observations of the mission. Monitors should note that Monitoring Reports are accessible to all EuropeAid, DG Development & RELEX and Delegation staff via the CRIS database. Whilst National Authorities and PMUs do not have direct access to this database, the Delegation concerned is encouraged to include them in their distribution list of the Monitoring Report.

WHEN TO PREPARE THE MONITORING REPORT After the Background Conclusion Sheets, the next step is to complete the Monitoring Report according to the Template. Templates for ongoing and ex post monitoring are now the same because all MR are uploaded in the same format in CRIS. It is advisable to start drafting some elements as soon as possible during the mission. The draft MR has to be sent to the Brussels office within the timeframe indicated on the contract. Remember that final reports have to be uploaded into CRIS 10 working days after the mission ended.

THE DIFFERENT SECTIONS OF THE MONITORING REPORT The MR consists of a front page table and normally 2 pages of descriptive text. The front page contains the projectâ&#x20AC;&#x2122;s basic data, financial data and the grading. The text contains the summary and conclusions of the monitoring exercise. The text should occupy approximately 2 full pages (Arial 10, simple space). THE SUMMARY AND CONCLUSION PART OF THE MR IS DIVIDED IN 6 SECTIONS, ONE PER EACH ROM PARAMETER AND ONE FOR RECOMMENDATIONS. INSTRUCTIONS FOR FILLING EACH SECTION ARE ACCESSIBLE THROUGH THE FOLLOWING NAVIGATOR.

Navigate TROUGH the manual

NAVIGATING THROUGH THE MR INSTRUCTIONS (use the buttons below to navigate through the different sections)

INTERVENTION AND FINANCIAL DATA

RELEVANCE AND QUALITY OF DESIGN

EFFICIENCY OF IMPLEMENTATION TO DATE EFFECTIVENESS TO DATE

IMPACT PROSPECTS

POTENTIAL SUSTAINABILITY

KEY OBSERVATIONS AND RECOMMENDATIONS (BY WHOM AND IN ORDER OF PRIORITY)

MONITORING REPORT

MONITORING REPORT (MR)

103

MONITORING REPORT (MR)

Added value in Monitoring Reports ROM fulfills a double role, which is providing useful data for EU headquarters in an aggregate way, and facilitating to Delegations and project partners an external vision of performance and useful recommendations. Some skepticism about the usefulness of ROM is sometime expressed by users, most commonly in Delegations. We have come to the conclusion that such perceptions are linked to a lack of added value in many Monitoring Reports. The main features justifying this perception are: Absence of any new information and remarks on the project; Reports focusing on formal aspects and poor in technical content; Lack of pertinent recommendations. While experience is of course a determinant factor in reporting quality, there are also some methodological clues that may ensure putting more added value in the reports. Here is what we consider major clues to ensure added value in Monitoring Reports: Write reports readable to all readers: PCM jargon and packing too much information, obscuring the main features of the report, are common issues. Style is important! â&#x20AC;&#x153;What is conceived well is stated clearly ..... And the words to say it come easily!â&#x20AC;? (Nicolas Boileau, 1674). Be as specific as possible on your discussion of the main aspects of project quality and performance, and introduce in the BCS the most relevant facts and observations, avoiding too general phrasing and above all, NEVER repeat the questions of the BCS in the text or include the full phrasing o project objectives etc, unless absolutely necessary. A report mostly composed of general concepts and PCM jargon repeats what everybody knows and is unlikely to have any added value for anybody. Discriminate what are the most relevant features explaining performance: those which justify mostly the DAC criteria scoring, especially extreme scores (a, c and d) are those which must be discussed clearly and justify recommendations. Your observations and conclusions on those issues must be carefully checked. Limit yourself to the most important and pertinent recommendations: a report with too many recommendations is unlikely to have much added value in most cases, especially if those are phrased in very general terms. The Task Manager is supposed to give follow up to the recommendations only if found pertinent. Our Quality Control will look carefully at the added value in your reports; we know by experience that is a major quality factor influencing the feedback of the Delegations.

104

MONITORING REPORT (MR)

INTERVENTION/FINANCIAL DATA

programmes indicate Pacific regional or Caribbean regional or any other region which is appropriate.

All the data are available in the CRIS fiche and the FA and its riders. You must introduce the data in the table heading the MR template. These data are those available on CRIS before the mission. You are supposed to check their validity during the mission, preferably during briefing with the project task manager. Change any data when needed.

Delegation Responsible: this is the task manager in charge of the project at the respective EU Delegation.

The Intervention data in the MR must be consistent with those in the PS. The following instructions must be applied to fill in the information. Monitoring report type: either ongoing or ex post Project type: type either national or regional) CRIS Decision Number: this is the CRIS number found in CRIS fiche. This cell must be filled only for projects financed with EDF funds and regulated by Financing Agreements. CRIS Contract Number: this number appears on the CRIS fiche for all other type of projects which are regulated by Grand Contracts or Contribution Agreements (see page 37 for definitions). Project Title is the official name appearing in the financing agreement/contract. DAC-CRS Sector (CRIS): this is the DAC code normally indicated in the CRIS fiche. Additional DAC-CRS code (compulsory): if you consider that the DAC code in CRIS does not express correctly the sector or subsectors covered by the project, select an additional code in the DAC codes list (at the end of this manual). Otherwise just repeat the official one from the cell above. Date Financing Agreement/contract signed: this is the latest date that the Financing Agreement/contract signed by one of the parties. NOT the decision date! Geographical Zone (Country): country name for country based projects and for country based components of regional projects; for regional

MONIT ORING

HQ responsible : in case the project is still centrally managed through Brussels indicate the TM name. Monitor: that is you. Project Authority (../implementation agency) : the name of the agency officially in charge of project implementation according to Financing Agreement or contract (for EDF projects do not put the NAO but the technical implementation agency). Type of Implementing Partner (only code): categories are / 0. not known/ 1. International NGO, CSOs or Universities (European or international) / 2. Local NGOs, CSOs, Universities/ 3. Profit oriented organizations/ 4. International government organizations (non UN)/ 5. UN agencies including WB/ 6. Partner countries public administration/ 7. European countries public administration/ 8. European governmental organizations, EU agencies / 9. regional bodies (CARICOM etc) / 10. mix of different partners . Start date – planned: this is the starting date stipulated in the FA/contract (if not specified otherwise, starts with the date of the last signature on the FA). End date-planned: this is the date of the end of the operational phase as stated in the Financing Agreement or contract (see Art. 4 of specific conditions and do not put the final date for execution which is usually two years later for EDF projects); if there is an approved rider modifying this date use this new date. Start date – actual: this is the actual starting date of the operational phase. Always refer to official dates agreed to written by the parties. End date – likely: this date may be different from the planned one only if a rider is being processed or likely to be approved. Monitoring visit date from: indicate from briefing date to debriefing date.

105

MONITORING REPORT (MR)

The financial data in the MR must be consistent with those in the PS. There are more data to fill in the MR template. The following instructions must be applied to fill in the information. Financial data are also provided in the Excel datasheet sent before the mission. ALL FINANCIAL DATA IN MR MUST BE IN PLAIN NUMBERS, NO DOTS, NO COMMAS, NO DECIMALS Primary commitment (EC financing): the total amount of the EC contribution as per Financing Agreement or Contract or modified by rider, in €. Budget allocated to Technical Assistance: in case there is a specific TA contract or TA component in the Project budget, in €. Secondary commitment (funds contracted of EC contribution): as per latest CRIS data, normally indicated as contracted. For contracts primary and secondary commitments are the same. Other funding (government and/or other donors): contributions from other sources, in €. This information is normally indicated in the CRIS fiche or in the FA/Contract. Total budget of operation: primary + other fundings in €. Total EC funds disbursed: as per latest CRIS data, in € (see column “paid”) Financial data as at : date of latest CRIS data. If you do not have access to a recent CRIS fiche or data are not available, ask the EU Delegation (task manager).

THE GRADES IN THE MR The grades for the 5 ROM criteria MUST IN ALL CASES CORRESPOND TO THE GRADES OF THE BCS. Check well the consistency between BCS and MR.

SUMMARY OF CONCLUSIONS As mentioned in ROM manual, the monitor must justify his conclusions in section “Summary of

Conclusions” of the MR. The text of the MR is the most important product of the ROM exercise because this is the report which will be read by the wider public. Comments should be accurate, concise and direct. Conclusions and grading will be derived from the Background Conclusion Sheet by summarizing the most crucial aspects. The MR must provide the reader with a complete and selfsustaining explanation of monitor’s opinion about the project at the time of the ROM visit. Each criterion should be commented upon separately in the space allocated in the MR template. Style and content - The text should summarise the main findings under each criterion and not only copy sections of the BCS. The text must be readable and readily understandable by persons not familiar with the project. At the same time it must avoid telling stakeholders what they already know. Think of “added value” when you redact it. Start each criterion with a sentence summarizing your appreciation and clearly justifying the scoring. This sentence will be used for the de briefing note also. Then summarize the most relevant observations already made in the BCS. Avoid being descriptive (description of the project is in the PS), be precise and factual. How to avoid unnecessary misunderstandings The ROM exercise is an external exercise and monitors are expected to give their independent views based on the objective use of the methodology. In any case, when the situation is not optimal, remarks are easily taken as criticisms and provoke defensive reactions from stakeholders. Monitors are not expected to be complacent. However, a good rule of thumb when analyzing delicate situations is to stick to facts and their consequences on the project performance without specifically blaming any stakeholder: if your observations are clear and sound, the authorities responsible for taking corrective measures will not need finger pointing to know where problems come from. Acronyms - Explain all acronyms in the MR the first time you use them in the text. Avoid using too many acronyms that lead to unintelligible sentences like “the OVIs of the PP are verified through SoVs and should ensure the OO”. Avoid “development jargon” as much as possible.

106

MONITORING REPORT (MR)

RELEVANCE DESIGN

AND

QUALITY

Start with a well formulated sentence summarizing your assessment. Then justify it and point out the most relevant issues.

DO NOT write here what the project objectives etc are. This is supposed to be in the PS. Do not dwell on less significant details of the logframe. Do not be too superficial on relevance.

What should the â&#x20AC;&#x153;Relevance and quality of designâ&#x20AC;? section include (whenever relevant to performance)? ROM context relate the ROM exercise to the nature of the operation monitored (decision, grant contact, service contract) and comment if any, relate the ROM exercise to phase of implementation (early, middle, late, ex-post) and comment if any. Identification aspects refer to the antecedents (previous related operations) and how they influenced project identification and design, refer to the process and method of project preparation (participatory /not participatory), appreciate the quality of the reference documents (feasibility study or similar), briefly describe the main problem that the project pretends to solve, assess if the problematic was (correctly/incorrectly) identified at the time of identification, assess if the chosen beneficiary population (qualitative, quantitative) and geographic location were adequate. Policy reference refer to the local or national policy (or programme) in which the project is situated, assess the level of priority for the country, refer to the programming/implementation instrument of the EU, assess the level of priority within the EU strategy for the country/region/thematic area. Strategy and LF analyze if the proposed solutions were well/badly designed, analyze if the LF was suitably/unsuitably formulated, in the above analysis refer especially to: intervention logic as a whole, quality of the PP and respective OVI coherence of each result and respective OVI coherence of activities and adequateness of the related inputs quality of the risk analysis and assumptions, describe the main deficiencies of the LF (if any) and suggest improvements. Other aspects of project design assess the institutional set up and implementation organization, assess the adequateness of the implementation method and comment on any eventual obstacle detected, that reduces projectâ&#x20AC;&#x2122;s capacity to achieve the PP, appreciate the adequateness of the project calendar or existing issues in this aspect. Project evolution along the time assess if the problematic originally identified persists/was modified, assess if the logic of the project remains valid/has changed, analyze if the project has modified/not modified its logic to adapt to (possible) changes in context, describe which aspects of the LF have been changed or need to be changed.

MONIT ORING

107

MONITORING REPORT (MR)

EFFICIENCY OF IMPLEMENTATION TO DATE

Do not dwell on details of implementation that are not likely to have a significant bearing on efficiency.

Start with a well formulated sentence summarizing your assessment. Then proceed to justify it and point to the most relevant issues.

What should the “Efficiency” section include (whenever relevant to performance)? Overall progress report on the advances (%) of the project with respect to the original calendar, assess the overall rate of execution to date, with respect to the global programming, comment on the possible causes of the observed delays (if any), asses if the calendar represents/doesn’t represent an obstacle for the PP achievement and, in the affirmative, analyze the possible consequences/measures to adopt. Inputs provision assess the punctuality in different inputs contributions, comment on the quality of inputs provided so far, assess the accomplishment of beneficiary’s commitment (… %), financial or in kind, comment on the problems or positive aspects detected in relation to beneficiary’s contribution, comment on the possible corrective measures adopted to overcome the previous problems (are they sufficient/insufficient?). Activities execution comment if the global activity programming is adequate/not adequate, assess if activities are being realised in time/not in time, with adequate/not adequate quality, comment on possible problems and causes of deviations from the original planning of activities, define if a reprogramming of activities is needed. Outputs production assess the advance (significant, moderate, no advance…) with respect to each output’s production, report on the % of achievement in relation to each of the respective IOVs, define qualitatively the observed outputs (highly, moderately, little satisfactory). Organization, management, coordination asses if the established organization is adequate/not adequate to manage the project, comment on the capacity of the main partners to contribute to produce results, describe aspects to modify/improve it (if any), asses the quality of the cooperation between the Delegation and the beneficiary institution.

MONIT ORING

108

MONITORING REPORT (MR)

EFFECTIVENESS TO DATE Start with a well formulated sentence resuming your assessment. Then justify it and point out the most relevant issues.

Be sure you are talking about effectiveness (outcomes and project purpose).

What should the â&#x20AC;&#x153;Effectivenessâ&#x20AC;? section include (whenever relevant to performance)?

Outcomes achievement describe the main results produced up-to-date, describe the beneficiary groups of these results (type and quantity), indicate if the results observed are in line or not with the planned ones, explain the causes of the deviations, if any, with respect to plans , indicate if the quality of the services provided so far is adequate/not adequate and if they meet the expectations of the planned beneficiaries, based on the elements observable at the time, analyze if the planned final benefits will be obtained within the timeframe of the project, indicate if corrective measures are needed to assure that at the end of the project the beneficiaries will obtain the planned final benefits, indicate if the different groups who compose the planned target have obtained, thanks to the project, the improvements indicated with respect to the deficiencies/problems originally identified , analyze if these improvements are adequate or not adequate with respect to those planned in the design, indicate if the beneficiaries are actively involved in the management of these services , PP Achievement analyze if within the terms of the Project the predicted effects (OVIs of the PP) will be obtained and they correspond to those originally anticipated, indicate what objective (OVI) is reasonably attainable, indicate what are the main factors that prevent the reaching of the objective and if these are related to the main risk factors, analyze if in this respect the Project is or isnâ&#x20AC;&#x2122;t realizing the necessary corrective measures. Unplanned effects if applicable, describe the undesired effects that affect the achievement of the PP, explain if these negative effects could be corrected and if the necessary corrective measures are in place, describe if the initiatives taken to date and the corrective effects seem sufficient/insufficient or if additional measures are necessary, describe the unplanned positive effects (if any), the additional benefits expected for the foreseeable beneficiaries. Indicate if it is anticipated that the OVI of the PP will be increased and how, indicate if the obstacles that, according to the previous monitoring, prevented the achievement of the PP have been adequately addressed in the meanwhile and if the adopted measures have managed to overcome these obstacles.

MONIT ORING

109

MONITORING REPORT (MR)

IMPACT PROSPECTS Start with a well formulated sentence summarizing your appreciation. Then justify it and point out the most relevant issues. Be sure you are talking about impacts (overall objectives).

Define clearly the actual situatin as to the probability of achieving positive impacts. Do not write lots of unqualified opinions: if impact is too difficult to discuss at this stage, just state it.

What should the “Impact” section include (whenever relevant to performance)? OO achievement indicate if the perspectives that the partial or total achievement of the specific objective contribute to produce the overall objective is high/moderate/low, indicate if, with view of generating a greater impact, the Project would need to take additional measures that could help it to increase its contribution to the overall objective. Hypothesis verification indicate if the other factors (main hypothesis identified in the design) are on the way or far from being realized, mention if the main hypothesis are or aren’t being verified and mention the respective favorable or unfavorable causes. Coordination and synergies mention if coordination/synergies with other actions (mention which ones) in the same sector (national, international, EU member states) are sufficient or insufficient . Obstacles and negative impacts mention possible negative impacts (if any) to avoid and if some corrective measures are needed, indicate if the obstacles that according to previous monitoring/evaluations were limiting project’s contribution to an identified wider impact have/have not been corrected in the meanwhile and if the corrective actions have been sufficient/insufficient.

MONIT ORING

110

MONITORING REPORT (MR)

POTENTIAL SUSTAINABILITY Start with a well redacted sentence resuming your appreciation. Then proceed to justify it and point to the most relevant issues.

Concentrate on the most important issues of institutional, economic and social sustainability, and do not forget that problems with sustainability may come at least in good part from inadequate design.

What should the “Sustainability” section include (whenever relevant to performance)? Financial sustainability indicate if the services introduced/improved with the help of the project, and which have/haven’t been subsidized during their execution, will continue to be provided, after their finalization, and if their costs will be covered (explain financing mechanism), in order to assure the continuity of these services in favor of the beneficiaries, indicate if the organization selected to give continuity to the services is adequate/not adequate) to assure the financial self-sustainability of these services and whatare the planned forms of management, indicate the possible measures to avoid the risks (if any) connected with the above. Social sustainability indicate if the population has participated (actively/moderately/insufficiently) in the design and the execution of the Project and if it is expected that, after the Project, the same will maintain the flow of the benefits achieved. Political sustainability indicate if the project has been identified/implemented within the frame of national sector policy/programme, under the coordination of a sector organization and if this presents little/moderate/excellent perspective of future continuity, indicate if the project responds/does not respond to the interests of specific groups (private sector, civil society….), having received/not received from these groups support under the form of institutional/financial participation, likely to be continued after the project. Management capacity indicate if the organization responsible for project implementation is sufficiently/insufficiently consolidated or in the process of being consolidated and therefore able/incapable to assume, in perspective, the management of post project activities. indicate if the strategy of progressive hand-over of the functions from the project is/is not taking place correctly, and the measures needed to improve the transition.

MONIT ORING

111

MONITORING REPORT (MR)

KEY OBSERVATIONS AND RECOMMENDATIONS (BY WHOM AND IN ORDER OF PRIORITY) This is the most important section of the Monitoring Report. Monitors must identify their key observations based on the most important issues they have identified in the BCS. Monitors must also recommend what actions must be taken and by whom. If several issues require follow-up, they should be grouped according to the intended 9 recipient and placed in order of priority . Some important rules to be respected: Recommendations must constitute clear responses to the most important issues identified in the Monitoring Report; Recommendations unequivocal;

must

clear

and

Recommendations must address a few of the most important issues, they do not have to address all of them; Recommendations must be addressed to the stakeholder in a position to implement them. Some recommendations are to be avoided: DO NOT recommend directly an extension of project/contract unless it is already in the books and favourably recommended by the EC Delegation. If not the case, just state clearly that the project will not be able to achieve its expected results in the available time. Do not make obvious recommendations without any added value (like: “improve the quality of the logframe”) Do not make recommendations that are not feasible at the stage of implementation (ex. recommending improvements of the logframe for a project in its last year is not opportune).

Indicate importance and urgency by assigning at the end of each recommendation a value: (1) = important and urgent; (2) = important but less urgent; (3) = urgent but less important; (4) = less important and less urgent.

MONIT ORING

Suggested possible “Recommendations” Relevance deepen the knowledge of the context better define the political priorities reformulate some aspects of the ML modify contractual aspects improve the institutional framework modify the financing plan modify the calendar Efficiency speed up the use of the financial resources improve the management and administration system strengthen the activities or improve their quality improve the level of outputs production or their quality improve the system of monitoring activities and results improve the coordination between the concerned actors and their participation Effectiveness measures to increase the quality of the benefits for the target population possible realistic restructuring of the OVI of the PP (on condition that it does not enter in contradiction with the commitments assumed by the FA/GC), measures to increase the use/appropriation of the incorporated services, corrective measures against the obstacles that prevent the achievement of the PP Impact collaborations that the project needs to realize or to extend possible actions concerning other actors (politicians, institutional, cooperators) to increase the incidence of the results of the project on the wider sector context measures to minimize possible negative impacts Sustainability measures to improve the financial selfsustainability, measures to increase ownership of the project results by the beneficiaries, measures to increase the project’s insertion in public policies measures to increase the institutional/organizational management

112

III Grading:

MONITORING REPORT DRAFT I. Intervention data: Monitoring report type (ongoing / ex post) Aid modality (project approach, SPSP) Project type (national, regional) Financed via thematic line (yes, no) CRIS Decision Number (if decision monitored) CRIS Contract Number (if contract monitored) Project Title (as in financial agreement) DAC-CRS Sector (as in CRIS) Additional DAC-CRS code (compulsory) Geographical Zone (Country, region if regional) Date Financing Agreement/contract signed (**/**/****) HQ responsible (if centrally managed) Delegation Task manager (Name SURNAME) Monitor (Name SURNAME) Project Authority name (or/implementation agency) Type of Implementing Partner code (1 to 10) Start date – planned (**/**/****) End date – planned (**/**/****) Start date – actual (**/**/****) End date – likely (**/**/****) Monitoring visit date from (briefing date) To (debriefing date)

II FINANCIAL DATA: Primary commitment (EC financing) Budget allocated to Technical Assistance Secondary commitment (funds contracted of EC contribution) Other funding (government and/or other donors) Total budget of operation (primary EC + other donors) Total EC funds disbursed so far Financial data as at (**/**/****)

III GRADING: 1. Relevance and quality of design

2. Efficiency of implementation to date

3. Effectiveness to date

4. Impact prospects

5. Potential sustainability

B a = very good; b = good; c= problems; d = deficiencies

113

IV Summary of conclusions:

IV SUMMARY OF CONCLUSIONS: RELEVANCE AND QUALITY OF DESIGN

EFFICIENCY OF IMPLEMENTATION TO DATE

EFFECTIVENESS TO DATE

IMPACT PROSPECTS

POTENTIAL SUSTAINABILITY

KEY OBSERVATIONS AND RECOMMENDATIONS

See the check list in the following pages This will be used by the Quality Control staff group in the CO, Brussels. It also provides you with a synthesis of quality control criteria useful to prepare the reports. In case your first version does not answer those criteria, you will receive (by e-mail) the observations about what should be improved. Please be aware that as long as you have not answered the observations sent by QC, your report will not be accepted.

114

IV Summary of conclusions:

MONITORING REPORT QUALITY ASSURANCE CHECKLIST (Not to be erased before QC) Aspects of Monitoring Report to be checked

Intervention and financial data

Comments (reserved to QC) To be revised

Project data copied from Excel table data and pasted in corresponding cells in intervention data table in MR.

If data revised / updated must be according to latest available CRIS data and manual guidelines. Check CRS codes!

Data must correspond to those in PS (names, dates etc).

Financial data correspond to latest available CRIS fiche.

Financial data without comma, dots, decimals or â&#x201A;Ź sign.

Grading

To be revised

Grading of 5 DAC criteria must correspond with those in BCS.

General aspects of text (compulsory) Template respected: Text Arial, 10 pt, English (UK), Justified space: simple. Title Arial 11 pt, bold, space below 6 pt.

Fill up 2 pages of text as much as possible; not more unless SPSP or specially complex project.

No unnecessary descriptive information that is already in the PS.

Check orthography and syntax!!!

Check style and clarity! Avoid jargon unintelligible to outsiders who will read the MR! Explain all acronyms (between brackets). Follow logical structure of BCS and resume important comments from BCS in MR text. Most important aspects are always those with bearing on DAC criteria grading Use BCS as guideline but do not repeat BCS questions or irrelevant information in MR.

To be revised

115

IV Summary of conclusions:

Relevance and quality of design

To be revised

Always: short introductory phrase on overall relevance and quality explaining score.

Discuss only aspects of relevance and quality of design, no other.

Always: comment on important features of relevance especially on aspects of likelihood of solutions proposed by project to solve problems addressed and alignment with sector policy. Follow instructions of the Eptisa ROM Monitor Quality Assurance Manual (section 5) and select most important aspects from BCS (section 1.1).

Always: comment on important features of formal quality of design, which is translation of proposed solutions into an adequate logical framework. Follow instructions in guidelines and select most important aspects from BCS (section 1.2).

Always: comment on important features of technical quality and feasibility of design, which is the quality and feasibility of proposed solutions from a technical point of view (technical design and formal design are not necessarily of the same quality). Follow instructions in the Eptisa ROM Monitor Quality Assurance Manual (section 5) and select most important aspects from BCS (section 1.3).

Always if relevant and bearing on grading: comment on the process of identification and formulation in terms of appropriation by local stakeholders. Follow instructions in the Eptisa ROM Monitor Quality Assurance Manual (section 5) and select most important aspects from BCS (section 1.4).

Always: comment on quality of design of implementation framework in terms of coordination, management and financing arrangements. Follow instructions in the Eptisa ROM Monitor Quality Assurance Manual (section 5) and select most important aspects from BCS (section 1.4).

Always if relevant and bearing on grading: comment on the quality of gender, environment, human rights and good governance mainstreaming aspects. Follow instructions in the Eptisa ROM Monitor Quality Assurance Manual (section 5) and select most important aspects from BCS (section 1.5 and 7).

Efficiency Always: short introductory phrase on overall efficiency explaining score

Discuss only aspects of efficiency, no other. Check definition of efficiency!!!

Always: comment on important features of inputs availability and use (funds, human resources mainly). Follow instructions in the Eptisa ROM Monitor Quality Assurance Manual (section 5) and select most important aspects from BCS (section 2.1).

Always: comment on important features of implementation of activities. Follow instructions in the Eptisa ROM Monitor Quality Assurance Manual (section 5) and select most important aspects from BCS (section 2.2).

Always: include specific information on progress of implementation of activities (not just generic).

To be revised

116

IV Summary of conclusions:

Always: include specific information on progress of delivery and quality of outputs (not just generic). Careful! Outputs are direct products from activities! Follow instructions in Eptisa ROM Monitor Quality Assurance Manual (section 5) and select most important aspects from BCS (section 2.3). Always: include specific information on partner contribution and coordination (not just generic). Follow instructions in Eptisa ROM Monitor Quality Assurance Manual (section 5) and select most important aspects from BCS (section 2.4).

Effectiveness

To be revised

Always: short introductory phrase on overall effectiveness explaining grading. Discuss only aspects of effectiveness, no other. Check definition of effectiveness!!! Always: include specific information and comment on achievement of results (intermediary outcomes). Outcomes are what target groups do with the project outputs!!Follow instructions in Eptisa ROM Monitor Quality Assurance Manual (section 5) and select most important aspects from BCS (section 3.1). Always: Discuss likelihood of achieving the project purpose. This is probably going to be speculative so that you have to comment on any issues that make your comments speculative (lack of OVIs etc). Follow instructions in Eptisa ROM Monitor Quality Assurance Manual (section 5) and select most important aspects from BCS (section 3.2). If relevant and bearing on grading: include specific information on changes in external conditions and unplanned effects (negative or positive) likely to influence the achievement of the project purpose.

Impact

To be revised

Always: short introductory phrase on potential impact or observable impact explaining grading. Discuss only aspects of impact, no other. Check definition of impact!!! Always: include specific information and discussion on contribution of project to overall objective(s). This is probably going to be speculative so that you have to comment on any issues that make your comments speculative (lack of OVIs etc). Follow instructions in Eptisa ROM Monitor Quality Assurance Manual (section 5) and select most important aspects from BCS (section 4.1). If relevant and bearing on grading: include specific information on unplanned effects (negative or positive) likely to influence the impact (based on your comments in BCS, section 4.2).

Sustainability Always: short introductory phrase on potential sustainability or observe sustainability explaining grading. Discuss only aspects of sustainability, no other. Check definition of sustainability!!! Always: include specific information and discussion on financial and economic sustainability. Follow instructions in the Eptisa ROM Monitor Quality Assurance Manual (section 5) and select most important aspects from BCS (section 5.1). Always: include specific information and discussion on institutional and social sustainability. Follow instructions in Eptisa ROM Monitor Quality Assurance Manual (section 5) and select most important aspects from BCS (section 5.2).

To be revised

117

IV Summary of conclusions:

If relevant and bearing on grading: include specific information on policy support aspects likely to influence sustainability (based on your comments in BCS, section 5.3). Always: include specific information on capacity building support likely to influence sustainability (based on your comments in BCS, section 5.4).

Key observations and recommendations Always: include recommendations when an issue is identified in MR text especially if the issue justifies a “c” or “d” grading. Always: explain to whom recommendations are addressed (Project Management, others). Always: number recommendations (Arabic numbers starting from 1 per each stakeholder). Always: include specific recommendations and avoid generic, self evident recommendations (“improve OVIs...”; “speed up implementation” etc). Precise identification of issue = precise recommendation! Always: avoid unrealistic, unchecked recommendations. Always: limit number of recommendations to important issues where they have an added value and can be followed by project management and EU delegation. For ex posts: no recommendations for project, lessons learnt addressed to relevant stakeholders

Other comments by QC if relevant

To be revised

118

IV Summary of conclusions:

119

Guidelines for ROM mission leaders

The Mission Leader has a specific role to play in each ROM mission. The official ROM Handbook 2009 defines this role and we have developed practical guidelines based on this Handbook (see box1) and EptisaÂ´s experience, as well as specifics of the ROM ACP 2011-2014 contract.

1 MISSION PLANNING: The Mission Leader participates in the coordination of the mission agenda with the ROM Coordination Office and the Delegation, in particular in the following aspects: 1.1 Scheduling of the mission briefing and debriefing with the Delegation ROM focal person. 1.2 Overview of mission schedule planning: the Mission Leader needs to contact the whole team, as soon as the confirmation of terms of reference for the mission and the project contacts are provided by the ROM Coordination Office. He must follow up mission planning in general to ensure that each monitor has contacted his/her project team and task manager and agreed on an overall schedule for the field mission. 1.3 Coordination of linkage between monitors: Ideally all monitors should stay in the same hotel, or nearby hotels, before the briefing and debriefing, so that it is easy to organize the necessary coordination meetings (see further).

Navigate TROUGH the manual

Box 1 - Extract from ROM Handbook 2009 The introductory meeting upon arrival in country is the first step to build team dynamics and clarify responsibilities. This helps to avoid inconsistent or contradictory activities or statements by individual monitors and facilitates immediate attention when such situations arise. The primary responsibility for managing and ensuring effective teamwork is with the Mission Leader. Monitors must be team players and committed to share information and knowledge. It is therefore highly recommended that the monitors stay in the same Hotel (at least for the first and last days of the mission). Ongoing dialogue among team members also ensures that monitors do not work in isolation, and that less experienced monitors have full support from the team as a whole. In addition, this increases understanding of the issues that will be discussed at the debriefings. It is essential to continuously update the monitors on good practices and changes in the ROM system. The Mission Leader must ensure monitors are aware and understand the current guidelines including all updates of templates, instructions and methodologies. Less experienced monitors might require specific attention while on mission. ROM contractors have to provide all relevant information on recent ROM developments to the Mission Leaders and monitors.

GUIDELINES MISSION LEADER

GUIDELINES FOR ROM MISSION LEADERS

120

Guidelines for ROM mission leaders

2- MISSION FIELD PHASE This is when the Mission Leader role is most important. His/her duties are as follows: 2.1 Organization of a pre-briefing mission: the team pre briefing meeting must be organized ideally the day before the briefing at DELUE. It is especially important that experts with no or little previous ROM experience are present at this meeting. The Mission Leader must brief the team on latest updates in methodology, reporting templates etc, agree with the team on the organization of the briefing presentations, and ensure communication between the team members during the field mission. If previous training/coaching is needed for some experts at this stage it must be organized beforehand so that at least half a day is made available for this. 2.2. Delegation briefing: The monitoring team’s first formal task is to brief the Delegation and other stakeholders. The format of the briefing may vary slightly according to the Delegation’s wishes and time availability, but a few necessary steps must be followed in all cases. At the briefing it is the Mission Leader’s responsibility to introduce the monitoring team; describe the ROM system (and new developments) and the role of the monitor; explain the strategy for the mission; answer any questions by the Delegation staff; and arrange a date for the de-briefing of the Delegation. A short presentation in Power Point is available if a general ROM introduction is requested, although in most cases Delegation staff is now sufficiently aware of the methodology. In general, after an introductory meeting, each Monitor should be able to meet separately with the relevant task managers to discuss the respective projects and monitoring needs more in detail. The Mission Leader must remind all monitors, that the project data provided by the ROM Coordination office (first section of MR) should always be checked with the Task Manager. A very important role of the Mission Leader, is to make sure that appropriate solutions are found for organizational problems that may arise at the briefing (see box 2).

The Mission Leader must record, in the corresponding template, the persons present at the briefing, as well as any important issues/ remarks raised during the meeting. This must be sent by e-mail to the ROM Coordination Office as soon as possible after the de-briefing. This sheet is to be part of the “End of ROM mission Note” which is a contractual requirement for the ROM Coordination Office. 2.3. NAO briefing: depending on whether they have been invited or not to the Delegation briefing, the NAO staff should be also briefed at the onset of the mission. A separate briefing at the NAO may be organized for the monitors concerned (depending on the NAO responsibility over projects) but should always be agreed upon with the Delegation (through the ROM Focal person).

Box 2. Role of the Mission Leader in case of emergency If a mission member is faced with an emergency (accident, civil disruption etc) the Mission Leader should be informed at once. If he is not reachable, the monitor should inform the ROM coordination office as first priority. The Delegation is not legally responsible for emergencies but should also be informed, whether by the Mission Leader or, if not reached in time, by the monitor directly. Whenever repatriation and insurance interventions are needed, the monitor should follow the procedure indicated in the Insurance Annex. Assistance should by requested by making a reverse charge call to the number 91.572.43.43 and for internationals the number is 34.91.572.43.43; the following information must be provided: Name and surnames. Policy number. Address and telephone number of the place where you are located. Description of the existing problem.

121

Guidelines for ROM mission leaders

2.4. Field phase: during the field phase the Mission leader must remain in contact with the ROM team and Delegation for any eventuality. It is very important to ensure before the briefing, that the schedule of each monitor is clear and that contact numbers are available. In all cases the Mission Leader should exchange mobile numbers with the team. An important role for the Mission Leader is to liaise and intervene in case of emergency affecting a member of the team. The Mission Leader must also be available to counsel and coach monitors about issues of methodology and mission organization during the field phase. 2.5. Coordinating de-briefing fiches: the Mission Leader must instruct all monitors to fill the debriefing fiches for each monitoring exercises, following the instructions and templates, and provide them in advance for the debriefing, in time to prepare the Power Pint presentation. 2.6. Pre de-briefing meeting and preparation of debriefing presentation: the Mission Leader must organize a meeting of the whole team previous to the debriefing, in order to review all the debriefing fiches, discuss main conclusions and recommendations, and ensure the organization of the de-briefing with the team. Cross-cutting issues and conclusions for the mission may also be discussed. It is the responsibility of the Mission Leader to compile all debriefing fiches into the Debriefing Presentation PowerPoint template. 2.7. De-briefing session: the de-briefing is the culminating point of the ROM mission. This is the moment when the whole team presents its preliminary assessment of the projects, and a quick feedback from the Delegation is obtained. The quality and efficiency of the presentation aretherefore fundamental. In order to make discussion more transparent, and avoid tiresome endless meetings (generally Delegation staff has limited time available), Eptisa has developed over the years a debriefing presentation template presenting in very synthetic form, the preliminary conclusions and observations of the mission.

Box 3. Solving mission organization problems at the briefing Despite careful preparation, last minute problems often arise at the onset of the mission, either for unforeseen reasons or lack of proper communication. Some examples of frequent problems and the way the Mission Leader should intervene: Problems with availability of a monitor: flights problems etc may have delayed the arrival of a monitor and upset his/her schedule. If a monitor is not present at the briefing the ML must make sure that proper measures are taken so that upon arrival, he/she can brief the TM and the schedule is reviewed. The ROM Coordination office should be informed immediately. Security issues: the Mission Leader must make sure that any security warnings/ instructions provided by the Delegation are followed by the team. By verifying ex ante security status of the country the relevant issues may be raised at planning stage and reviewed during the briefing. Unexpected problems for the field visit: miscommunication may produce misunderstanding and disrupt a monitor schedule. For instance, a longer time than expected may be needed to reach a field area, disrupting the whole schedule. The ML must help find solutions within the contractual scope of the ROM exercise. In some cases, when proper field visits are proven impossible (last minute security problems for instance), a ROM exercise may need to be downgraded from Monitoring Report to Monitoring Note. These cases must be reported immediately to the ROM CO. It is very important that the Mission Leader agrees in advance with the Delegation on the de-briefing format before organizing the presentation. Depending on time available, monitors may be instructed to take more or less time for each project. In cases there are many projects, it can be useful to propose to organize the de-briefing by Delegation sections, so that the staff does not have to sit through a lengthy meeting to wait for â&#x20AC;&#x153;theirâ&#x20AC;? projects.

122

Guidelines for ROM mission leaders

As in the briefing, the Mission Leader must record, in the corresponding template, the persons present at the de-briefing, as well as any important issues/ remarks raised during the meeting. This must be sent by e-mail to the ROM Coordination Office as soon as possible after the de-briefing. This sheet is to be part of the “End of ROM mission Note” which is a contractual requirement for the ROM Coordination Office.

3- POST-MISSION PHASE In other contracts Mission Leaders have a key role to play in quality control of monitors’ reports. In our case, our ToRs have defined a centrally managed quality control. Therefore, the Mission Leader’s duties after the mission are limited to the following: Providing the outline of briefing and debriefing report (see above); Ensure his/her own reporting commitments; Any suggestions to improve future missions; Give proper feedback to Eptisa ROM coordination (team leader) whenever the quality of the work, or behaviour of a monitor has presented a challenge. This feedback is needed to help the monitor improve or, avoid further problems in the future.

123

Box 4. The debriefing presentation The format used for debriefing presentations has been validated as very useful in dozens of ROM missions. Some important points must be kept in mind by the Mission Leader: Ratings as presented are preliminary; the meaning of the ratings has to be explained again at each debriefing. Only the most important observations and recommendations must be presented orally; the ML must ensure that all monitors avoid reading their slides and concentrate on significant issues. Feedback from the Delegation is important at this point; whenever a task manager or other stakeholder question the findings, it has to be noted; factual mistakes must be investigated and corrected by the Monitor. The ML must make sure that important remarks are taken into account while ensuring at the same time that the independence of the monitor is maintained. Whenever serious issues of quality of monitoring are outlined in the debriefing, the ML should give some feedback to the ROM Coordination office.

This report needs a strong input from the TL. If PP presentation contains aggregate considerations, summarize them in the EMR text. Comments and recommendations made by the Delegation during the debriefing should be reported in the EMR. Logistic shortcoming should be mentioned, with suggestions on how to avoid them in future missions.

CONFLICTS OF INTEREST

4 - END MISSION REPORT (EMR)

Independence and avoidance of conflicts of interest

INDEPENDENCE AND AVOIDANCE OF CONFLICTS OF INTEREST The present section describes how EPTISA Consortium deals with the issue of potential and/or actual conflicts of interest in providing external monitoring services. Conflicts of interests must be considered at two levels: Conflicts of interest of the firm(s) within the consortium: as a result of having previously been involved in the execution of the project in question through the provision of Technical Assistance or other services. Also occurs when the monitoring involves a project/programme where one of the members of the Consortium has a particular interest and is considering participating in tenders for the provision of services (generally ITA). In these cases, the company’s privileged access to information that could be useful to their participation in the tender would be unfair to competitors. Conflicts of interest of the individual experts: these conflicts occur when an expert is asked to monitor a project in which he/she has had previous involvement and where the current success/failure of the project could be attributed to the prior intellectual/operational contribution of that expert. This includes participation in the formulation stage for the project being monitored, the elaboration of Terms of Reference for a follow-on project, provision of technical expertise to the project, the implementation of a previous phase of the project. Other conflicts of interest may arise when individual experts are linked to institutions involved in the projects in a way that may alter their independence. For both cases, EPTISA Consortium adopts two types of measures: Preventive measures: for member companies, a list of operations where they have involvement that may be a conflict of interest with ROM, to be updated regularly against lists of projects to be monitored; for individual experts, the signature of a “Declaration of independence and objectiveness”.

Navigate TROUGH the manual

Corrective measures: for member companies, a special procedure is put in place to ensure that operations where there are actual or potential conflicts.

Avoidance of conflicts of interest by the monitoring experts Individual experts are required, before the final composition of the Mission Terms of Reference, to give any information about EU funded projects with which they have maintained a contractual relationship. In case a potential conflict exists, the expert is not assigned to the corresponding project. In any case, all experts are required to sign a declaration of objectivity and independence in which they commit themselves to avoid any conflict of interest. Especially the locally based regional experts, are asked to provide information on any institutional conflict of interest that might affect their independent judgement. Local experts are preferably not used to monitor projects in their native areas, at least in rural areas where social pressure may affect their objectivity.

124

Independence and avoidance of conflicts of interest

125

LIST OF CRS PURPOSE CODES

126

LIST OF CRS PURPOSE CODES CRS CODE

DESCRIPTION

Clarifications / Additional notes on coverage

110

EDUCATION

111

Education, level unspecified

The codes in this category are to be used only when level of education is unspecified or unknown (e.g. training of primary school teachers should be coded under 11220).

11110

Education policy and administrative management

Education sector policy, planning and programmes; aid to education ministries, administration and management systems; institution capacity building and advice; school management and governance; curriculum and materials development; unspecified education activities.

11120

Education facilities and training

Educational buildings, equipment, materials; subsidiary services to education (boarding facilities, staff housing); language training; colloquia, seminars, lectures, etc.

11130

Teacher training

Teacher education (where the level of education is unspecified); in-service and pre-service training; materials development.

11182

Educational research

Research and studies on education effectiveness, relevance and quality; systematic evaluation and monitoring.

112

Basic education

11220

Primary education

Formal and non-formal primary education for children; all elementary and first cycle systematic instruction; provision of learning materials.

11230

Basic life skills for youth and adults

Formal and non-formal education for basic life skills for young people and adults (adults education); literacy and numeracy training.

11240

Early childhood education

Formal and non-formal pre-school education.

113

Secondary education

11320

Secondary education

Second cycle systematic instruction at both junior and senior levels.

11330

Vocational training

Elementary vocational training and secondary level technical education; on-the job training; apprenticeships; including informal vocational training.

114

Post-secondary education

11420

Higher education

Degree and diploma programmes at universities, colleges and polytechnics; scholarships.

11430

Advanced technical and managerial training

Professional-level vocational training programmes and in-service training.

120

HEALTH

Navigate TROUGH the manual

CRS CODES

DAC 5 CODE

LIST OF CRS PURPOSE CODES

121

Health, general

12110

Health policy and administrative management

Health sector policy, planning and programmes; aid to health ministries, public health administration; institution capacity building and advice; medical insurance programmes; unspecified health activities.

12181

Medical education/training

Medical education and training for tertiary level services.

12182

Medical research

General medical research (excluding basic health research).

12191

Medical services

Laboratories, specialised clinics and hospitals (including equipment and supplies); ambulances; dental services; mental health care; medical rehabilitation; control of noninfectious diseases; drug and substance abuse control [excluding narcotics traffic control (16063)].

122

Basic health

12220

Basic health care

Basic and primary health care programmes; paramedical and nursing care programmes; supply of drugs, medicines and vaccines related to basic health care.

12230

Basic health infrastructure

District-level hospitals, clinics and dispensaries and related medical equipment; excluding specialised hospitals and clinics (12191).

12240

Basic nutrition

Direct feeding programmes (maternal feeding, breastfeeding and weaning foods, child feeding, school feeding); determination of micro-nutrient deficiencies; provision of vitamin A, iodine, iron etc.; monitoring of nutritional status; nutrition and food hygiene education; household food security.

12250

Infectious disease control

Immunisation; prevention and control of infectious and parasite diseases, except malaria (12262), tuberculosis (12263), HIV/AIDS and other STDs (13040). It includes diarrheal diseases, vector-borne diseases (e.g. river blindness and guinea worm), viral diseases, mycosis, helminthiasis, zoonosis, diseases by other bacteria and viruses, pediculosis, etc.

12261

Health education

Information, education and training of the population for improving health knowledge and practices; public health and awareness campaigns.

12262

Malaria control

Prevention and control of malaria.

12263

Tuberculosis control

Immunisation, prevention and control of tuberculosis.

12281

Health personnel development

Training of health staff for basic health care services.

130

POPULATION POLICIES/PROGRAMMES AND REPRODUCTIVE HEALTH

13010

Population policy and administrative management

Population/development policies; census work, vital registration; migration data; demographic research/analysis; reproductive health research; unspecified population activities.

13020

Reproductive health care

Promotion of reproductive health; prenatal and postnatal care including delivery; prevention and treatment of infertility; prevention and management of consequences of abortion; safe motherhood activities.

127

LIST OF CRS PURPOSE CODES

13030

Family planning

Family planning services including counselling; information, education and communication (IEC) activities; delivery of contraceptives; capacity building and training.

13040

STD control including HIV/AIDS

All activities related to sexually transmitted diseases and HIV/AIDS control e.g. information, education and communication; testing; prevention; treatment, care.

13081

Personnel development for population and reproductive health

Education and training of health staff for population and reproductive health care services.

140

WATER SUPPLY AND SANITATION

14010

Water resources policy and administrative management

Water sector policy, planning and programmes; water legislation and management; institution capacity building and advice; water supply assessments and studies; groundwater, water quality and watershed studies; hydrogeology; excluding agricultural water resources (31140).

14015

Water resources protection

Inland surface waters (rivers, lakes, etc.); conservation and rehabilitation of ground water; prevention of water contamination from agro-chemicals, industrial effluents.

14020

Water supply and sanitation - large systems

Water desalination plants; intakes, storage, treatment, pumping stations, conveyance and distribution systems; sewerage; domestic and industrial waste water treatment plants.

14030

Basic drinking water supply and basic sanitation

Water supply and sanitation through low-cost technologies such as handpumps, spring catchment, gravity-fed systems, rain water collection, storage tanks, small distribution systems; latrines, small-bore sewers, on-site disposal (septic tanks).

14040

River development

Integrated river basin projects; river flow control; dams and reservoirs [excluding dams primarily for irrigation (31140) and hydropower (23065) and activities related to river transport (21040)].

14050

Waste management/disposal

Municipal and industrial solid waste management, including hazardous and toxic waste; collection, disposal and treatment; landfill areas; composting and reuse.

14081

Education and training in water supply and sanitation

150

GOVERNMENT AND CIVIL SOCIETY

151

Government and civil society, general

15110

Economic and development policy/planning

Macro-economic, fiscal and monetary policy and planning; social planning; economic and social analysis and forecasting; development planning and preparation of structural reforms; organisational development; support to ministries involved in aid co-ordination; other ministries and government departments when sector cannot be specified. (Use code 51010 for budget support to macroeconomic reforms.)

15120

Public sector financial management

Strengthening financial and managerial accountability; public expenditure management; improving financial management systems; tax assessment procedures; budget drafting; field auditing; measures against waste, fraud and corruption.

15130

Legal and judicial development

Constitutional development, legal drafting; institutional strengthening of legal and judicial systems; legal training and education; legal advice and services; crime prevention.

15140

Government administration

Systems of government including parliament, local government, decentralisation; civil service and civil service reform. Including general services by government (or commissioned by

128

LIST OF CRS PURPOSE CODES

government) not elsewhere specified e.g. police, fire protection; cartography, meteorology, legal metrology, aerial surveys and remote sensing; administrative buildings.

15150

Strengthening civil society

Community participation and development; co-operatives; grassroots organisations; development of other participatory planning and decision making procedures and institutions (see code 15220 for civilian peace-building, conflict prevention and resolution).

15161

Elections

Electoral assistance and monitoring, voters’ education [other than in connection with UN peace building (15230)].

15162

Human rights

Monitoring of human rights performance; support for national and regional human rights bodies; protection of ethnic, religious and cultural minorities [other than in connection with UN peace building (15230)].

15163

Free flow of information

Uncensored flow of information on public issues, including activities that increase the professionalism, skills and integrity of the print and broadcast media (e.g. training of journalists).

15164

Women’s equality organisations and institutions

Support for institutions and organisations (governmental and non-governmental) working for gender equality and women’s empowerment.

Conflict prevention and resolution, peace and security

N.B. Further notes on ODA eligibility (and exclusions) of conflict, peace and security related activities are given in paragraph 39 of the DAC Statistical Reporting Directives.

15210

Security system management and reform

Technical co-operation provided to parliament, government ministries, law enforcement agencies and the judiciary to assist review and reform of the security system to improve democratic governance and civilian control; technical co-operation provided to government to improve civilian oversight and democratic control of budgeting, management, accountability and auditing of security expenditure, including military budgets, as part of a public expenditure management programme; assistance to civil society to enhance its competence and capacity to scrutinise the security system so that it is managed in accordance with democratic norms and principles of accountability, transparency and good governance.

15220

Civilian peace-building, conflict prevention and resolution

Support for civilian activities related to peace building, conflict prevention and resolution, including capacity building, monitoring, dialogue and information exchange.

15230

Post-conflict peace-building (UN)

Participation in the post-conflict peace-building phase of United Nations peace operations (activities such as human rights and elections monitoring, rehabilitation of demobilised soldiers, rehabilitation of basic national infrastructure, monitoring or retraining of civil administrators and police forces, training in customs and border control procedures, advice or training in fiscal or macroeconomic stabilisation policy, repatriation and demobilisation of armed factions, and disposal of their weapons; support for landmine removal). Direct contributions to the UN peacekeeping budget are excluded from bilateral ODA (they are reportable in part as multilateral ODA).

15240

Reintegration and SALW control

Reintegration of demobilised military personnel into the economy; conversion of production facilities from military to civilian outputs; technical co-operation to control, prevent and/or reduce the proliferation of small arms and light weapons (SALW) – see para. 39 of the DAC Statistical Reporting Directives for definition of SALW activities covered. [Other than in connection with UN peace-building (15230) or child soldiers (15261)].

15250

Land mine clearance

Explosive mine removal for developmental purposes [other than in connection with UN peace-building (15230)].

15261

Child soldiers (Prevention and demobilisation)

Technical co-operation provided to government – and assistance to civil society organisations – to support and apply legislation designed to prevent the recruitment of child soldiers, and to demobilise, disarm, reintegrate, repatriate and resettle (DDR) child soldiers.

152

129

LIST OF CRS PURPOSE CODES

160

OTHER SOCIAL INFRASTRUCTURE AND SERVICES

16010

Social/ welfare services

Social legislation and administration; institution capacity building and advice; social security and other social schemes; special programmes for the elderly, orphans, the disabled, street children; social dimensions of structural adjustment; unspecified social infrastructure and services, including consumer protection.

16020

Employment policy and administrative management

Employment policy and planning; labour law; labour unions; institution capacity building and advice; support programmes for unemployed; employment creation and income generation programmes; occupational safety and health; combating child labour.

16030

Housing policy and administrative management

Housing sector policy, planning and programmes; excluding low-cost housing and slum clearance (16040).

16040

Low-cost housing

Including slum clearance.

16050

Multisector aid for basic social services

Basic social services are defined to include basic education, basic health, basic nutrition, population/reproductive health and basic drinking water supply and basic sanitation.

16061

Culture and recreation

Including libraries and museums.

16062

Statistical capacity building

Both in national statistical offices and any other government ministries.

16063

Narcotics control

In-country and customs controls including training of the police; educational programmes and awareness campaigns to restrict narcotics traffic and in-country distribution.

16064

Social mitigation of HIV/AIDS

Special programmes to address the consequences of HIV/AIDS, e.g. social, legal and economic assistance to people living with HIV/AIDS including food security and employment; support to vulnerable groups and children orphaned by HIV/AIDS; human rights of HIV/AIDS affected people.

TRANSPORT AND STORAGE

Note: Manufacturing of transport equipment should be included under code 32172.

21010

Transport policy and administrative management

Transport sector policy, planning and programmes; aid to transport ministries; institution capacity building and advice; unspecified transport; activities that combine road, rail, water and/or air transport.

21020

Road transport

Road infrastructure, road vehicles; passenger road transport, motor passenger cars.

21030

Rail transport

Rail infrastructure, rail equipment, locomotives, other rolling stock; including light rail (tram) and underground systems.

21040

Water transport

Harbours and docks, harbour guidance systems, ships and boats; river and other inland water transport, inland barges and vessels.

21050

Air transport

Airports, airport guidance systems, aeroplanes, aeroplane maintenance equipment.

21061

Storage

Whether or not related to transportation.

21081

Education and training in transport and storage

210

130

LIST OF CRS PURPOSE CODES

22010

Communications policy and administrative management

Communications sector policy, planning and programmes; institution capacity building and advice; including postal services development; unspecified communications activities.

22020

Telecommunications

Telephone networks, telecommunication satellites, earth stations.

22030

Radio/television/print media

Radio and TV links, equipment; newspapers; printing and publishing.

22040

Information and communication technology (ICT)

Computer hardware and software; internet access; IT training. When sector cannot be specified.

230

ENERGY GENERATION AND SUPPLY

23010

Energy policy and administrative management

Energy sector policy, planning and programmes; aid to energy ministries; institution capacity building and advice; unspecified energy activities including energy conservation.

23020

Power generation/non-renewable sources

Thermal power plants including when heat source cannot be determined; combined gas-coal power plants.

23030

Power generation/renewable sources

Including policy, planning, development programmes, surveys and incentives. Fuelwood/ charcoal production should be included under forestry (31261).

23040

Electrical transmission/ distribution

Distribution from power source to end user; transmission lines.

23050

Gas distribution

Delivery for use by ultimate consumer.

23061

Oil-fired power plants

Including diesel power plants.

23062

Gas-fired power plants

23063

Coal-fired power plants

23064

Nuclear power plants

Including nuclear safety.

23065

Hydro-electric power plants

Including power-generating river barges.

23066

Geothermal energy

23067

Solar energy

Including photo-voltaic cells, solar thermal applications and solar heating.

23068

Wind power

Wind energy for water lifting and electric power generation.

23069

Ocean power

Including ocean thermal energy conversion, tidal and wave power.

23070

Biomass

Densification technologies and use of biomass for direct power generation including biogas, gas obtained from sugar cane and other plant residues, anaerobic digesters.

23081

Energy education/training

Applies to all energy sub-sectors; all levels of training.

131

LIST OF CRS PURPOSE CODES

23082

240

Energy research

Including general inventories, surveys.

BANKING AND FINANCIAL SERVICES

24010

Financial policy and administrative management

Finance sector policy, planning and programmes; institution capacity building and advice; financial markets and systems.

24020

Monetary institutions

Central banks.

24030

Formal sector financial intermediaries

All formal sector financial intermediaries; credit lines; insurance, leasing, venture capital, etc. (except when focused on only one sector).

24040

Informal/semi-formal financial intermediaries

Micro credit, savings and credit co-operatives etc.

24081

Education/training in banking and financial services

250

BUSINESS AND OTHER SERVICES

25010

Business support services and institutions

Support to trade and business associations, chambers of commerce; legal and regulatory reform aimed at improving business and investment climate; private sector institution capacity building and advice; trade information; public-private sector networking including trade fairs; e-commerce. Where sector cannot be specified: general support to private sector enterprises (in particular, use code 32130 for enterprises in the industrial sector).

25020

Privatisation

When sector cannot be specified. Including general state enterprise restructuring or demonopolisation programmes; planning, programming, advice.

311

AGRICULTURE

31110

Agricultural policy and administrative management

Agricultural sector policy, planning and programmes; aid to agricultural ministries; institution capacity building and advice; unspecified agriculture.

31120

Agricultural development

Integrated projects; farm development.

31130

Agricultural land resources

Including soil degradation control; soil improvement; drainage of water logged areas; soil desalination; agricultural land surveys; land reclamation; erosion control, desertification control.

31140

Agricultural water resources

Irrigation, reservoirs, hydraulic structures, ground water exploitation for agricultural use.

31150

Agricultural inputs

Supply of seeds, fertilizers, agricultural machinery/equipment.

31161

Food crop production

Including grains (wheat, rice, barley, maize, rye, oats, millet, sorghum); horticulture; vegetables; fruit and berries; other annual and perennial crops. [Use code 32161 for agroindustries.]

31162

Industrial crops/export crops

Including sugar; coffee, cocoa, tea; oil seeds, nuts, kernels; fibre crops; tobacco; rubber. [Use code 32161 for agro-industries.]

132

LIST OF CRS PURPOSE CODES

31163

Livestock

Animal husbandry; animal feed aid.

31164

Agrarian reform

Including agricultural sector adjustment.

31165

Agricultural alternative development

Projects to reduce illicit drug cultivation through other agricultural marketing and production opportunities (see code 43050 for non-agricultural alternative development).

31166

Agricultural extension

Non-formal training in agriculture.

31181

Agricultural education/training

31182

Agricultural research

Plant breeding, physiology, genetic resources, ecology, taxonomy, disease control, agricultural bio-technology; including livestock research (animal health, breeding and genetics, nutrition, physiology).

31191

Agricultural services

Marketing policies & organisation; storage and transportation, creation of strategic reserves.

31192

Plant and post-harvest protection and pest control

Including integrated plant protection, biological plant protection activities, supply and management of agrochemicals, supply of pesticides, plant protection policy and legislation.

31193

Agricultural financial services

Financial intermediaries for the agricultural sector including credit schemes; crop insurance.

31194

Agricultural co-operatives

Including farmersâ&#x20AC;&#x2122; organisations.

31195

Livestock/veterinary services

Animal health and management, genetic resources, feed resources.

312

FORESTRY

31210

Forestry policy and administrative management

Forestry sector policy, planning and programmes; institution capacity building and advice; forest surveys; unspecified forestry and agro-forestry activities.

31220

Forestry development

Afforestation for industrial and rural consumption; exploitation and utilisation; erosion control, desertification control; integrated forestry projects.

31261

Fuelwood/charcoal

Forestry development whose primary purpose is production of fuelwood and charcoal.

31281

Forestry education/training

31282

Forestry research

31291

Forestry services

313

Including artificial regeneration, genetic improvement, production methods, fertilizer, harvesting.

FISHING

31310

Fishing policy and administrative management

Fishing sector policy, planning and programmes; institution capacity building and advice; ocean and coastal fishing; marine and freshwater fish surveys and prospecting; fishing boats/equipment; unspecified fishing activities.

133

LIST OF CRS PURPOSE CODES

31320

Fishery development

31381

Fishery education/training

31382

Fishery research

Pilot fish culture; marine/freshwater biological research.

31391

Fishery services

Fishing harbours; fish markets; fishery transport and cold storage.

321

Exploitation and utilisation of fisheries; fish stock protection; aquaculture; integrated fishery projects.

INDUSTRY

32110

Industrial policy and administrative management

32120

Industrial development

32130

Small and medium-sized enterprises (SME) development

32140

Cottage industries and handicraft

32161

Agro-industries

Staple food processing, dairy products, slaughter houses and equipment, meat and fish processing and preserving, oils/fats, sugar refineries, beverages/tobacco, animal feeds production.

32162

Forest industries

Wood production, pulp/paper production.

32163

Textiles, leather and substitutes

Including knitting factories.

32164

Chemicals

Industrial and non-industrial production facilities; includes pesticides production.

32165

Fertilizer plants

32166

Cement/lime/plaster

32167

Energy manufacturing

Including gas liquefaction; petroleum refineries.

32168

Pharmaceutical production

Medical equipment/supplies; drugs, medicines, vaccines; hygienic products.

32169

Basic metal industries

Iron and steel, structural metal production.

32170

Non-ferrous metal industries

32171

Engineering

Manufacturing of electrical and non-electrical machinery, engines/turbines.

32172

Transport equipment industry

Shipbuilding, fishing boats building; railroad equipment; motor vehicles and motor passenger cars; aircraft; navigation/guidance systems.

Industrial sector policy, planning and programmes; institution capacity building and advice; unspecified industrial activities; manufacturing of goods not specified below.

Direct support to the development of small and medium-sized enterprises in the industrial sector, including accounting, auditing and advisory services.

134

LIST OF CRS PURPOSE CODES

32182

322

Technological research and development

Including industrial standards; quality management; metrology; testing; accreditation; certification.

MINERAL RESOURCES AND MINING

32210

Mineral/mining policy and administrative management

Mineral and mining sector policy, planning and programmes; mining legislation, mining cadastre, mineral resources inventory, information systems, institution capacity building and advice; unspecified mineral resources exploitation.

32220

Mineral prospection and exploration

Geology, geophysics, geochemistry; excluding hydrogeology (14010) and environmental geology (41010), mineral extraction and processing, infrastructure, technology, economics, safety and environment management.

32261

Coal

Including lignite and peat.

32262

Oil and gas

Petroleum, natural gas, condensates, liquefied petroleum gas (LPG), liquefied natural gas (LNG); including drilling and production.

32263

Ferrous metals

Iron and ferro-alloy metals.

32264

Nonferrous metals

Aluminium, copper, lead, nickel, tin, zinc.

32265

Precious metals/materials

Gold, silver, platinum, diamonds, gemstones.

32266

Industrial minerals

Baryte, limestone, feldspar, kaolin, sand, gypsym, gravel, ornamental stones.

32267

Fertilizer minerals

Phosphates, potash.

32268

Offshore minerals

Polymetallic nodules, phosphorites, marine placer deposits.

323

CONSTRUCTION

32310

331

Construction policy and administrative management

Construction sector policy and planning; excluding construction activities within specific sectors (e.g., hospital or school construction).

TRADE POLICY AND REGULATIONS AND TRADE-RELATED ADJUSTMENT

33110

Trade policy and administrative management

Trade policy and planning; support to ministries and departments responsible for trade policy; trade-related legislation and regulatory reforms; policy analysis and implementation of multilateral trade agreements e.g. technical barriers to trade and sanitary and phytosanitary measures (TBT/SPS) except at regional level (see 33130); mainstreaming trade in national development strategies (e.g. poverty reduction strategy papers); wholesale/retail trade; unspecified trade and trade promotion activities.

33120

Trade facilitation

Simplification and harmonisation of international import and export procedures (e.g. customs valuation, licensing procedures, transport formalities, payments, insurance); support to customs departments; tariff reforms.

33130

Regional trade agreements (RTAs)

Support to regional trade arrangements [e.g. Southern African Development Community (SADC), Association of Southeast Asian Nations (ASEAN), Free Trade Area of the Americas (FTAA), African Caribbean Pacific/European Union (ACP/EU)], including work on technical barriers to trade and sanitary and phytosanitary measures (TBT/SPS) at regional level;

135

LIST OF CRS PURPOSE CODES

elaboration of rules of origin and introduction of special and differential treatment in RTAs.

33140

Multilateral trade negotiations

Support developing countriesâ&#x20AC;&#x2122; effective participation in multilateral trade negotiations, including training of negotiators, assessing impacts of negotiations; accession to the World Trade Organisation (WTO) and other multilateral trade-related organisations.

33150

Trade-related adjustment

Contributions to the government budget to assist the implementation of recipients' own trade reforms and adjustments to trade policy measures by other countries; assistance to manage shortfalls in the balance of payments due to changes in the world trading environment.

33181

Trade education/training

Human resources development in trade not included under any of the above codes. Includes university programmes in trade.

332

TOURISM

33210

Tourism policy and administrative management

400

MULTISECTOR/CROSS-CUTTING

410

General environmental protection

Non-sector specific.

41010

Environmental policy and administrative management

Environmental policy, laws, regulations and economic instruments; administrational institutions and practices; environmental and land use planning and decision-making procedures; seminars, meetings; miscellaneous conservation and protection measures not specified below.

41020

Biosphere protection

Air pollution control, ozone layer preservation; marine pollution control.

41030

Bio-diversity

Including natural reserves and actions in the surrounding areas; other measures to protect endangered or vulnerable species and their habitats (e.g. wetlands preservation).

41040

Site preservation

Applies to unique cultural landscape; including sites/objects of historical, archeological, aesthetic, scientific or educational value.

41050

Flood prevention/control

Floods from rivers or the sea; including sea water intrusion control and sea level rise related activities.

41081

Environmental education/ training

41082

Environmental research

430

Including establishment of databases, inventories/accounts of physical and natural resources; environmental profiles and impact studies if not sector specific.

Other multisector

43010

Multisector aid

43030

Urban development and management

Integrated urban development projects; local development and urban management; urban infrastructure and services; municipal finances; urban environmental management; urban development and planning; urban renewal and urban housing; land information systems.

43040

Rural development

Integrated rural development projects; e.g. regional development planning; promotion of decentralised and multi-sectoral competence for planning, co-ordination and management;

136

LIST OF CRS PURPOSE CODES

implementation of regional development and measures (including natural reserve management); land management; land use planning; land settlement and resettlement activities [excluding resettlement of refugees and internally displaced persons (72010)]; functional integration of rural and urban areas; geographical information systems.

43050

Non-agricultural alternative development

Projects to reduce illicit drug cultivation through, for example, non-agricultural income opportunities, social and physical infrastructure (see code 31165 for agricultural alternative development).

43081

Multisector education/training

Including scholarships.

43082

Research/scientific institutions

When sector cannot be identified.

500

COMMODITY AID AND GENERAL PROGRAMME ASSISTANCE

Note: Sector specific programme assistance is to be included in the respective sectors, using the sector programme flag if appropriate.

510

General budget support

Budget support in the form of sector-wide approaches (SWAps) should be included in the respective sectors.

General budget support

Unearmarked contributions to the government budget; support for the implementation of macroeconomic reforms (structural adjustment programmes, poverty reduction strategies); general programme assistance (when not allocable by sector).

51010

520

Developmental food aid/Food security assistance

52010

Food aid/Food security programmes

Supply of edible human food under national or international programmes including transport costs; cash payments made for food supplies; project food aid and food aid for market sales when benefiting sector not specified; excluding emergency food aid.

Other commodity assistance

Non-food commodity assistance (when benefiting sector not specified).

53030

Import support (capital goods)

Capital goods and services; lines of credit.

53040

Import support (commodities)

Commodities, general goods and services, oil imports.

530

600

ACTION RELATING TO DEBT

60010

Action relating to debt

60020

Debt forgiveness

60030

Relief of multilateral debt

60040

Rescheduling and refinancing

60061

Debt for development swap

Allocation of debt claims to use for development (e.g., debt for education, debt for environment).

60062

Other debt swap

Where the debt swap benefits an external agent i.e. is not specifically for development purposes.

Actions falling outside the code headings below; training in debt management.

Grants or credits to cover debt owed to multilateral financial institutions; including contributions to Heavily Indebted Poor Countries (HIPC) Trust Fund.

137

LIST OF CRS PURPOSE CODES

60063

Debt buy-back

Purchase of debt for the purpose of cancellation.

700

HUMANITARIAN AID

Within the overall definition of ODA, humanitarian aid is assistance designed to save lives, alleviate suffering and maintain and protect human dignity during and in the aftermath of emergencies. To be classified as humanitarian, aid should be consistent with the humanitarian principles of humanity, impartiality, neutrality and independence.

720

Emergency Response

An emergency is a situation which results from man made crises and/or natural disasters.

72010

Material relief assistance and services

Shelter, water, sanitation and health services, supply of medicines and other non-food relief items; assistance to refugees and internally displaced people in developing countries other than for food (72040) or protection (72050).

72040

Emergency food aid

Food aid normally for general free distribution or special supplementary feeding programmes; short-term relief to targeted population groups affected by emergency situations. Excludes non-emergency food security assistance programmes/food aid (52010).

72050

Relief co-ordination; protection and support services

Measures to co-ordinate delivery of humanitarian aid, including logistics and communications systems; measures to promote and protect the safety, well-being, dignity and integrity of civilians and those no longer taking part in hostilities. (Activities designed to protect the security of persons or property through the use or display of force are not reportable as ODA.)

Reconstruction relief and rehabilitation

This relates to activities during and in the aftermath of an emergency situation. Longer-term activities to improve the level of infrastructure or social services should be reported under the relevant economic and social sector codes. See also guideline on distinguishing humanitarian from sector-allocable aid.

Reconstruction relief and rehabilitation

Short-term reconstruction work after emergency or conflict limited to restoring pre-existing infrastructure (e.g. repair or construction of roads, bridges and ports, restoration of essential facilities, such as water and sanitation, shelter, health care services); social and economic rehabilitation in the aftermath of emergencies to facilitate transition and enable populations to return to their previous livelihood or develop a new livelihood in the wake of an emergency situation (e.g. trauma counselling and treatment, employment programmes).

Disaster prevention and preparedness

See codes 41050 and 15220 for prevention of floods and conflicts.

Disaster prevention and preparedness

Disaster risk reduction activities (e.g. developing knowledge, natural risks cartography, legal norms for construction); early warning systems; emergency contingency stocks and contingency planning including preparations for forced displacement.

730

73010

740

74010

910

ADMINISTRATIVE COSTS OF DONORS

91010

920

Administrative costs

SUPPORT TO NON- GOVERNMENTAL ORGANISATIONS (NGOs)

Official funds to be paid over to national and international private voluntary agencies for use at the lattersâ&#x20AC;&#x2122; discretion.

92010

Support to national NGOs

In the donor country.

92020

Support to international NGOs

92030

Support to local and regional NGOs

In the recipient country or region.

138

LIST OF CRS PURPOSE CODES

930

REFUGEES IN DONOR COUNTRIES

93010

998

Refugees in donor countries

UNALLOCATED/ UNSPECIFIED

99810

Sectors not specified

Contributions to general development of the recipient should be included under programme assistance (51010).

99820

Promotion of development awareness

Spending in donor country for heightened awareness/interest in development co-operation (brochures, lectures, special research projects, etc.).

139