EUROPEAN COMMISSION EuropeAid Co-operation Office EuropeAid
Results-Oriented Monitoring of EC External Assistance Guidelines for Monitors for Regional Programmes
December 2008 ROM Coordination: EuropeAid/E5
Project Title
Coordination of the Results-Oriented Monitoring System of Projects and Programmes of External Assistance financed by the European Community -
Reference
EuropeAid EVA / 141 - 048
Contract Supervisor
Unit E5
Contractor
in association with ATOS Origin, Development Associates, QUEST Consult, South Research Date of report
16th December 2008
Report authors
Mary Hall, based on Study by Alain Sancerni, Veronique Girard and Serani Siegel.
i
TABLE OF CONTENTS TABLE OF CONTENTS ...................................................................................................... I List of Figures.................................................................................................................. i List of Acronyms ............................................................................................................ ii 1. INTRODUCTION.......................................................................................................... 2 1.1. Context ................................................................................................................... 2 1.2. Study Purpose, Approach and Results ............................................................... 2 1.3. Key Recommendations......................................................................................... 2 1.4. Guideline Structure................................................................................................ 3 2. DEFINITION AND CHARACTERISTICS OF RP ...................................................... 4 2.1. Definition of RPs .................................................................................................... 4 2.2. Added Value of the Regional Dimension of RPs................................................ 4 2.3. Risks and Weaknesses of RPs ............................................................................ 5 2.4. What to Monitor and How to Monitor RPs........................................................... 5 3. WHAT TO MONITOR – ROM RP CLASSIFICATION AND TOOLS....................... 5 3.1. Typology of Intervention Logic ............................................................................. 5 3.2. The BCS ................................................................................................................. 8 3.3. The Grading Process. ........................................................................................... 8 4. HOW TO MONITOR – THE PLANNING AND IMPLEMENTATION OF AN RP MISSION. ......................................................................................................... 9 4.1. Typology by RP Configuration.............................................................................. 9 4.2. Planning Phase...................................................................................................... 9 4.3. ROM for RP Implementation .............................................................................. 11 4.4. ROM Deliverables for RP ................................................................................... 12 4.5. Report Writing ...................................................................................................... 12 4.6. Quality Control ..................................................................................................... 13 4.7. Dissemination and Follow-up ............................................................................. 13 4.8. Encoding of Reports............................................................................................ 13 4.9. Role of the EC Delegations and HQ.................................................................. 14 4.10.Monitoring Team ................................................................................................. 14
List of Figures Figure 1 Conceptual diagram of the regional intervention logic typology ..................................................... 6
Guidelines for Monitors for Regional Programmes 2008
ii
List of Acronyms ACP BCS CMTP DCI EDF ENPI MERCOSUR MR NAO NIP OO PC PP PS RIP RM RS SAARC
Africa – Caribbean – Pacific states (under EDF fund) Background Conclusion Sheet Centrally Managed Thematic Programmes (DCI) Development Co-operation Instrument European Development Fund (Cotonou Agreement) European Neighborhood and Partnership Instrument Common Market of the South (“Mercado Comun del Sur”) Monitoring Report National Authorizing Officer (ACP countries) National Indicative Programme Overall Objective Partner Country Project Purpose Project Synopsis Regional Indicative Programme Regional Monitor Response Sheets South Asian Association for Regional Co-operation
Guidelines for Monitors for Regional Programmes 2008
2
1. INTRODUCTION 1.1. Context ROM was originally developed for national projects which were managed from headquarters. However three developments: the increasing number of regional projects and programmes (RP) , the deconcentration of most projects and programmes to delegations and a diverging approach across Regions/Lots to the monitoring of RP have consequences for the traditional ROM . Furthermore an internal audit on ROM made two relevant recommendations: •
Recommendation N° 8 – Coverage of ROM: - Due to the inherent high risk related to the implementation and project structure, an enhanced coverage of RPs through the ROM shall be ensured. - Monitor planning and implementation of ROM missions. - Further involve EC Delegations and HQ in ROM of Regional Projects.
•
Recommendation N° 9 – ROM methodology for Regional Projects. - Complement current ROM methodology. Given the varying character of RPs, ranging from a rather loose basket of fairly independent operations to clusters of very-interdependent country components i n regional integration programmes and the sometimes small size of their components, monitoring them can pose a real challenge to both the relevant Directorates and regional ROM contractor s. 1.2. Study Purpose, Approach and Results This guideline is based on a very extensive study 1, the purpose of which was to review the current RP ROM vis-à-vis its appropriateness for monitoring RP. It also involved identifying the key regional elements of the different types of RP in order to establish what exactly to monitor in order to capture this regional dimension. This required some adaptations to the ROM methodology. At the same time there was a need to propose some degree of harmonisation/standardisation on how RP are monitored in terms of approach; days spent, numbers o f countries visited etc. The study combined information from a series of interviews with Task Managers from the relevant geographical units of AidCo and key personnel from each of the ROM contractor teams with a desk review of different MR from a wide var iety of RPs. ROM Coordination and the commissioned expert held a workshop based on the data gathered in order to agree an RP typology and from that to derive the various approaches of ROM for RP . 1.3. Key Recommendations The study proposed the following key re commendations to address the requirements of the Court of Auditors recommendations :
1
Amendment of ROM Methodolo gy for the Regional Programmes and Centrally Managed Thematic Programmes. This was disseminated to all geo-coordinators, ROM contractors and 47 EC Delegations of the Aid Effectiveness Network in September 2008
Guidelines for Monitors for Regional Programmes 2008
3
• • • • • •
Essentially to maintain the main principles of the standard ROM methodology and tools for the monitoring of RP but to take more adequately into consideration their regional dimension. To involve all the concerned ECD, not just the “chef du file”, more in the ROM missions, especially in the preparation and implementation of the monitoring missions ( e.g. in the sampling, systematic visits to the ECD). To introduce an identification process based on the RPs’ typologies to establish the appropriate monitoring approach. To introduce desk phases for certain categories of RP to ensure proper monitoring coverage To draw up a list of general points to cover when completing the BCS in order to cover the regional dimension To clarify the role of the Consolidated Regional MR, i.e. the horizontal MR which gives the overview of the programme , from the regional perspective viz a viz the Component MR, which provides information on a component or country element of the regional project from a more local perspective. In particular the grading mechanism for the different types of reports needs to be paid close attention to. All MR are produced on the standard MR template for ongoing projects.
These guidelines can also be used for ex -post ROM monitoring of RP. If the ex -post methodology is adapted in the future then all adaptations will also apply to RP. 1.4. Guideline Structure These key recommendations have been summarised and simplified to crea te this guideline which comprises 4 further chapters. • • •
Chapter 2 looks at RP, their definitions, added value, risks and weaknesses and consequences for the current ROM system. Chapter 3 presents what to monitor - the aspects or RP that need to be captured by the ROM exercise Chapter 4 presents how to monitor –the process of planning and implementing a RP monitoring.
The intention of this guideline is to avoid creating new templates or changing the existing methodology, rather it is to give a perspective t hrough which to approach the monitoring of an RP in order to capture the necessary regional dimension. This guideline has not taken the specifics of CMTPs into consideration. However to the extent that many of them have similar characteristics to other RP s this guideline should provide information useful for their monitoring.
Guidelines for Monitors for Regional Programmes 2008
4
2.
DEFINITION AND CHARA CTERISTICS OF RP 2.1. Definition of RPs
As a first definition, according to Regulation No 1905/2006 of the European Parliament and of the Council of 18 December 2006 establishing a financing instrument 2 for development cooperation – (Official Journal of the European Union) a region is defined as a “geographical entity comprising more than one developing country”. In terms of geographic parameters EC cooperation is div ided into global regions such as ACP countries (Africa, Caribbean and Pacific), ENPI (Eastern Europe and Mediterranean), Asia, Latin America, and the Gulf region. Within each region there are sub -regions e.g.: West Africa, South Asia (SAARC), Andean Community & Mercosur. 2.2. Added Value of the Regional D imension of RPs It is important that the ROM of RPs reports accurately on the regional dimension of the programmes and in particular reports on the intended added values in the programme. The primary added value functions of an RP are related to its objectives and impact, for instance: • Regional integration. • Promotion and optimisation of common resources and capacities . • Solving a common problem (water, environment, migrations…) Secondary added value functions of the regional dimension of an RP are related to cost -efficiency and synergies (efficiency and effectiveness). The added value of the regional dimension of a n RP to the partner countries can include one or more of the following elements: • Setting-up common standards and models, with possible national modifications. • Facilitating exchanges of resources, goods, c apacities benefitting each country . • Building up common, similar or complementary capacities for national development. • Establishing regional institutio ns, resources and dynamics, markets, etc. • Saving costs at national level . • Improving governance, planning, policy. The added value of the partner countries ’ dimension to the region (bottom up) could be: • Additional national resources and capacity (experienc e, models…). • Complementary national programmes and policy, in the same or related sectors . • Favourable national policies: adjustment of the weakest models and practices to the most advanced and dynamic.
2
The Commission’s Financing Instruments can be found at:
http://ec.europa.eu/europeaid/what/delivering -aid/funding-instruments/index_en.htm
Guidelines for Monitors for Regional Programmes 2008
5
2.3. Risks and Weaknesses of RPs Regional ROM will have to take into account in its methodology and approach the specific r egional dimension of the RP which brings with it the following possible inherent risks and weaknesses at various levels: •
Lack of ownership as Partner Countries and EC Delegations or HQ often find it difficult to be fully involved and thus responsible for RP as they may only have fragmented information and minimal input .
•
Lack of coherency between regional and national programmes/policies: RP are not sufficiently supported by national interests o r programmes in related sectors or areas. Low/limited co-ordination between HQ and ECD and between different ECDs concerned in the same RP. Lack of co-ordination between similar EC programmes or between EC and other Donors ’ similar or complementary RP. Differences in capacity levels between countr ies in the same region: the partner countries may not have the same capacities either to contribute or to absorb new resources . Weakening of the regional dimension of the programme as due to various factors e.g. weak design, low management, excess ive work, political nationalism etc. the regional dimension fades away, and the national activities and interests prevail exclusively
• • • •
2.4. What to Monitor and How to Monitor RPs Building a comprehensive and operational methodo logy for monitoring RP requires first a clear classification by typology of these programmes, taking into account the main factors that define and distinguish them. The rest of this guide deals concretely with what to focus on when monitoring a RP and how to go about it. • The typology by intervention logic has an impact on ROM in terms of what should be monitored as this depends on the regional quality or dimension of the RP •
The typology by RP configuration only has an impact on the ROM in terms of how it should be done logistically and operationally.
3. WHAT TO MONITOR – ROM RP CLASSIFICATION AN D TOOLS 3.1. Typology of Intervention Logic Through analys ing many RPs it appears that they can essentially be divided into 4 qualitative categories, independently from the sources of funding or from the geographical dimension of the programme: • Exclusively RP (categories A and B) • Hybrid RP (category C), • Pseudo-RP (category D) This broad categorization is presented below in diagrammatic form with examples. T he horizontal axi s is the objectives continuum from national to regional focus and the vertical axis is the implementation continuum from full national implementation to no national implementation.
Guidelines for Monitors for Regional Programmes 2008
6
For the purpose of explanation the categorisation is greatly simplified an d in fact the categories are not so distinct, a project may move from one to another during its implementation , usually from A or B to C. This is often due to the deterioration of its regional dimension. The different characteristics of each typology in terms of objective, purpose, results, activities, impact and sustainability are summarised below. A checklist is provided in Annex 1 to decide which typology an RP most closely fits.
Figure 1 Diagram of the regional intervention log ic typology National implementations
- Water Facility - Programme of business cooperation with the EU Asia Invest, Water and Energy facility, URB-AL
D
- Environmental programmes (tourism, regional natural parks) EUMEDIS, EUROSOLAR
Integrated programme for Rum industry in Caribbean
ECOFAC
C
National GO/SO
- Pipeline - Regional road - Environmental programme (against pollution)
B
Environmental Collaboration for the Black Sea, EUBAM
Regional GO/SO
- Regional network programme - Support to regional institutions EUMEDCONNECT, MERCOSUR, Appui technique et institutionnel a la CEMAC
A No national implementations
Guidelines for Monitors for Regional Programmes 2008
7
3.1.1. Exclusively RP (categories A and B) In these cases: • • •
• •
The overall objective (OO) and project purpose (PP) are exclusively regional: they aim at reinforcing regional integration, developing regional capacities or addressing a regional problem. The expected results are regional and directed towards regional capacities or benefit s, with secondary national effects/achievements . The direct impact of the program me is regional e.g. regional integration, development of regional capacities, new improved regional situation. But the national and local impact s are also necessarily taken into account . Thus, as a consequence of each country reaching its own specific objectives the regional objective is also reached. The sustainability of the programme is regional: either through institutional sustainability e.g. by setting up a permanent regional institution or network, funding sustainability, or through results and benefits sustainability e.g. the development of regional capacity, resources. Success in all involved countries is necessary to the achievement of the programme
There are 2 scenarios for exclusively RP, depending on the programme construction and activities: A - Activities are exclusively regional: • The programme has no “national” activities or components . • Activities cover the global interests of the whole of the partners’ countries, but may not be the same in each country. For example a regional training centre may be set up in one country but it is to benefit all the R P partner countries. The logic of the programme and of its components is purely horizontal. B- Activities are regional and national (or even only national): • All the national activities are necessary but may be different as they aim to achieve the regional OO and PP. All are absolutely necessary and interdependent (example: a pipeline crossing several countries, some environment programmes…).The failure or the weakness of only one country has negative consequences on the whole of the program me. 3.1.2.
RP with autonomous national components (hybrid programmes, category C)
In these cases: • • • •
The OO is fully regional, but is partially based on the reinforcement of the national capacities. The OO is first and foremost regional and subsequently national. Essentially th e focus is reinforcement/promotion of regional capacities and resources, but this is done partly through strengthening the national capacities rather than creating purely regional capacities. Results are both regional and national in various proportions. The whole of the national results, added to the regional results, reinforces the region. Activities are: − National: they are carried out in each country and aim at national and regional benefits.
Guidelines for Monitors for Regional Programmes 2008
8
•
•
− Regional: they are shared by partner countries (training, net works, capacity building). They coordinate and complement national activities. The impact is regional and national: − The programme has a specific regional impact (regional integration, promotion…) from regional and national activities. − It has national impacts, as the countries and populations are the ultimate beneficiaries. Sustainability: − Institutional sustainability: when regional institution or coordination mechanisms (networks, consultation processes) have been established. − Benefits sustainability: regional and national (populations, national policies and capacities). 3.1.3. RP without regional dimension (pseudo -regional, category D):
This category consists of programmes with regional financing but without a regional dimension . Essentially the regional fund is only used for “opportunist” local or national activities . In some cases, the regional fund is used to compensate for the absence of a crucial sector in the national indicative programme (NIP). Each beneficiary country develops its own autonomous programme without regional considerations. Sometimes a regional programme results from the opportunist gathering “telephone child” of 3 countries of a region so as to respect the rules of RP but with no real regional content. In some cases, “efficiency” i.e. s ynergy, cost efficiency etc, may be the only regional criterion or added value. The programme should be then considered as “national”, as its objectives are actually national Thus, the "standard" (national) ROM can be used without consideration of regional added value. 3.2. The BCS It is at the level of methodology where this guidelines makes several new proposals in order to capture the regional aspect of RPs. Essentially, the latest version of the BCS (2008) is applicable and it has not been further amended but a list of general points to cover depending on the RP’s typology of intervention has been generated in order to interpret the BCS issues in a regional way. These general points are presented in Annex 1. It is for the monitors, the Mission Leader in parti cular, to decide when this is useful and to incorporate the answers in to the most relevant sub -criteria within the classic BCS . Essentially the purpose of these points is to capture the value added of RPs (see 2.2) and see if risks and assumptions are holding true (see 2.3). 3.3. The Grading Process. To date the main problem in grading of RP is that there has not been a clear differentiation between the types of RP where poor performance in certain DAC criteria has consequences for the entire RP (as in typologies A and B) and type C RPs where the consolidated report could be more or less and average across the countries involved. Type D RPs often do not warrant a conso lidated report unless there is a requirement to look at the efficiency of the RP approach to th e component included within it or for any other possible synergies. This has now been addressed and Annex 1 provides clear guidance on the grading process for the consolidated reports as well as for the individual reports related to any particular RP.
Guidelines for Monitors for Regional Programmes 2008
9
4. HOW TO MONITOR – THE PLANNING AND IMP LEMENTATION OF AN RP MISSION. 4.1. Typology by RP Configuration Essentially an RP may cover very different numbers of countries closely or widely dispersed which has its own consequences for the logistics of monitoring an RP. T hese different configurations are presented below and examples are given in Annex 4 Configuration a: These are RP covering a small cluster of countries (from 1 to 4 – figures are indicative) over a limited geographic area. Configuration b: RPs including a cluster of 6-8 countries in the same region e.g. MEDA, Africa or African sub-region, or in a more extended area. Configuration c: RPs including a large group (around 15) of countries within the same region. Others: Further configurations are possible su ch as: • RP including series of scattered countries over 2 or 3 regions i.e. with no regional coherence, but a thematic one. • Special case: regional programme under “Contribution Agreements”, characterised by multi Donors’ co-financing (international organiza tions, partner countries…), either through a joint or a parallel fund. 3 “Other” configurations need to be dealt with on a case by case basis. A standardised approach to the planning and implementation of RPs is proposed in this chapter to establish some consistency in the way diffe rent types of RP are monitored . 4.2. Planning Phase. As with all ROM monitoring the planning phase involves identifying the projects and programmes to be included within a particular planning cycle. 4.2.1. Identification and classification In regard to RP there needs to be an additional identification step during which the ROM contractor in Brussels (or in the regional offices, e.g. ENPI East) classifies the RP according to the two different typologies: the typology by intervention logic (i.e. A, B, C or D?) and by configuration (i.e. a, b. c, others?). Essentially it is expected that this can be done as a one-off exercise which then only needs to be checked annually in case the programme has officially changed its focus. 1) Financial instrument This basic step identifies, if the project/programme or component to monitor is formally using a geographic instrument with a regional dimension . See http://ec.europa.eu/europeaid/what/delivering -aid/funding-instruments/index_en.htm
3
EC Strategy is currently promoting this configuration which has considerable effects on the ROM methodology. In the case of parallel funding, the EC contribut ion and activities can easily be identified and assessed. This is not possible in case of joint funding and the ROM has to deal with the global situation of the programme. The ROM implementation needs also a sound knowledge of donors and partners’ methodol ogy, as well as a strong capacity of dialogue.
Guidelines for Monitors for Regional Programmes 2008
10
2) Typology by intervention logic Identifying the typology of the RP helps to gives guidance on what to focus on during monitoring exercise in order to make sure that the regional dimension is properly covered. See previous chapter. The typology identified should be the type th e RP was designed to have as per the Financing Agreement rather than what it has become, unless such changes have been officially accepted through an addendum. For the purpose of missions there is no difference between typology A and B 3) Need for sampling The need for sampling depends very much on the number of countries and their proximity/distance to/from each other. It also depends on the typology of the RPs as for some it is more important to have full coverage than for others – this will be covered more fully in 4.2.2 4) Need for a desk study Similarly the identification phase will show if a desk phase i.e. an exercise which, from desk research, endeavours to gain an impression of how the RP is performing in other countries , is necessary. This is most likely in RPs where sampling takes place or which cut across different regions . More details are given in 4.2.3 In summary the completed matrix shown in Table 1 gives all the potential variants of RPs : Table 1 – The RP Variant Matrix Name of Regional Fi nancing Instrument Typologies a (1-4 countries)
A/B No sampling
C No sampling
D No sampling Each country/project could be included in a ROM standard national mission
b (6-8 countries)
Ideally no sampling. If sampling is necessary then a desk phase should really focus on the regional aspect of the programme and cover ALL missing countries
c (9+ countries)
Unlikely to be many Unlikely to be many in this category, in this category. Monitor as variant b
Possible sampling No sampling and Desk Phase if Each country/project time and budget could be included in allow a ROM standard national mission
Monitor as variant b
Guidelines for Monitors for Regional Programmes 2008
No sampling Each country/project could be included in a ROM standard national mission
11
4.2.2. Sampling In cases where the identification phase has pro posed a sampling approach , almost always in the case of typology “C” RP the ROM contractor propose s a selection of countries/components to be monitored. This should be done in agreement with the ROM Task Manager in coordination with the geocoordinator and the delegations involved in the RP. The sampling should be based on the following principles: • The sample should select countries/projects/components which give a comprehensive overview of the overall programme. • It should take into account the future possib ility of re-monitoring, which can either be done with the same sample or with a different sample. • It should not be driven by, but take into account logistical, budgetary and specific requests and other “realism meets methodology” reasons. • Cover the different types of components or results of the RP , if any e.g. sometimes, like Asia Invest, you have: capacity building, technical assistance, training, trade fairs, etc….. 4.2.3. Desk phase 4 In certain cases, especially typology C programmes, a desk phase is necessar y when not all countries can be visited. 2-5 days should be allocated for this task depending on the complexity of it. It should provide complementary information on the non -visited countries from desk research. If further important information can be gain ed from a few key emails or phone calls to the key stakeholders and implementers then this should be done. Ideally the mission leader should undertake this exercise before the mission takes place and be provided as background or briefing to all involved m onitors to clearly establish the context of their monitoring exercise by indicating what is going on elsewhere. Most importantly the desk phase feeds into the consolidated report. The desk phase for one year could help identify the countries for monitoring in subsequent years 4.3. ROM for RP Implementation The conventional approach of ROM works reasonably well for RP. Nevertheless it is necessary to make sure that sufficient components are properly monitored to permit the compilation of an accurate consolidated report. At the same time it is accepted that there is a need for flexibility in approach to ensure cost-effectiveness. 4.3.1. When to monitor Again flexibility is possible but the Court of Auditors recommendation was that coverage should be increased:
4
During the study that forms the basis of these guidelines the idea was also mooted that a Desk Study covering regional issues which looked at variances in capacities, common difficulties, areas of divergence , in general a regional overview to give a context for all the ROM activities would be very valuable. The idea remains valid but is not directly a part of these guidelines.
Guidelines for Monitors for Regional Programmes 2008
12
• •
Usual frequency seems to be appropriate: 1 monitoring per year, and according to the demand of the Task Manager as indicated in the RS. In cases of pseudo -RP, category D including a multi-country scheme, b or c: clusters of countries can be separately monitored throughout the year, on a regional or on a multi national basis, with a consolidated report derived on a cost-effectiveness basis when several reports have been produced . 4.3.2. Mission Structure
The following table shows the recommended mission structure to cover the different types of RPs in order to properly capture the regional aspect. Table 2 Mission Structure Approach Typologies
A/B
C
D
a (1-4 countries)
1 mission all 1 mission countries visited. countries visited
all Structure the missions on the mo st cost effective basis
b (6-8 countries)
2 missions to cover 1-2 mission to cover all countries sample plus desk phase
c (9+ countries)
Unlikely, Monitor as Unlikely, Monitor as As above variant b variant b
As above
4.4. ROM Deliverables for RP In several regions RPs tended to rely quite extensively on Monitoring Notes. This practice is no longer acceptable as it easily results in “lost” information to the ROM system. From now on the deliverables are: • 1) 1 MR produced per country/component (PS, MR, and BCS). • 2) 1 Consolidated Regional MR produced (PS, MR, BCS) by the Mission Leader for A , B and C type RPs. A Consolidated Regional MR should be produced for type D only if requested . If there has been any desk analysis it should be incorporated in the BCS and MR of the consolidated report. The role of the Consolidated Report is very important and the focus of its content varies according to the typology of the RP. 4.5. Report Writing The national MR should be the usual length – 2 pages. However the consolidated MR could be longer but should still be concise and to the point – maximum 4 pages.
Guidelines for Monitors for Regional Programmes 2008
13
4.6. Quality Control Although the changes introduced for the monitoring of RP are quite limited in number their consequence for reporting to ensure that the regional dimension is p roperly covered are quite significant. It is important that quality control plays a role in ensuring a consistent approach towards the different typologies of MR and in particular that the C onsolidated MR capture s the added value or lack of added value of the regional dimension , as well as during the encoding in CRIS (see 4.8) . In addition care should be taken with the figures put in the financial section of the MR. The Consolidated MR should have financial data based on the total value of the RP and the country/component MRs should reflect their relevant part of the financial data. Where this cannot be identified from CRIS the first section of text in the country/component MR should indicate that total RP figures are given. 4.7. Dissemination and Follow-up The Country/Component MR, PS and BCS should be disseminated to all the usual EC parties; the Task Manager and the Delegation Advisor . The ECDs should then disseminate to the relevant TA and key players in the Partner Governments as appropriate . The Consolidated MR, PS and BCS should be disseminated to all those involved in the RP . Through the new CRIS online response sheets all involved delegations will have the opportunity to comment on the reports and the “Delegation chef de file "should take the lead in the c onsolidation of recommendations, remarks and follow up. 4.8. Encoding of Reports The new search function on CRIS permits all reports related to an RP to be identified. 5 However to date a variety of approaches have been used by different contra ctors to the production of the consolidated reports such that there is no consistent way of identifying what is a consolidated report and what is a component report. For the future: • All report which relate to a specific country element of an RP should be encoded with the country name. • Every consolidated report should be encoded under the regional geographic zone. • Where components are also regional in nature they should be encoded under the regional geographical zone There still remains the issue of how to identify a C onsolidated MR as opposed to a Component MR and this will be addressed in early 2009 and is likely to involve the ticking of a relevant box for a Consolidated MR by pressing a button when encoding.
5
This search function is described in the searching guidelines which are currently in draft form.
Guidelines for Monitors for Regional Programmes 2008
14
4.9. Role of the EC Delegations and HQ To make RP monitoring as usef ul as possible and to address a Court Of Auditors recommendation to “Further involve EC Delegations and HQ in ROM of Regional Projects” there needs to be a higher level of involvement of the EC, especially those delegations which do not have a direct role in the management of the RP. Whilst acknowledging that there are always time and workload constraints certain steps can be taken such as: • Ensuring that all relevant delegations are kept in copy during the setting up of the monitoring mission, including the ir views in any desk phase. • Including a visit to the delegation whenever an element of an RP is visited in their country both for briefings and debriefings . • Making sure that the MR s are copied to all delegations in countries where the RP is active and that they have the chance to comment on it (RS) . 4.10.
Monitoring Team
It is essential that experienced monitors are used for this exercise, particularly in the case of the Team Leader. It is very complex to clearly write down in a guideline the difference in focus that needs to be kept to collect both the national and the regional dimensions. Much depends on specific circumstances and it is only possible to draw up general guidelines therefore the monitors have to use the ir experience and discuss the reports, especi ally the consolidated report with the colleagues that have monitored other elements of the same RP.
Guidelines for Monitors for Regional Programmes 2008
15
ANNEX 1: TYPOLOGY OF INVERTENTION LOGIC CHECKLIST
CATEGORY A / B Programmes exclusively regional / “Purely regional programmes” :
• The Global and Specific objectives of the programme are regional • Success in all involved countries is necessary to the achievement of the programme There are two possible types:
• The programme does not include national components (B) • The programme includes national components that are completely concentrated on regional objectives. (A) CATEGORY C Regional programmes with national dimensions / “Hybrid Programmes” : This category includes a lot of different cases which spread out between types A/B and D
• Global and specific objectives are regional • The RP includes national components and has some national activities and results • This type of RPs have national and regional impacts • The national objectives of the programme tend to prevail over the regional ones. CATEGORY D Regional programmes without regional dimension / “Opportunist programmes” :
• The programme only includes national components • The global objectives are regional but specific objectives are national. The national or local programmes don’t have any horizontal consistency or objectives.
Guidelines for Monitors for Regional Programmes 2008
16
ANNEX 2: GENERAL POINTS TO CONSIDER WHEN C OMPLETING AND GRADIN G THE BCS OF A RP
CRITERIA Relevance
Typology A – no National Activities
Grading
Has the project a regional relevance (i.e. independent from national intere sts)?
Consolidated Report Grading ( CRG) prevails as relevance is essentially regional.
Have all Partner Countries (PC) equal interest in the project?
National Report Grading (NRG) is indicative, and may show differences in national interests or importance of the project.
Have they an equal commitment? Are the national policies coherent with the project? Are all PC equally necessary to the project achievement?
Efficiency
Is the regional approach cost effective? Is the regional organization appropriate, wel l designed (institutional arrangements, coordination…)? Do all the PC duly contribute to the project?
Effectiveness
Are the regional results achieved? Does (to which extent) each PC equally benefit from/use the regional results
Impact
Regional impact of the project Impact (positive or negative) of the project on each country?
Sustainability
NRG - the national /component view of the regional level of efficiency as there are no national activities CRG - effectiveness is essentially regional. NRG - the national /component view of the regional level of effectiveness as there are no national activities CRG - impact is essentially regional.
6
Sustainability of a regional institution / mechanism /policy… Sustainability of regio nal results/impacts Sustainability of national policy, services or institutions Follow-up and coordination mechanisms
Risks
CRG – efficiency is essentially regional.
NRG essentially the national/component view of the regional level of impact as there are no national activities . But if there are cases where there is a perceived negative impact which could result in reduced commitment then it should be noted. CRG - sustainability is essentially regional. NRG - the national/component view of the regional level of sustainability taking into account national commitment, policies and financial participation
- Some PCs are less or not involved: particularly because of the absence of national components. - ECD are not involved, committed, coordinated - Regional conflict
6
Political, economic, social impact may be variable according to countries. National impact is dissociated from Regional impact
Guidelines for Monitors for Regional Programmes 2008
17 - Lack of coordinatio n (capacity) - Regional coordination is costly (transport…) - Low visibility at national and regional levels
CRITERIA Relevance
Typology B – some national activities Has the project a regional relevance (i.e. independent from national intere sts)? Have all PC an equal interest in the project? Have they an equal commitment?
Grading CRG prevails. NRG is indicative, and may show differences in na tional interests or importance of the project.
Are the national policies coherent with the project? Are all PC equally necessary to the project achievement?
Efficiency
Is the regional approach cost effective? Is the regional organization appropriate, well designed (institutional arrangements, coordination…)? Do all the PCs duly contribute to the project?
Effectiveness
Are the regional results achieved? Are the national results achieved? Do the PC equally contribute to the regional results?
CRG syn thesis from a regional perspective not an average of national/component gradings. NRG are indicative, and may show where stren gths and weaknesses are located in different countries or components CRG takes the lowest grade of all the national/component grades as a weakness in one element is a n overall weakness from the regional perspective. NRG looks at effectiveness prim arily from a national/component perspective
Impact
Regional impact of the project Impact of the project on each country? (1) National impact of national components?
Sustainability
Sustainability of a regional institution / mechanism /policy… Sustainability of regional results/impacts Sustainability of national components, policy, services or institutions Follow-up and coordination mechanisms
Risks
CRG synthesis from a regional perspective not an average of national/component gradings. NRG looks at impact primarily from a national/component perspective . CRG takes the lowest grade of all the national/component grades as a weakness in one element is a n overall weakness from the regional perspective. NRG looks at impact primarily from a national/component perspecti ve.
- Some PC are less or not involved - ECD are not involved, committed, coordinated - Regional conflict
Guidelines for Monitors for Regional Programmes 2008
18 - Lack of coordination (capacity) - Regional coordination is costly - Low visibility at national and regional levels
CRITERIA Relevance
Typology C
Grading
Has the project a regional relevance, additional with national interests?
CRG – a mix of the national/component findings with a regional perspective – not necessarily the average of NRG
Is the project relevant for all PC? Are the national components relevant both at national and regional levels? Are PC committed to both levels?
NRG – from a national/component perspective.
Coherence between national components/ policies and regional component/dimension?
Efficiency
Is the regional appr oach cost-effective? Is the regional organization appropriate, well designed (institutional arrangements, coordination…)?
CRG – a mix of the national/component findings with a regional perspective – not necessarily the average of NRG NRG – from a national/ component perspective determined by project performance within the country .
Effectiveness
Effectiveness at regional level Effectiveness at national (components) level How do the regional results support the national projects/activities? How do the nation al activities contribute to the regional results?
Impact
Regional impact of the project Impact of the regional impact at national level Impact of the national components on national level
Sustainability
Sustainability at regional level: results/impact and policy, regional institution and mechanisms Sustainability of national components Follow-up and coordination systems
Risks
CRG – a mix of the national/component findings with a regional perspective – not necessarily the average of NRG NRG – from a national/component perspective determined by project performance within the count ry. CRG – a mix of the national/component findings with a regional perspective – not necessarily the average of NRG NRG – from a national/component perspective determined by project performance within the country . CRG – a mix of the national/component findings with a regional perspective – not necessarily the average of NRG NRG – from a national/component perspective determined by project performance within the country .
- National dimension/components prevails (nationalist attitude) – loss or reduction of the regional dimension. - Personal in charge of both national and regional activities
Guidelines for Monitors for Regional Programmes 2008
19 are overloaded and work exclusively on national ones. - Low visibility of the regional dim ension , or visibility concentrated on the managing institution - All the ECD are not equally involved, committed and coordinated
CRITERIA Relevance
Typology D Not relevant on regional level.
Grading NRG – as per standard MR
Only national assessment. Conditions to set up an effective regional project could be assessed.
Efficiency
Effectiveness Impact Sustainability
Is the project cost -efficient?
NRG – as per standard MR
Is the regional dimension costly compared with a pure national approach?
If a consolidated report is requested this area should be focussed on as the regional dimension may be cost effective, costly or neutral.
No regional effectiveness.
NRG – as per standard MR
National assessment.
National grading.
No regional impact.
NRG – as per standard MR
National assessment.
National grading.
National assessment.
NRG – as per standard MR
Recommend to use national funds for such programmes.
Risks
No specific (regional) risks, except over -costing.
.
Guidelines for Monitors for Regional Programmes 2008