Guidelines for Impact Monitoring in Economic and Employment Promotion Projects

Section 41 Economic and Employment Promotion

GUIDELINES FOR IMPACT MONITORING, PART I

Martina Vahlhaus, Thomas Kuby

Guidelines for Impact Monitoring in Economic and Employment Promotion Projects with Special Reference to Poverty Reduction Impacts Part I: Why Do Impact Monitoring? - A Guide

1

GUIDELINES FOR IMPACT MONITORING, PART I

2

Section 41 Economic and Employment Promotion

GUIDELINES FOR IMPACT MONITORING, PART I Poverty Reduction through Economic and Employment Promotion

Martina Vahlhaus, Thomas Kuby

Guidelines for Impact Monitoring in Economic and Employment Promotion Projects with Special Reference to Poverty Reduction Impacts Part I: Why Do Impact Monitoring? - A Guide

Eschborn, March 2001

3

GUIDELINES FOR IMPACT MONITORING, PART I

Published by: Deutsche Gesellschaft für Technische Zusammenarbeit (GTZ) GmbH Postfach 5180, 65726 Eschborn Germany Internet: http://www.gtz.de Section 41 – Economic and Employment Promotion Author: Martina Vahlhaus, Thomas Kuby Editing: Petra Müller-Glodde Layout: seifert media inform, 65929 Frankfurt Print:4 Universum Verlagsanstalt, 65175 Wiesbaden

GUIDELINES FOR IMPACT MONITORING, PART I

CONTENTS

Preface .........................................................................................................................7 1. Introduction ...............................................................................................................9 1.1 Poverty Reduction through Economic and Employment Promotion ....................9 1.2 Goal of the Guidelines ......................................................................................13 2. Why Do We Need Impact Monitoring and What For? ..............................................15 2.1 What is the Purpose of Impact Monitoring? ......................................................15 2.2 Impact Monitoring as Part of the GTZ’s Quality Management...........................15 2.3 A New Quality Management .............................................................................17 2.3.1 Creating prerequisites for good services........................................................18 2.4 A New Approach to Evaluation .........................................................................19 2.5 The GTZ’s New System of Evaluation ..............................................................21 3. Our Definition of Impact Monitoring .........................................................................24 3.1 Monitoring as an Instrument and a Process......................................................24 3.2 Different Levels of Monitoring ...........................................................................26 3.3 Which Impacts are We Dealing With?...............................................................27 3.4 Conventional and Participatory Monitoring........................................................33 4. Six Steps for Implementing and Carrying Out Impact Monitoring.............................35

5

GUIDELINES FOR IMPACT MONITORING, PART I

6

GUIDELINES FOR IMPACT MONITORING, PART I

PREFACE Today in development co-operation demands on the quality of projects are increasing while at the same time means are becoming scarce. Thus, organisations have to arrange their work particularly efficient and sustainable. The pressure to provide “empirical proof” of results and impacts is increasing as well and the reference to achievements as activities and results is not sufficient, the focus lies on the actual impacts of these achievements. Unfortunately, regular monitoring of impacts still is exceptional in today’s development co-operation.

One aim in publishing this paper is to help closing this gap between demands and reality. Already now, it can be stated that these guidelines are an important completion to the range of instruments of the project-internal monitoring and evaluation system of the GTZ. The working group of the sector project “Poverty Reduction through Economic and Employment Promotion (EEP)” developed these guidelines in a participatory process and succeeded in maintaining this procedure up to the application of impact monitoring.

Those involved understand impact monitoring as an activity ! ! !

that meets the interests and needs of the stakeholders whose know-how and experience is taken into account and that in the long-term makes it possible to carry out impact monitoring without the support of the project.

“Best practices” from projects and various publications, in particular qualitative methods of impact observation, analysis and evaluation, are hereby made available to AP/advisors in the projects, partner experts and planners in the field of EEP who have decided to implement impact monitoring in their respective projects or those who have implemented it already.

These guidelines provide them with a comprehensive and systematic description how to develop a participatory impact monitoring system. Also clear definitions of monitoring, impacts and impact monitoring are included. Finally, six methodical steps for structuring and implementing impact monitoring are outlined. In Part II these steps are described in detail quoting examples from the day-to-day work of projects in the field of EEP and at the very end a broad range of methods and instruments is presented.

7

GUIDELINES FOR IMPACT MONITORING, PART I

This paper was written for projects in the field of EEP with special reference to poverty reduction in particular, nevertheless, the concept it is based on allows for a broader field of application.

Günter Schröter

8

GUIDELINES FOR IMPACT MONITORING, PART I

1. INTRODUCTION

1.1

Poverty Reduction through

Economic and Employment ProA precise understanding of results and

motion

impacts throughout the course of a project contributes towards improving

It is our understanding that economic

controlling and thus to the success of

and employment promotion encom-

the project. At the same time it provides

passes the promotion of the smallest,

reliable accountability to the commis-

small and medium-sized enterprises

sioners of the project and the general

(SSMEs), the development of a finan-

public. The guidelines at hand provide

cial system, vocational training, as well

practical help for implementing and

as job market policies, for example, job

enforcing poverty reduction impact ob-

placement.

servations as well as for analysing in Economic and Employment Promotion

By strengthening the competitiveness

projects. It provides answers to the re-

of the SMEs with growth potential by

quirements of one department, but is at

means of financial and non-financial

the same time a part of the GTZ’s reori-

services, as well as vocational training

entation in terms of management pol-

measures,

icy.

growth impulses are initiated in the po-

medium-

and

long-term

litical economics that initiate both posiThe topic “Impact Observation� has be-

tive income and employment effects

come impressively popular at the GTZ

and also increase the room for redistri-

over the past two years. On the leader-

bution in favour of the poor.

ship level, strategic decisions have been taken to create the necessary

The income and employment impacts

room to develop the initiatives for ob-

achieved in this manner do not, how-

serving impacts.

ever, in most cases automatically reach the poor in the short- and medium-term.

The sector project in the Department of

Targeted strategies, approaches and

Economic and Employment Promotion

measures are necessary so that the

has initiated and operationalised these

poor gain qualifications and develop

guidelines for poverty-related impact

their ability to articulate and organise

monitoring which are valid for the whole

themselves. They are then in a better

of the GTZ.

position to find employment or to be-

9

GUIDELINES FOR IMPACT MONITORING, PART I

come self-employed in order to be able

also in the lack of access to infrastruc-

to co-operate in economic life and also

ture, educational opportunities, health,

to stand up for their own interests as

in malnutrition, high infant mortality

equal partners and in some cases to

rates, inadequate living conditions, etc.,

use the possibilities that still have to be

as well as particularly in the lack of a

created. Systematic approaches are

share and participation in the political,

required with intervention on the micro,

social and economic life of the country.

meso and macro level. Within the framework of these approaches, rea-

Measures implemented as part of eco-

sonable basic conditions are created, a

nomic and employment promotion can

connection between a peoples' econ-

and should not cover all areas. In some

omy and the modern sector is made

cases measures in other areas, for ex-

easier and strategic alliances of differ-

ample, basic education and health are

ent actors from the state, private econ-

prerequisites for a successful promotion

omy and civil society, on the communal

in the field of economic and employ-

or regional level, are initiated. The goal

ment promotion.

is to secure and create employment and income for the poor. The strategies

The indicators against which successful

to be followed in co-operation with the

poverty alleviating economic and em-

poor and instruments and measures to

ployment promotion can be measured

be employed must, on the grounds of

are certainly the stabilisation and in-

the differing initial prerequisites of poor

crease in employment and income of

target groups, be adapted to their re-

poor target groups and improved work-

quirements and adequately differenti-

ing conditions. Prospering undertakings

ated.

can provide

indications of possible

improvements in the economic and Against this background, the aim of the

employment situation of the poor, but

sector project is to achieve positive in-

are not sufficient as the only indicators.

come and employment effects in poor

The poor often cite, for example, better

population groups through measures to

nourishment and living conditions as

alleviate poverty.

poverty

well as children attending school as a

does not only express itself in terms of

result of improved income as indicators

poverty of income due to lack of em-

for their improved situation.

Naturally,

ployment or under-employment, but

10

GUIDELINES FOR IMPACT MONITORING, PART I

Against the background of a mostly

poverty reduction, which reach the poor

limited share and participation of the

target groups via longer impact chains.

poor in the decision-making processes,

In accordance with the Guidelines for

greater significance is attached to the

Assessing Poverty Reduction of Co-

goal of increasing occupational and

operation Projects of 24 November

enterprising competence, as well as

1997, the following criteria must be met

building up the frequent lack of self-

to varying degrees:

confidence in dealings with the gov-

•

ernment and non-government institutions, i.e., the poor’s capacity to articu-

The poor are a part of the target group.

•

late and organise themselves.

The project improves the living conditions of the poor and promotes their productive potentials.

In order to be able to assess which im-

•

The poor participate in the project.

pacts are actually attained by imple-

•

The project is embedded in a pov-

menting various measures, they have

erty alleviating environment.

to be observed and analysed continuously throughout the course of the proj-

Beyond that, there are projects which

ect.

are oriented to common development policy, in which, however, these criteria

We can differentiate between projects

of poverty reduction are not fulfilled.

to directly alleviate poverty, which reach the poor target groups via short impact chains, and projects aimed at overall

11

GUIDELINES FOR IMPACT MONITORING, PART I

Identifying Poverty Reduction Projects Overall Poverty Reduction

Immediate Poverty Reduction Projects with target groups which are delimitable and can

Projects with target groups which are not delimitable and cannot be directly reached

be reached directly (via short impact chain)

Criteria

1. Are the poor part of the target groups?

(via longer impact chain)

SHA

SUA

MSA

EPA

Self-help oriented

Other immediate

Overall poverty

General develop-

poverty reduction

poverty reduction, esp. basic social services

reduction on a macro and sector level

ment policy

The proportion of poor in the target groups equals

Poor in the region or in the country bene-

– least 50%

fit from the indirect impacts of the project to an appreciable extent

– at least the same share as that of the poor in the population in the corresponding region (lower limit 30%)

2. Does the project improve the living conditions of the poor and promote their

X

X

X

Projects are worthy of promotion based on general development considera-

productive potentials?

3. Are the poor involved?

4. Is the project embedded in a povertyoriented project?

Personally responsible and organising themselves

Involvement in and identification with the project

Local basic conditions in the project area are favourable Room for NGOs guaranteed

Source: BMZ, Dept. 411, 24 November 1997

12

Criteria are not or only partly fulfilled:

Important mechanisms towards participation of the poor in political and social process are basically available Plausible impact chain between project and improvement of living conditions of the poor

tions.

GUIDELINES FOR IMPACT MONITORING, PART I

Depending on their targets and basic

1.2

Goal of the Guidelines

impact chains, projects will thus contribute to poverty alleviation to varying de-

Our1 aim in publishing these Guidelines

grees. Poverty will be seen to be re-

was to animate you, together with those

duced to correspondingly different de-

responsible for the project, to continu-

grees.

ously observe the project impacts in order to learn more about project im-

Nevertheless, we believe that it is im-

plementation. We would like to provide

portant that projects that make no ex-

you with ideas, methods and instru-

plicit reference to poverty also observe

ments with which to carry out impact

the projects’ impacts on the poor. Less-

monitoring with the available human and

qualified poor population groups or

financial resources in a sustainable way.

greater pressure on working conditions, for example, can emerge due to the

Our special concern is the monitoring of

promotion of SMEs. These could be

poverty alleviating impacts. In order to

unintended impacts. When impacts have

do justice to the different type of eco-

actually been observed and analysed,

nomic and employment promotion proj-

the next step is to decide whether

ects, we have selected a procedure

measures should be taken up in the

which is flexible enough to allow it to be

project itself or in the project environ-

integrated into the different projects.

ment which can cushion the impact or

These projects can work with differing

compensate

effects.

goals, strategies and clusters of meas-

Those responsible for the project require

ures and, depending on the intervention

adequate instruments and methods to

level, pass along impact chains of vary-

do this.

ing lengths until the goal of poverty re-

the

observed

duction is achieved.

The procedures, instruments and methods described in these Guidelines indi-

Impact monitoring should support those

cate both general steps for implement-

involved in the project in observing in-

ing and enforcing impact monitoring, as

tended and unintended, especially pov-

well as more specific suggestions for monitoring poverty alleviating impacts in Economic and Employment Promotion projects.

1

The Sector Project “Poverty Reduction through Economic and Employment Promotion� and the Staff Section 04

13

GUIDELINES FOR IMPACT MONITORING, PART I

erty-related impacts, to analyse and rep-

impact monitoring. The steps are de-

resent, as well as to (re)orient project

scribed in detail in Part II in relation to

planning and implementation around the

the different areas of employment.

effects of project intervention and thus to improve the quality of project services.

In order to understand Part II you do not necessarily have to have read Part I.

In the first part, we address AP/advisors in Economic and Employment Promo-

Part II, which is considerably longer,

tion projects, partner experts and plan-

addresses AP/advisors in Economic and

ners who have not implemented impact

Employment Promotion projects, partner

monitoring as yet, as they, for example,

experts, planners and experts who have

believe that the time or cost would be

decided to introduce impact monitoring

too high or the impacts of their project

in their respective projects or have im-

are such that they can neither be ob-

plemented it already. We want to pro-

served nor measured.

vide you with concrete tools.

Point 2 shows why impact monitoring is

For this purpose point 1 describes in

important, how it can help you and on

detail six steps for implementing and

which concepts the result- and impact-

enforcing impact monitoring, quoting

oriented monitoring and evaluation sys-

examples from the day-to-day work of

tem are based in the GTZ today.

projects in the field of economic and employment promotion. Based on short

Point 3 describes what our definition of

descriptions, point 2 introduces exam-

monitoring, impacts and impact moni-

ples from day-to-day project manage-

toring and, especially, how poverty alle-

ment practice, the procedures of which

2

viating impacts can be incorporated.

we have attributed to individual steps in order to indicate the different possibili-

Point 4 briefly describes six methodical

ties of applying the six steps in practice.

steps for structuring and implementing

Point 2 presents different methods and instruments which you can apply.

2

We are here strictly following the GTZ publication of the Staff Section 04 “Monitoring im Projekt”, as well as Department 45 and a Swiss Development Organisation “Guidelines for Impact Monitoring – Sustainable Land Management”.

14

GUIDELINES FOR IMPACT MONITORING, PART I

2. WHY DO WE NEED IMPACT MONITORING AND WHAT FOR? 2.1 What is the Purpose of Impact Monitoring?

tended, especially poverty alleviating impacts, and hence improve the quality of project services.

Furthermore, impact monitoring supports the project participants in their accountability to your commissioners

Impact monitoring enables you and oth-

and financiers (BMZ / national govern-

ers involved in the project to observe, as

ments / international organisations), by

well as to learn about the positive and

providing you with the necessary infor-

negative changes in participating institu-

mation to describe what the stake-

tions, target groups and in the project

holders achieved by means of the proj-

environment.

ect work and how they have achieved it.

For example, you can experience • which desired or undesired changes

2.2

Impact Monitoring as Part of

the GTZ’s Quality Management

set in where; • how these changes come about; • why some desired changes do not set in.

The GTZ already has many institutions, instruments and procedures for guaranteeing the quality of its work with respect to content and administration. The

Observing changes and “learning” from experience gained enables you, together with those involved, to find answers to questions such as: Can we proceed further in the present form?

What can and should we do differently in order to achieve the desired changes?

Impact monitoring consequently creates the basis on which to adapt project

assessment of current and completed technical co-operation (TC) projects of the past years has shown that, in view of the objective difficulties of development co-operation, the work carried out on location, is of an increasingly satisfying standard.3

In view of the challenge to secure the quality of its world-wide development

planning and implementation in the course of the project in such a way that they can be oriented towards the in-

3

Cf. “Erreicht die Technische Zusammenarbeit die gesetzten Ziele”, GTZ 1998.

15

GUIDELINES FOR IMPACT MONITORING, PART I

work, under conditions of decentralisa-

nection between detailed planning and

tion and flexibility, too, the GTZ must

development success at all, it seems to

follow a new course and adapt its range

be moving rather in the opposite direc-

of instruments.

tion. Project practice shows us time and again that planning, by establishing

Up to now quality assurance was above

more and more details, does not be-

all based on “quality at entry”. First of all,

come more realistic, as has been as-

the prerequisites for a project were thor-

sumed for so long, but rather the oppo-

oughly explained. Then, all the steps

site is true: it becomes more “unrealis-

were planned logically and in a target-

tic”. Guided by plans, with often ambi-

oriented manner. This method of proce-

tious overall goals and targets, as well

dure seemed to guarantee quality and

as an imposing list of activities, our pos-

success. However, project practice over

sibilities in everyday project implemen-

many years has shown us that this does

tation of grasping unexpected opportu-

not always hold true – at any rate not

nities with both hands and elegantly

when we define “success” as positive

circumnavigating unexpected obstacles

results of development.

rapidly diminish.

Through planning, coupled with moni-

In order to escape this “logic of failure”,

toring orientated to a target-performance

we must change decisive aspects of our

comparison,

implementation

methods – established routines with

according to plan” becomes possible.

which we approach, implement and ob-

Nevertheless, this is not necessarily

serve a project. The central point is that

identical with actual development suc-

we want to turn our backs on planning

cess. There is evidence to suggest that

and move towards results and impacts.

the extent to which we invest in the ex-

We must reduce the amount of effort put

act preplanning of individual projects

into planning to a necessary minimum

does not adequately correlate with the

and decisively invest the remaining en-

advancement of economic and social

ergy in the systematic observation and

development

analysis of results and impacts. This

countries.

“project

processes

More

planning

in

partner

does

not

information gives us clues, if need be, to

automatically lead to more success. If,

the required changes on the different

today, in times of very dynamic devel-

levels of project work.

opment co-operation, there is any con-

16

GUIDELINES FOR IMPACT MONITORING, PART I

The GTZ has already made good head-

Against this background, the manage-

way in reorganising itself into a modern

ment of the GTZ made a far-reaching

service provider. The working group has

decision in November 1997 on quality

already taken up the subject of “Impact

management, by formulating the follow-

Monitoring” under its own initiative with

ing three guiding principles:

great enthusiasm. At the same time, the GTZ’s highest management level has

a)

“For the GTZ, quality means

made basic decisions mandatory for this

matching its services with the as-

change in vision.

pirations of the client.” b)

2.3

“The most conspicuous basis for the general high quality of GTZ

A New Quality Management

services is a business culture diMilestones for implementing quality assurance at the GTZ have been the intro-

rected entirely towards quality.” c)

“At the end of the day the GTZ

duction of ZOPP in 1983, the project

can only satisfy its clients by

“Planning and Sustainability” in 1992,

means of the usefulness of its

the creation of a “Quality Assurance of

service results.”

Offers” in 1993, the new version of the “Decision-Making Patterns for Preparing

Up until now, quality had an absolute

and Implementing Projects” in 1996, the

and normative character at the GTZ. It

optimisation of internal service providers

was to be measured, amongst others,

in 1996, the analysis of the GTZ’s key

by means of the degree of poverty re-

tasks and the establishment of inter-de-

duction, the promotion of women, the

partmental “Quality Advisory Services”

protection of environment. New chal-

team in 1997. If one looks at the entire

lenges were continuously being added

process, one can detect a move in terms

to these, putting staff in projects and at

of quality criteria away from “doing” to-

Headquarters under increasing pres-

wards “advising”. Methods of procedure

sure, but little changed the course and

which, although they describe the proj-

result of the work.

ect conception in detail, but do not adequately assess the project success, are

The above-mentioned new orientation

increasingly being criticised.

towards quality opens up new vistas.

Based on the recognition of the fact that in our development co-operation we can

17

GUIDELINES FOR IMPACT MONITORING, PART I

only achieve what our “clients” want, the

all actors to reach a common denomi-

GTZ now sees that matching its services

nator, i.e., the GTZ itself too. Should this

with the wishes of the client is not the

not work, it can, if need be, withdraw

only, but the most significant criteria for

from the commission.

quality.

How does the GTZ recognise whether The criteria of quality are no longer ab-

its different clients are satisfied with the

solute, but relative!

services or not?

But who is the client? Clients of the GTZ

As already stated, good prospects and

are the commissioners, the counterparts

precisely formulated plans are not suffi-

and the target groups, hence a broad

cient. For our clients – commissioners,

spectrum of actors, who normally have

counterparts and target groups – it is

at least partially differing interests.

above all the results of the work that count and also what has actually been

Consequently, one cannot speak of the

accomplished under the real-life devel-

aspirations of clients, but rather of partly

opment conditions. If we take the satis-

diverging interests of the individual ac-

faction of our clients as a standard for

tors participating in a project.

quality and know at the same time that this satisfaction is based on useful re-

One of the most important tasks of the

sults and impacts, then that means that

GTZ is to define the commission. Based

we must

on requests for support, the different

•

targets and aspirations are worked out and, if possible, consolidated as part of an advisory process with the different

create prerequisites for good services, but also

•

assess time and again how successful our activities actually are.

stakeholders and interest groups.

2.3.1 Creating prerequisites for This work includes an advisory service.

good services

Should the GTZ estimate the target or the suggested action areas as inade-

The GTZ has already taken important

quate or in contradiction to the profes-

steps to improve the prerequisites for

sional ethics of its development work, it

good services. The Reform of Human

will try, during the advisory process, for

Resources Management was guided by

18

GUIDELINES FOR IMPACT MONITORING, PART I

the recognition that GTZ staff are by far

2.4 A New Approach to Evaluation

the most important support for providing good quality services. The adjustments

How and from what do we learn? Nu-

made to selection procedures have

merous analyses of individual and insti-

been influenced by the experience that it

tutional learning have been carried out

is a matter of employing the “right peo-

and many books written on the subject,

ple” from the beginning and not of over-

all of which have considerably extended

estimating the corrections which are

our understanding of the learning pro-

possible by means of further training.

cess. A more detailed analysis of this

Without suitable human resources, good

topic would go beyond the scope of this

services are not possible, but that is not

report. However, it is generally recog-

even the whole picture.

nised that one must look at one’s own results and impacts in order to be able

Another important pillar of good services

to act correctly and appropriately. Engi-

are suitable organisational and imple-

neers differentiate in this connection

menting structures. The best staff can-

between the terms “controlling” and

not work to their fullest potential if the

“regulation”. If all the details of a pro-

wings of their ability and motivation are

duction process are determined and

clipped through out-dated organisation

cannot be disturbed by external influ-

and procedures. The GTZ has clearly

ences, then it can be “controlled” suc-

improved the prerequisites for main-

cessfully by means of a fixed pro-

taining high-quality standards through

gramme. However, when production is

decentralisation and flexibility, as well as

subject to fluctuating conditions, certain

through the accompanying new rules.

targets are attained only through “regulation”. Controlling is always based on

A further condition for high quality still

the same execution of the predeter-

remains, namely systematic assessment

mined orders, during which regulation

of the actual success of our efforts. A

orients itself to its results and flexibly

new approach to evaluation based on

corrects the observed decisions. The

learning was required for that.

symbol for control is a straight arrow; the symbol for regulation a bow, also known as “feedback”. Learning is based on such feedback bows: the observation and assessment of results and impacts

19

GUIDELINES FOR IMPACT MONITORING, PART I

of one's own activities are its most im-

2.4.1 A paradigmatic change

portant source.

This leads to a paradigmatic change in In order to strengthen the learning

planning and evaluation, which espe-

method of monitoring and evaluation

cially increases the chances of a de-

(M&E), we must find new ways in these

mand-oriented project practice in the

areas too. The methods of target-per-

rather complex approaches of poverty-

formance comparison which form the

oriented

core of our M&E process are inadequate

promotion.

economic

and employment

for acquiring the required “feedback” for successful activities. We have repeat-

The traditional method of procedure is

edly experienced in development co-

characterised by linear thinking and divi-

operation the phenomenon that the re-

sion of labour. “In a systematically

sults and impacts of a project always

structured learning process (...) how-

deviate from what was actually intended,

ever, an iterative method of procedure

or that what was achieved was not in-

dominates in which each solved problem

tended, i.e., that many results and im-

throws up new questions and creates

pacts of our actions go far beyond what

new problems (...) The project does not

can be recorded by the target-oriented

plan in detail and ex ante a complex

observation of the actual conditions.

development process, but reacts flexibly to an actually arising need – obviously

The decisive question for assessing

against the background of the ex ante

success is not, therefore, whether the

defined common targets. Flexibility and

planned results were achieved. Instead

demand-orientation should not be mis-

it asks which results were achieved. The

taken for aimlessness and popularity”4.

question of whether the arising impacts

Learning from feedback leads to a dif-

correspond to the planned targets is less

ferent approach to plans, targets and

important than the answering of the

deviations registered by them in the

question: which impacts have actually

course of the project:

occurred?

4

Meyer- Stamer, Qualmann, “Wirtschafts- und Beschäftigungsförderung in systemischer Perspektive: KMU-Förderung, Förderung des Subsistenz-Kleingewerbes und Beschäftigungsförderung – Ansatzpunkte und Komplementaritäten”, 1999, page 25.

20

GUIDELINES FOR IMPACT MONITORING, PART I

“Previously, a deviation from the plan,

has now been established that the indi-

from the envisaged goal or the intended

vidual project is very seldomly the deci-

impacts, were experienced as some-

sive “unit of account”. The success of a

thing negative, leading to opposition and

project is not always an immediate de-

frustration amongst the stakeholders.

velopment success. Projects can be

But what happens when the project tar-

successfully completed without having

get and plan are consciously seen as

promoted the social and economic de-

variables which not only change during

velopment in their environment; they can

the course of the project, but should also

even have had a negative influence on

develop?”

5

the development process. These impacts cannot then be put down to indi-

As usual, we must obviously be ac-

vidual projects, but rather to project-

countable to the employers and the pub-

independent impact monitoring, such as

lic as to whether and to what extent the

described under point 3.3.3 below.

project has obtained its target, if need be also why the originally planned tar-

2.5 The GTZ’s New System of Eva-

gets could not be achieved and, ac-

luation

cordingly, how the project concept has changed. For that reason and for our

At present the GTZ is in the process of

own quality management, we, therefore,

establishing a new system of evaluation

also need information on the unintended

according to the above-mentioned ap-

results and impacts of a positive as well

proach and in consultation with its com-

as of a negative kind.

missioners.

The new, approach to evaluation which

This system of evaluation must corre-

is geared towards impact paves the way

spond to the following two fundamental

for further possibilities. It not only goes

requirements in order to unfold the ex-

above and beyond target-performance

pected benefit, as well as to be ac-

comparison, but can also go beyond the

cepted both internally and externally:

consideration of the individual case. It • 5

Quotation, Mueller-Glodde, Rrainer “Wie plant man, was sich nicht denken lässt”, in Drehscheibe, GTZ, OE 4206, Edition no. 9, October 1997, Pages 3-6.

It must fit into the assessment of success of German technical cooperation (TC) as newly classified by the BMZ, and

21

GUIDELINES FOR IMPACT MONITORING, PART I

•

It must correspond to the GTZ’s ba-

In this manner, the GTZ’s system of

sic requirements of decentralisation.

evaluation becomes part of the system of progress review of German TC which

Correspondingly, the GTZ’s new system

has been redesigned by the BMZ.

of evaluation has been classified in a new way. As the external evaluation of

The second requirement relates to the

the GTZ is increasingly targeted towards

consideration of the basic principles of

bigger assessment units which go be-

decentralisation.

yond the bounds of individual projects,

based on the trust in the competence

such as cross-section topics, sectors

and the readiness of the staff to take on

and instruments, the evaluation tasks

responsibility for quality at their own

which the GTZ has to take on have been

place of work. This is the prerequisite for

extended. The catalogue of tasks as-

client-oriented services. Quality assur-

signed to the GTZ today not only in-

ance should not, therefore, be delegated

cludes regular course of the project and

to the central supervision and assess-

impact monitoring and special project

ment units. A consistently high level of

supervision, such as Project Progress

quality can be realised only on the basis

Review (PPR) and final assessment, but

of an overall business culture geared

also the observation of offers and proj-

towards quality.

Decentralisation

is

ect preparation. The GTZ was also entrusted with a large proportion of individ-

One can deduce from this that monitor-

ual project assessment (which has up to

ing and self-evaluation of the projects

now been carried out by the BMZ), un-

and programmes must form the basis of

der the condition that it respect the

the GTZ’s overall system of evaluation.

BMZ’s evaluation model and the princi-

Whether quality assurance places its

ples of evaluation formulated by the

trust in control or autonomy makes an

DAC6.

immense difference. Experience has taught us that self-evaluation, as a rule,

6

Independence, credibility and utility are the decisive quality characteristics of each evaluating project. As per the requirement that independence must result after each step of the evaluation process, i.e., both during the planning of the entire evaluation programme, as well as in the formulation of the terms of reference and the selection of evaluation team. Evaluation tasks should be always be observed separately from operative management. Nevertheless, the institu-

22

is more critical and cost-effective than

tional separation should not be carried so far as to lose the connection between evaluation results and operative decisions. One possibility, for guaranteeing the required balance is to transfer the evaluation function to a special organisational unit placed directly under the highest management organ.

GUIDELINES FOR IMPACT MONITORING, PART I

external controlling and that it contrib-

operative management. The evaluation

utes to learning both in the project as

unit should support the self-evaluation

well as within the GTZ as an organisa-

measures of the operative units and

tion.

carry out the aggregation of individual information, should do the necessary

Nevertheless, the GTZ’s system of

groundwork for the external evaluation

evaluation cannot only support itself by

and for the overall smooth functioning of

means of the two pillars of evaluation

the entire system. At the moment, these

through

self-

tasks are implemented by the Internal

evaluation of projects and operative ar-

Evaluation Team in Staff Section 04. In

eas. Although approaches are integral

autumn 1999, the GTZ will decide on

parts of a modern system of evaluation,

how to reorganise its internal evaluation

they do not fulfil all the requirements.

capacity.

external

units

and

External evaluation measures in the first instance serve accountability; the op-

The GTZ’s system of evaluation which is

erative areas can learn only minimally

at presently being established is, as

from them. The opposite, however, ap-

shown below, based on a pyramid

plies, in that learning is at the forefront

structure and creates a connection to

of self-evaluation measures, whilst in-

the external evaluation by means of its

termediary steps are required for ac-

own evaluation unit. Important parts of

countability, with which the individual

the system are already operational and

piece of information has to be aggre-

are to be fully functional by mid-2000.

gated and condensed.

Therefore, a third element of internal evaluation is required, namely, an evaluation unit which is independent of the

23

GUIDELINES FOR IMPACT MONITORING, PART I

The GTZ System of Evaluation

Type

Focus

GTZ Work and System of Evaluation

GTZ Portfolio and Business Strategy

External Evaluation

BMZ, FederalAudit Office, Auditor

i.e. Staff Section (under GF, independent of operative areas

Operative Areas (Regional Areas, Office. PuE)

Country Portfolios and Sectors

Internal Evaluation Projects (Projects and Programmes) Benefit

The BMZ, Federal Audit Office and the contacted auditors are responsible for the external evaluation of the work of the GTZ. They not only assess the development success, but also the reliability of the GTZ’s internal System of Evaluation. The operative areas (country areas, office and PuE) evaluate country portfolios and sectors, during which the projects direct their attention mainly towards the immediate benefit of development services. An independent evaluation unit, subordinated to the management, focuses on the entire portfolio and business strategy, and supports the operative areas and projects during self-evaluation.

3.

OUR DEFINITION

OF

for the project, i.e., our “counter-

IMPACT

parts”8 and their advisors to continu-

MONITORING

ally direct their work towards the desired project results and project tar-

3.1 Monitoring as an Instrument

gets, to document these and to pre-

and a Process

sent them and to improve the quality of the project performances;

In accordance with the general understanding within the GTZ, we define monitoring as an7 •

instrument of programme management which helps those responsible

7

Cf. “Monitoring im Projekt”.

24

8

We define “counterpart” as those organisations or units in partner countries which we work directly with. They are the clients of the GTZ advisory services and the recipient of GTZ services.

GUIDELINES FOR IMPACT MONITORING, PART I

•

•

organised communication and un-

instrument of organisational devel-

derstanding process between the

opment with which a learning and

different stakeholders concerning the

development process is initiated and

following questions: What do we

carried out autonomously by the

want? Where do we stand? Where

stakeholders in a project.

are going? Which corrections need to be made?

Monitoring is an integral part of “Project Cycle Management”. Monitoring activities must also be planned and equipped

decide

observe

with adequate resources.

act

Project Planning and Implementation

observe

decide

act

Monitoring is not only the acquisition of information “Monitoring is concerned with consciously selecting questions which we follow sys-

and the handling (measuring, describing, evaluating, discussing and furthering)

tematically and in a target-oriented manner.

The

insights

gained

serve

the

stakeholders during decision-making processes, in controlling and shaping their project, thus leading to the best possible achievement of targets.”

of information.

Monitoring is also allowing the results to be integrated into planning and implementation to improve,

adapt and to

change them.

“Monitoring not only needs instruments; monitoring also requires the readiness to talk.”

9

9

Quotation from “Monitoring – mit der Realität in Kontakt bleiben","Monitoring in the Project”.

25

GUIDELINES FOR IMPACT MONITORING, PART I

3.2 Different Levels of Monitoring

mean that the project stakeholders have to continuously observe all these strate-

We differentiate three levels of monitor-

gic parameters.

ing.10 More importantly, as a result project Activity Monitoring:

staff should observe whether the in-

Which activities have we planned?

tended results and impacts, or which

Which activities have we carried out?

results

(target-performance comparison)

achieved in order, in the case of devia-

and

impacts,

have

been

tions, to be able to look for the causes Result Monitoring:

and to make corrections in the respec-

What have we achieved?

tive areas. The projects should reach an agreement with the stakeholders on the

Impact Monitoring:

question of which type of monitoring

What impact does our work have?

they can additionally implement under consideration of their targets and as well

These guidelines concentrate on the

as temporal and financial resources.

monitoring of impacts.

More information is available on the other levels of monitoring. We can par-

As mentioned above, impact monitoring

ticularly recommend the publications

provides the basis for orienting project

“Monitoring im Projekt – Eine Orien-

and programme implementation towards

tierung für Vorhaben in der Technischen

the intended impacts. Naturally, making

Zusammenarbeit”, “Monitoring – der

corrections to a project which does not

Realität in Kontakt bleiben”, as well as

achieve the intended impacts means

“Processmonitoring – Eine Arbeitshilfe

changing its concept, and/or its organi-

für Projektmitarbeiter/-innen”.11

sation and/or environment and/or its target relationship, etc., i.e., its strategic parameters. This, however, does not

10

In addition, one can differentiate two additional levels: 1. Area Monitoring: Which external factors promote or hinder our work? 2. Process Monitoring: How have we attained something? Why have we not attained something?

11

26

See Bibliography (in Part II).

GUIDELINES FOR IMPACT MONITORING, PART I

Main Target

Further Impacts

Benefit 2 Impact Hypotheses

Impact Monitoring

Benefit 1 Dev. Target

Project Target

Results

Result

Benefit

Services

Monitoring Activities

Activity Monitoring

3.3 Which Impacts are We Dealing

tiate between benefit, direct benefit and

With?

extensive impacts:

“Projects are effective if the target

•

Benefit – the first impact level – de-

groups require the project services, if

scribes whether and how the project

they benefit from them and develop fur-

services are used by the target

ther in the direction they desire.”12

groups.

•

Direct Benefit – the second impact

We define impacts as the desired and

level – describes the benefit for the

undesired, intentional and unintentional

target group achieved directly from a

impacts of project interventions or proj-

benefit.

ect services.

•

Extensive

indirect

highest-aggregated

impacts

and

development

In accordance with the generally ac-

progress – the additional impact lev-

cepted GTZ understanding, we differen-

els – describe the changes that affect our work beyond the direct benefit in the area or also long-term

12

Cf. “Monitoring im Projekt”.

27

GUIDELINES FOR IMPACT MONITORING, PART I

impacts, which appear only after the

Of late, the insight has, however, arisen

project has ended.

more and more that, as a rule, it is not the

programme

management

which

3.3.1 The difficulties in measuring

fails, but instead that the project is usu-

impact

ally being asked to do the impossible.

Although many TC projects certainly do

Up until now, projects and programmes

contribute towards economic and social

were over-burdened with requirements

development, this can hardly be statisti-

made of their success monitoring. Most

cally isolated in the network of real de-

M&E systems are directed towards tar-

velopment processes – given an aver-

get-performance

They

age endowment of DM 5 million. In

should, however, still provide proof of

technical terms this is known as a too

what they have contributed towards the

low “factor weight”.

comparisons.

attainment of the highest-aggregated development progress (e.g., poverty

Today's pressure to provide proof of

reduction in a region, in the country).

results and impacts appears to aggra-

Experience has shown that only few

vate the problem even further. Because

projects are in a position to do that. Re-

the commissioners and the public have

liable performance levels are rarely

to be reliably informed with regard to

available and if a relationship to highly-

what was actually achieved with the tax

aggregated development progress can

money invested in development co-

be established at all, then it is often via

operation, now “empirical proof” has to

long, usually incomprehensible impact

be presented, where up to now one

hypotheses which should explain that

rather worked with declarations of intent.

and why the project, for example, has not alleviated poverty in a given area.

But how are we to carry out impact monitoring if the experience of two years

The persuasive power of this “impact

of M&E practice show that the impacts

evidence” is meagre. In the past, one

of TC projects can only be reliably

quite often heard the opinion that the

traced for a short time and, more signifi-

corresponding project management was

cantly, when we are striving to link the

not carrying out its M&E tasks suffi-

German contribution to other develop-

ciently and with insufficient profession-

ment initiatives in the counterpart coun-

alism.

try? How is the contribution of an indi-

28

GUIDELINES FOR IMPACT MONITORING, PART I

vidual project to be isolated in the com-

today to end mere the good intentions of

plex development system?

impact analysis. In order not to suffer a setback during the first steps of imple-

3.3.2 The GTZ’s Impact Model

mentation, but rather to achieve a practicable solution, one must know that,

The GTZ’s Impact Model provides an

although necessary, it is difficult to

answer to these questions. This model

evaluate results and impacts. Although

is being used (as far as is known, for the

determining development progress is

first time in an evaluation concept of

difficult, it is still possible. The actual

international development co-operation),

difficulties lie in the “attribution”, i.e., in

to concede that a “classification gap”

the classification of a highly-aggregated

exists and to recognise it as a funda-

development

mental methodical problem of impact

projects.

progress

for

individual

analysis. If at all, then the time is right

29

GUIDELINES FOR IMPACT MONITORING, PART I

The Impact Model of the GTZ Highest-aggregated Development Progress (e.g., Poverty Reduction in the region)

Possible indirect impacts of the project (e.g., increase in employment figures in the region)

Project-independent evaluation

CLASSIFICATION GAP Monitoring and selfevaluation in the project

Direct Benefit (e.g., increase in turnover, increase in the number of people employed, income of undertaking and employment increases Benefit of services (e.g., entrepreneurs apply their skills in practice)

Services (e.g., entrepreneurs successfully take part in training, competence of entrepreneurs increases) Activities (e.g., enforcement of CEFE training)

Inputs (e.g., CEFE trainer, offices, didactic material)

The projects as a rule observe their own impacts only as far as direct benefit. Higher and highest-aggregated development steps, to which the project could have contributed, which could no longer be clearly classified with the usual means of monitoring and self-evaluation, are ascertained and evaluated by means of evaluation procedures applied independently of the project.

Source, GTZ, Staff Section 04, 1999

In order to understand the difficulties, it

the project unfolds “activities� (e.g.,

is helpful to look at a typical impact

CEFE training for the smallest

chain of economic and employment

nesses in the informal sector) and pro-

promotion on the basis of the model

duces services and results (entrepre-

described.

neurs take part in training). These serv-

busi-

ices are used by the stakeholders in inputs (here

such a way that they apply the knowl-

CEFE trainer, offices, didactic material),

edge acquired (e.g., developing new

Equipped with

30

certain

GUIDELINES FOR IMPACT MONITORING, PART I

product ideas or up-to-date cost obser-

tion gap it limits the tasks to be under-

vation) in their small business (benefit).

taken by the project itself.

Their direct benefit would then, for example, be the stabilisation of the busi-

3.3.3 Consequences of the impact

ness and, thus, the sources of income

model for impact monitoring in proj-

for small entrepreneurs or an increase in

ects

income, which contributes to being able to better feed the family (a poverty alle-

When they are based on this model, the

viating impact relating to an individual).

projects are expected to: a) observe their impacts themselves up

In this lower part of the impact chain, the classification is still relatively simple, as

to the level of direct benefit; b) use the results of observation for

the analysis is closely linked to the project. Nevertheless, causalities cannot

self-control; and c) be able to provide up-to-date information at any time.13

necessarily be proved here, as there is a whole series of other influencing factors which are rarely known to everyone and

In accordance with the impact hypothe-

normally cannot be isolated.

sis and impact model underlying the project, it is expected that the projects

The further we progress along the im-

will understand the poverty alleviating

pact chain (and thus distance ourselves

impacts of their work up to the level of

from the project), the more difficult the

direct benefit as defined in the impact

classification becomes. Therefore, be-

chain.

fore we reach the higher-aggregated development progress, in practically

According to this model, establishing

every case we come cross a classifica-

possible project impacts beyond the

tion gap which a project can hardly

direct benefit, however, is not one of the

bridge using its own funds.

tasks which have to be completed by the projects alone (in our model, e.g., the

The impact model adopted for the GTZ’s

increase in the number of poor women

system of evaluation locates the classification gap on the basis of a typical impact chain and based on this classifica-

13

If projects want to go further than the direct benefit, of course they can do so, but they should attempt to remain within a cost framework of about 5% of the entire budget.

31

GUIDELINES FOR IMPACT MONITORING, PART I

who are in adequate employment and

thodical preparation in order to become

have sufficient income in a region and

a reliably functioning element in the

thus poverty alleviating impacts in their

GTZ’s system of evaluation. However, it

region). Evaluation procedures which

is already clear today that the differen-

are independent of the project are used

tiation between “monitoring and self-

for this purpose, procedures which take

evaluation in the projects” and “project-

larger “units of account”, such as a re-

independent evaluation” is a very prom-

gion or a sector into account. Further-

ising, realisable concept for extending

more, they establish what has changed

the GTZ’s system of evaluation. The

in their development status (this limita-

projects can make what is surely the

tion is of great significance for the reali-

most significant contribution towards this

sation of the approach), without simulta-

extension by developing their own im-

neously trying to attribute the estab-

pact analysis limited to the direct benefit

14

lished changes to specific projects.

and, thereby, withstanding the attempts to risk their professional evaluation of

Ascertaining the higher-aggregated de-

proximal areas on speculations at the

velopment progresses and their classifi-

distant end of the impact chain.

cation are two separate steps. The third step consists in establishing a plausible

The second part of the guidelines thus

bridge between the results of project-

concentrates on project-internal impact

internal

monitoring.

monitoring

and

project-

independent consideration of development progress.

There is not always a provable causal connection between project services

Project-independent evaluations have

and

up until now been implemented only in a

ourselves from the activities, the bigger

few individual cases and still require

the classification gap becomes, as the

considerable organisational and me-

influence of other factors increases, i.e.,

impacts. The more we distance

we cannot prove in each case that the 14

Such procedures have been, e.g., carried out many times already within the framework of the World Bank’s poverty analysis. However, donorcounterpart alliances are required in order to yield their full potentials, which have not yet gone beyond modest beginnings. See also T. Kuby, “Making Evaluation Alliances Work”, GTZ, May 1997.

32

observed changes have occurred on the grounds of our activities. Nevertheless, we can always use various pieces of information from monitoring and evalua-

GUIDELINES FOR IMPACT MONITORING, PART I

tion to conclude that a plausible connec-

The focus during participatory monitor-

tion exists between activity and change.

ing is on building up the capacity and the will of the stakeholders to reflect, to

3.4 Conventional and Participatory

analyse and to take over the responsi-

Monitoring15

bility for implementation of the recommended changes.

As the following table shows, we define conventional monitoring in its pure form to be “experts” at a certain point in time measuring and assessing the progress of a project, on the basis of the predetermined indicators. The focus during conventional monitoring thus lies in obtaining as “neutral” and “objective” an assessment as possible in order to be able to be accountable to the financiers or other entitled groups of the project.

In contrast, stakeholders have a central and active role in participatory monitoring. They are responsible for creating and assessing information, as well as preparing recommendations for changes in planning and implementation. The role of the external experts during participatory monitoring is not to assess the monitoring results, but rather to participate in the process and to support continuing learning of the stakeholders.

15

Deepa Narayan in “Participatory Monitoring and Evaluation – Module VI”.

33

GUIDELINES FOR IMPACT MONITORING, PART I

Conventional Monitoring Who?

Participatory Monitoring

External experts

Stakeholders, incl. target groups, project staff, external moderators

When?

At intervals determined by the project

Continuously throughout the course of the

management

project (decisions on information collection made by the stakeholders)

How?

Indicators which measure inputs and

Indicators which were identified or developed

outputs – developed in workshops or by

by those involved in order to observe the

those responsible for the project

results and impacts

Questionnaires/interviews of external “neutral” evaluators not associated with

Simple, qualitative and quantitative methods, applied by the stakeholders themselves (if

the project

necessary with external support, moderation and information collection)

Why?

So that project and staff remain account-

In order to enable stakeholders to initiate

able to the commissioner

changes

In accordance with our understanding of

•

be implemented (in the long-term

impact monitoring, we do not consider a

also without the support of the proj-

purely conventional method of proce-

ect);

dure for implementing and enforcing

•

impact monitoring to be sensible.

provide reliable or realistic information; as well as

•

where necessary, lead to changes in

The reason is that different actors have

planning and implementation of the

various interests and perceptions, but

activities; and

also different knowledge and experiences, harbouring a large potential for

•

lead to the improvement of the quality of project services.

impact monitoring.

To what extent and how the stakeBy involving the different actors during

holders are involved in a basic partici-

the establishing and implementing of

patory procedure can take on very dif-

impact monitoring, one increases the

ferent forms and must above all be di-

probability that impact monitoring will

rected by the expectations stakeholders

correspond to the interests and needs of

have of impact monitoring.

the stakeholders and, therefore, that it will

34

GUIDELINES FOR IMPACT MONITORING, PART I

4. SIX STEPS

FOR IMPLEMENTING

CARRYING

AND

OUT

IMPACT

MONITORING

The chronological sequence of steps is appropriate for understanding and implementing impact monitoring. However, we are by no means suggesting that the individual steps can only be carried out

In analogy with the publication “Guide-

in this order. It is more important and

lines for Impact Monitoring – Sustain-

sensible to go back over a step preced-

16

able Land Management” , the following

ing the one you are carrying out. For

provides an overview of six methodical

example, it is to be recommended that,

steps towards participatory implementa-

after you have already formulated im-

tion and enforcement of impact moni-

pact hypotheses, you reconsider which

17

toring.

Above all, this is a matter of

impact areas are to be observed.

methodologically separating individual steps.

The Six Steps of Establishing and Carrying Out Impact Monitoring (IM) 2. Identify impact areas

1. Agree on the target of IM

3. Formulate impact hypotheses 6. Evaluate information and provide feedback

4. Develop indicators

5. Select collection methods and collect information

16

GTZ, Department 25. The publication describes seven steps, whereby we have here summarised step 6 “Inform” and step 7 “Evaluate and Feedback” in one step, as we find it difficult to understand the methodical separation of these steps.

17

These steps are thoroughly explained in relation to the respective area of tasks in Part II.

35

GUIDELINES FOR IMPACT MONITORING, PART I

STEP 1

the beginning of impact monitoring it is

Agreeing on the goals of impact

often sensible to concentrate on one or

monitoring

a few impact areas.

It is usually advisors who initiate impact

STEP 3

monitoring. Therefore, the first step is to

Formulating impact hypotheses

identify

the

stakeholders

in

impact

monitoring and clarify their respective

In a third step, together with the others

interests in and expectations of it.

involved, you should establish hypothe-

Building on that, the stakeholders should

ses concerning which project services

determine the goals of impact monitor-

will effect what changes on the various

ing. They should agree upon the meth-

levels and in various areas. This initiates

ods and instruments with which they

reflection of possible impacts and cre-

want to create the steps for structuring

ates an awareness of the numerous

and enforcing impact monitoring. One

desired and undesired impacts of project

person should be selected to be respon-

interventions.

sible for the process.

STEP 4 STEP 2

Developing indicators

Identifying impact areas In order to recognise whether and to In a second step, the most important

what extent the impact hypotheses hold

stakeholders should identify possible

true and the project attains the inten-

impact areas on different levels. It is

tional and unintentional changes, the

helpful to formulate guidelines: Which

project stakeholders need indicators or

changes, for example, in the awareness

“milestones�. Before you develop new

or

indicators together with the rest of the

actions

groups

and

of

which

organisations,

individuals

are

the

stakeholders,

you

should

assess

stakeholders aiming at or which do they

whether already functioning monitoring

expect? Together with the stakeholders

system with indicators exist with the help

you should then select the most impor-

of which you can recognise the desired

tant and the most relevant impact areas

changes. If this is not the case, you

which you want to observe together

should identify and formulate possible

throughout the course of the project. At

indicators together with the rest of the

36

GUIDELINES FOR IMPACT MONITORING, PART I

stakeholders, i.e., also with users/target

STEP 6

groups.

Evaluating information and feedback

STEP 5

The sixth step, the continuous feedback

Selecting methods of data collection

of information from monitoring in project

and collecting the corresponding in-

planning and implementation and intro-

formation

duction of corrections is the most important one, as the enforcement of the

In a fifth step, you should first of all,

earlier steps alone would make little

against the background of available

sense and satisfy only your interest in

materials,

re-

gaining more insights. The goals of im-

sources, clarify the expectations of

pact monitoring should, however, be to

those involved regarding the quality and

direct the project planning and imple-

quantity of data, such as, for example,

mentation towards their desired impacts,

accuracy, reliability and representativity

to learn from the experience gained and

of information, as well as the relevant

to improve the quality of project serv-

requirements of information-gathering

ices. In this step you should, therefore,

methods. On the basis of expectations

agree on when, how and by whom the

and requirements, you should prepare a

information gained from observation is

mixture of methods and instrument to-

to be analysed and evaluated. It should

gether with other stakeholders (full or

also be determined when and how the

partial collection of information, before-

resulting decisions are to be made and

and-after comparison and/or comparison

corrections introduced.

human

and financial

with control groups, qualitative, quantitative and/or semi-quantitative methods, written or oral questionnaires and/or observation). You can and should also develop your own methods and instruments which correspond to the specific requirements

of

the

project

and

stakeholders. Furthermore, you should determine who will gather the information (project staff or the external experts).

37

GUIDELINES FOR IMPACT MONITORING, PART I

38