EAST AFRICAN COMMUNITY LAKE VICTORIA BASIN SECRETARIAT
RESULTS-BASED MANAGEMENT
TOOLS AND TECHNIQUES
2014
ii
Results-Based Management Tools & Techniques
EAST AFRICAN COMMUNITY LAKE VICTORIA BASIN SECRETARIAT
RESULTS-BASED MANAGEMENT
TOOLS AND TECHNIQUES
Plan:Net Ltd
Project Services International
Microde Consult
Canada
Canada
Kenya
Lake Victoria Basin Secretariat
iii
Table of Contents ACRONYMS
vi
I. INTRODUCTION
1
How to Use This Guide
2
The Context for Results-Based Management at LVBC
3
Background on LVBC
4
Who should use this Guide?
5
II. RESULTS-BASED MANAGEMENT
7
Key RBM Definitions
11
Key RBM Concepts
13
Analytic Framework of the LVBC RBM System
16
Uses and Users
21
Governance – Management Relationships
24
RBMS Management Structure
25
III. RESULTS-BASED MANAGEMENT SYSTEM DESIGN TOOLS
iv
29
Logic Model (LM)
31
Risk Analysis and Management Table
32
Performance Measurement Framework
34
Reporting (narrative)
36
Results-Based Management Tools & Techniques
IV. HOW TO USE THE RBMS
37
RBMS Approach
37
Develop Initiatives Using RBMS
39
Make a Plan to Monitor and Evaluate Progress Using RBMS
52
Use the PMF to Document Progress – Monitor and Report on Results
58
V. RESULTS-BASED PROJECT IMPLEMENTATION
65
The Organisation Chart
66
Work Breakdown Structure
69
Schedule
73
Budget
75
Some Additional Resources
77
APPENDICES
79
Appendix A: Logic Model Template
81
Appendix B: Performance Measurement Framework Template
83
Appendix C: Risk Analysis & Management Template
85
Appendix D: Project Results Logic at Glance
87
Lake Victoria Basin Secretariat
v
ACRONYMS CSO
IPDET
Civil Society Organisation
International Programme in Development
DES-FAD
Evaluation Training
Deputy Executive Secretary Finance and
LVB
Administration
Lake Victoria Basin
DES-PP
LVBC
Deputy Executive Secretary Projects and
Lake Victoria Basin Commission
Programmes
EAC East African Community
LVEMP Lake Victoria Environmental Management Programme
EALA
LVFO
East African Legislative Assembly
Lake Victoria Fisheries Organisation
EALP EAC-AMREF
LWATSAN
Lake Victoria Partnership
Lake Victoria Water Supply and Sanitation
HR Human Resources
ICT
Programme
M&E Monitoring and Evaluation
Information Communications Technology
MCSLV
IDRC
Victoria Project
Maritime Communication for Safety of Lake
International Development Research Centre
vi
Results-Based Management Tools & Techniques
MEACA
RBMF
Ministries responsible for East African
Results-Based Management Framework
Community Affairs
MERECP Mount Elgon Regional Ecosystem Conservation Programme
MTEF Medium Term Expenditure Framework
NFP National Focal Point
RBMS Results-Based Management System
Sida Swedish International Development Agency
TWB-MRB Transboundary Water for Biodiversity and Human Health in the Mara River Basin Programme
OECD-DAC Organisation for Economic Cooperation
and Development – Development Assistance Committee
PCC Partnership Consultative Committee
PMET Planning, Monitoring and Evaluation Team
PMF Performance Measurement Framework
RBM Results-Based Management
Lake Victoria Basin Secretariat
vii
viii
Results-Based Management Tools & Techniques
I. INTRODUCTION This guide was prepared to help Lake Victoria Basin Commission (LVBC) staff to focus on
•
results as they design and manage Commission
activities. It targets managers and staff who will use the Results-Based Management System
•
Align activities with the LVBC’s
vision, mission and mandate, and with the Secretariat’s Strategic Plans to
implement that vision, mission and mandate;
Lake Victoria Basin Secretariat
activities to give managers clear
achievements, problems and risks; •
It is written to help LVBC managers and staff: •
Improve monitoring and evaluation and timely feedback on progress,
report on, and evaluate LVBC activities. It holds the RBMS in everything we do.
the activities they design, select and implement;
(RBMS) to identify, design, implement, monitor, thorough advice on how to set up and implement
Improve the quality and impact of
Help management to make decisions
based on evidence, lessons learned and
regular performance measurement; and •
Improve reporting and accountability to
stakeholders, including the East African Community (EAC), donors, and the people of the Lake Victoria Basin.
1
How to Use This Guide This guide explains how to use Results-Based
of these guidelines at least once to help you build
of LVBC activities. RBM is a management
place within it. Then turn to the section about
Management (RBM) to improve the impact
approach that focuses on the results we intend
to achieve. To do this, we continuously monitor progress and performance to improve the
effectiveness, efficiency and relevance of our
work. LVBC uses the RBM approach at all levels, from project components and individual support activities to entire programmes and processes, including the LVBC Secretariat as a whole.
This guide addresses many people at many levels with wide roles and responsibilities, so it has
several modules. We suggest you read through all
1
a more complete image of the RBMS and your
your specific questions or concerns. This diagram (Fig 1) of the Guide will help you find the right section.
Getting started is the best way to get comfortable with RBM tools and techniques. RBM becomes
easier as you use it to plan activities, programmes and projects, gather evidence of progress and
make management decisions to achieve better results – for LVBC and for the people of the basin. Let’s begin.
INTRODUCTION
- how to use this guide - context for RBM at LVBC
- definitions - key concepts - analytical framework: RBM @ LVBC - uses & users
2
RBMS
- logic model - risk analysis - performance measurement framework - reporting
3
RBMS TOOLS
- identify & design - identify & migitage risks - monitor, report, & evaluate - implement
4
HOW TO
- logic model - performance measurement framework - risk analysis
5
TEMPLATES
Fig 1. How To Use This Guide 2
Results-Based Management Tools & Techniques
Areas of Cooperation (Article 3) Promote trade, commerce and industrial development. Promote development of infrastructure and energy.
Promote equitable economic growth
Improve public health, specifically sanitation.
Promote measures aimed at eradicating poverty
Promote wildlife conservation and sustainable tourism. Environmental protection and management of the Basin. Promote development & management of wetlands.
Promote the protection of the environment within the LVB
Sustainable development, management and equitable use of water, forestry, and fisheries resources. Promote sustainable agricultural and land-use practices including irrigation.
Promote sustainable utilisation and management of natural resources
Maintain navigational safety and martitime security.
Promote compliance on safety of navigation
The Building Blocks Promote and facilitate harmonized policy, legal and regulatory frameworks in areas of cooperation to provide a solid foundation for cross-border cooperation and sustainable development.
Objectives (Article 33) Cross-cutting Issues Gender equity Research & capacity building Knowledge sharing Public participation
Mandate Promote, facilitate, and coordinate activities aimed at sustainable development and poverty eradication on the Lake Victoria Basin.
Fig 2. LVBC Mandate
The Context for Results-Based Management at LVBC The LVBC is a specialized institution of the East African Community (EAC). The Protocol for
Sustainable Development of Lake Victoria Basin Article 33(2) spells out LVBC’s mandate. Article 3 assigns it five objectives and fourteen areas of cooperation. Together, the mandate, objectives
and areas of cooperation are designed to create a development impact. (Fig 21).
1
Article 8 vests the mandate of management, development and utilization of fisheries resources in the Basin to the Lake Victoria Fisheries Organisation (LVFO).
Lake Victoria Basin Secretariat
3
Background on LVBC The EAC has been evolving since its start in
The EAC’s Council of Ministers, which sets
institutional reviews to keep its institutional
oriented management approach to support
March 1996. Over the years, it has done several framework responsive to changes in its operating environment. These have included adding Rwanda and Burundi in 2007, expanding
Community structures, and institutions, and
milestones in the integration process. The most
recent institutional review took place in 2011 as part of the EAC Development Strategy2. LVBC has grown and changed with the
EAC, doing its own reviews as the Secretariat has worked to address institutional capacity
challenges. The Commission has expanded in an
increasingly complex operating environment. For example, budgets grew from $2 million per year
the Community’s policy, also adopted a results effective and efficient performance in EAC institutions.
LVBC’s donors/development partners and its
Partner States are committed to this approach
as well. Managing for Results is endorsed as one of five Aid Effectiveness principles of the Paris Declaration (2005). This was confirmed most
recently at the Fourth High Level Forum on Aid Effectiveness in Busan, South Korea (2011). As
signatories, both EAC Partner States and donors have made commitments, paraphrased below: •
On the part of partners – to develop
to $16.6 million per year between 2008 and 2011.
better linkages between development
60% of these funds in a given year.3 With such
establish results- oriented reporting and
for results more than ever. In fact, these reviews
development strategies; and to track
that LVBC focus more on results to improve
do so.
LVBC was able to absorb and use only 50% to
strategies and budget processes; to
rapid expansion, LVBC must plan and manage
assessment frameworks that work with
and the EAC Secretariat have recommended
indicators as far as it is cost effective to
its performance and deliver on its mandate effectively and efficiently.
•
On the part of donors - to link country programming and resources to results; to align them with partner country
performance assessment frameworks; to 2
Institutional Review of the EAC, Organs and Institutions, EAC, March 2011.
3
Institutional Capacity Needs Assessment for Lake Victoria Basin Commission, ALPEX Consulting Ltd, August 2010; Human Resources Functional Analysis for the Lake Victoria Commission Secretariat (Draft), Dr. Florence Muinde, March 2012.
4
work with partner countries on resultsoriented reporting and monitoring frameworks; and to harmonize
monitoring and reporting requirements.
Results-Based Management Tools & Techniques
The Origins of the Lake Victoria Basin Commission The East African Community (EAC) is a Regional Economic Community comprised of the Republic of Burundi, Republic of Kenya, Republic of Rwanda, Republic of
Uganda and the United Republic of Tanzania. The Treaty for the Establishment of the EAC was signed in November 1999 and entered into force in July 2000. Rwanda and Burundi joined in 2007. Article 114 of the Treaty, on natural resource management, provided for the establishment of a body for the management of Lake Victoria.
In 2001 the Lake Victoria Programme Development Unit was set up at the EAC
Secretariat. This unit spear-headed negotiation of the Protocol for the Sustainable
Development of the Lake Victoria Basin, which was ratified in 2004. In that same year, the Lake Victoria Basin Commission (LVBC) was created to coordinate programs on Lake Victoria and its basin.
As donors like Sida, the World Bank and the
and momentum toward being a results-based
their results-based planning and management
time, to develop and implement a Results-Based
African Development Bank have introduced
plans through various partnership agreements, the pressure is on LVBC to boost its efforts
institution. This is why LVBC decided, at this Management System (RBMS) as part of its evolution into a results-based institution.
Who should use this Guide? The LVBC Secretariat is an institution where
results. The results that LVBC operations
operations, like finance and administration, are
LVBC’s project and programme results usually
projects start and end and where service
ongoing and repetitive. These services sustain the institution. One insight of RBM is that
operations can also gain from a focus on results. RBM is a management approach; it focuses on
the way the organisation is motivated, and how
produce are the services they provide. While
refer to outcomes that convey benefits to the
larger community in the Lake Basin, RBM also includes the service outputs that make those outcomes possible.
it uses processes and resources to reach targeted
Lake Victoria Basin Secretariat
5
Using an RBM approach, you as managers still
control the scope of your activities, the costs, and the timing and quality of delivery. Each of you
uses the RBMS to plan, cost, implement, monitor and measure the changes you produce, whether these are services that help achieve LVBC’s
development outcomes, or projects that enable, coordinate and promote development activities
in the Basin. You will use RBMS to ensure that
cash, supplies and human resources contribute to a logical chain of results that support the LVBC mandate, objectives and areas of cooperation. RBMS applies to both projects and service
areas. The key is to focus on what we produce,
not what we do. The next section of this guide will help us to understand RBM. Section III
will introduce key tools we will use to put an
RBMS into practice. Section IV describes how we can use them in our work, and Section V
provides some additional results-based project
management tools that can help during project implementation.
6
Results-Based Management Tools & Techniques
II. RESULTS-BASED MANAGEMENT LVBC is creating a Results-Based Management
longer term, “development” results. These show
routines, tools, techniques and staff capacity to
and ecosystems.
System (RBMS). This includes organisational
manage for results. You may have already heard about this approach toward project planning,
monitoring and evaluation. This is because public managers, including development donors, are using RBM more and more.
RBM is not very different from what we already do to manage our work. It shifts emphasis
beyond a traditional concern with operational aspects – inputs and activities – and their
immediate results – such as outputs – to focus on
Lake Victoria Basin Secretariat
up as important changes in the lives of people RBM grew from a global change in the
way governments manage and deliver their programmes and services. Historically,
governments focused their attention on resource inputs (what they spent), activities (what they
did) and outputs (what they produced). Accurate information at these levels is important, but it
does not ensure that everyone works toward the
same goal. Nor does it show longer-term changes
that these programmes and services aim to create.
7
Key Messages About RBM 1. You are not alone! RBM is a global movement that is basic to a much larger process of public administration reform (page 8).
2. It’s important to know this! RBM is so widespread, organisations that don’t understand it risk being marginalized: donors expect organisations to know what results they want and how they will prove success.
3. Everyone produces results, so RBM is for everyone! Managing for results is important for support services, projects and programmes. Service functions results are inside the organisation while programmes or projects produce results with and for outside stakeholders (pages 17, 18, 19 & 20).
4. Look out for the Logic Chain! It is central to most results-based approaches (page 10).
5. Each part of the Logic Chain has a Role to Play. It is important to understand inputs, activities, outputs, outcomes, impact (page 21).
6. The Logic Chain may be linear, but that doesn’t mean you have to use it in linear order. Project design does not usually begin with inputs. Developing the logic chain should be like putting together a puzzle.
7. It is important to get behind the language of Results. Don’t fixate on
definitions; focus on the essential idea - linking what you DO, with what you want to CHANGE.
8
Results-Based Management Tools & Techniques
8. You need to understand the context. Knowing the constraints and enablers
allows you to build in the right activities and/or make the right assumptions
for project success; it is a way to assess the risk that your project won’t succeed (see page 22).
9. Choosing the right information to gather is a tricky task. Good indicators
help you know whether you are making progress – but getting good ones is not easy (see page 23).
10. Use results-based approaches both for ACCOUNTABILITY and
NAVIGATION. RBM as a management tool mainly helps you make wise
management decisions, but its better quality data also serves your relationship with your partner or donor.
11. Results-based planning can help you think through how a project will benefit different people differently. Diversity and Gender Equality are important and every project should consider and measure how men and women of different
backgrounds are included, and how specific groups benefit differently at each level.
12. There are other ways to look at RBM. The Splash and Ripple metaphor
conveys complex information through analogy. It can help us understand results-based planning approaches (see page 5).
13. Properly completed, with lots stakeholder dialogue, just three planning tools can set you up as a results-focused manager. The results logic model (Page 21), the Performance Measurement Framework (page 23), and the Risk Analysis and Management Table (page 22).
Lake Victoria Basin Secretariat
9
A results-oriented management system calls
how the actions we take can make a difference.
activities and outputs, toward the longer-term
you seek and, in particular, clarify what in your
for managers – at all levels – to look beyond changes that we call outcomes and impacts.
It also demands attention from the beginning
of a project to its end. You must clearly define
what results are wanted, track performance and adjust along the way to improve efficiency and effectiveness.
RBM is not a complicated idea: it is easiest to
This image can help you visualize the change
environment you can actually control, and what you can only influence.
The splash and ripple metaphor helps us to
understand RBM. Dropping a rock into a pond shows us these important points: •
The meaning of, and the dynamic
understand when you have a mental picture of
relationship between, labels on the
be worth a thousand words. It conveys complex
potential that the output creates, while
symbols. Here, we use an image of a rock
influence by unleashing that potential on
how it works. Like a picture, a metaphor can
Results Chain (e.g., the splash is like the
information by likening it to widely recognized
the ripples are like the outcomes you
dropped in water (Splash and Ripple) to show
the pond);
Control
Spread
Inputs
Activities
Output
Outcomes
Impact Fig 3. Splash & Ripple
10
Results-Based Management Tools & Techniques
•
How Constraints and Enablers - other disturbances in the pond - can also
•
influence things. It is important to try
the pool can see the rock and the water
Assumptions and assess Risk;
Your control is strongest where the rock enters the water. Control falls off as the splash spreads out into a ripple;
management their own perspectives to
enrich the process - each person around
to account for these when you make •
People bring to planning and
from a different point of view; and lastly •
Being clear about limits is important. For example: »
whose vantage point you use to define the activity or project - or, who/what
•
Your sphere of influence widens as the
•
Attribution falls off as the height of the
»
•
Observing the Splash and Ripple effect
»
should drop the rock;
ripples widen;
who is directly engaged in a project or, who/ what is in the ‘splash’; and
outward moving ripple also reduces;
who or what the project influences
the first time allows you to improve the
indirectly –who/what is in the pool,
next time;
edge.
way you throw rocks into the pond the
from close to the splash to the pool’s
Key RBM Definitions Several words are highlighted in our discussion
Activities
in a precise way. You need to know that meanings
inputs, such as funds, technical assistance, and
of RBM: that is because RBM uses these words
Actions taken or work performed through which
can shift a little from source to source. Perhaps
other types of resources are mobilized to produce
the most widely accepted source is the OECD-
DAC. The many donors that support LVBC will understand these. Here are definitions of key RMB terms in alphabetic order, with a bit of
extra information to help you understand them. Section IV will guide you on how to use them, but throughout the document we will provide you with tips.
specific outputs. Activities describe what the
project does. In the “Splash and Ripple” example, this is the act of throwing the rock. You have the most control on the act of throwing it.
Assumptions Are the conditions that must be in place for the project to progress as planned toward intended results. For example, reaching all the results in
Lake Victoria Basin Secretariat
11
the LVBC Strategic Plan assumes that LVBC can mobilize sufficient resources to put into
action all the activities in its plan. If it is possible that this will not happen, then this should be
identified as a risk (See the definition below).
Constraints Negative conditions or influences in the context that could hinder progress.
Enablers Positive conditions or influences in the context
Inputs The financial, human, and material resources
used for the development intervention. Using the “Splash and Ripple” example, the person throwing the rock in the pond is a Human
Resource. The rock is his or her Equipment. You can control the size and shape of the rock and the strength of the person throwing it.
Outcome The likely or achieved short-term and mediumterm effects of an intervention’s outputs. This is
that could help progress.
a change that should logically occur after one or
Impacts
“purpose” level. They are usually achieved by the
Positive and negative, primary and secondary
long-term effects produced by a development
intervention, directly or indirectly, intended or unintended. The Impact is the highest-level
result that an organisation, policy, programme, or initiative can cause, and results from one or
more outcomes. The impact is at the goal level
more outputs are achieved. Outcomes are at the
end of a project/programme, and are usually seen as a change of behaviour or practice level among
beneficiaries. In the “Splash and Ripple” example, they are the ripple. You have direct influence over these changes, but not control.
Outputs
and usually represents an organisation, policy,
The products, capital goods and services that
takes the form of a sustainable change of state
include changes that result from the intervention,
only contribute to this change; you must assume
Outputs are the first results of your project – the
contributions too.
the organisation, policy, programme, or initiative.
Indicators
short-term and are usually seen as an increase in
programme, or initiative’s reason for existing. It
result from a development intervention; may also
among beneficiaries. Usually your activities can
which lead to the achievement of outcomes.
that other actors out there are making their
Splash. They are tied directly to the activities of
The measures that give you evidence or proof that the project, programme or initiative is
In terms of time frame and level, these are
awareness, skills or access among beneficiaries.
You have a lot of control over how these turn out.
progressing toward intended results.
12
Results-Based Management Tools & Techniques
Results
Risks
The output, outcome, or impact of a development
The chance that things may happen to prevent
intervention.
Results-Based Management (RBM)
your project, programme or initiative from
achieving its intended results. For example, to
achieve the results of our strategic plan we need
A management strategy that focuses on
financial support. But with the global financial
and impacts.
mobilise as much funding as we need. Risk has
performance and achieving outputs, outcomes
Results Chain (or Logic Model) The chain of events for a development
intervention which specifies the steps needed
to achieve desired objectives—beginning with
crisis there is a risk that we may not be able to
two aspects: how likely it is that the event may happen, and if it does, how much it will affect results.
Figure 4 shows how all the defined components work together in the RBMS.
inputs, moving through activities and outputs, and culminating in outcomes, impacts, and
feedback. Sometimes called Change Theory.
Key RBM Concepts While different donors use slightly different names and definitions, the idea behind the
•
approach is the same. RBM promoters want to
worldwide, there is pressure on those who
implement development programmes and
projects to describe, then track the results of their work using RBM concepts and tools. Applying
perspectives; •
Logical sequencing of inputs, activities,
•
Considering how external factors may
RBM requires a clear vision of what you want
to achieve, why you want to achieve it and how
well you are achieving it, so you can act to correct things where needed.
RBM places value on the following:
Lake Victoria Basin Secretariat
programme/project/service area design
and implementation by including more
see more evidence that funded activities produce long-term benefits. In public administration
Stakeholder participation - enriching
outputs, outcomes and impacts;
help or hinder a project’s progress along this logic flow;
•
Identifying both quantitative and
qualitative indicators and information gathering methods to help participants track progress toward results;
13
•
Organisational learning and
accountability - RBM generates knowledge and understanding so
that participants can refine activities/
programmes and projects as they work
their way through the planning cycle. It can also assure donor stakeholders who want to know how and for what that funds are used; and •
Flexibility – RBM accepts that a plan or performance framework must stay
open to change as outside circumstances influence progress and as project
participants learn more about the results of their work.
At its most basic, RBM is a way of thinking,
with a set of tools to help us to convert ideas into activities and anticipated changes.
The table shown in Figure 5 gives examples of
the inputs we typically need and the activities we typically do. It also highlights what changes we usually seek in outputs, outcomes and impacts. Activities, outputs, outcomes and impacts all represent distinct steps in the causal logic of
a policy, programme, or investment. (We will
discuss the logic model and other RBMS tools in more detail in Section III).
The first two levels (inputs and activities)
address the how. The next three levels (outputs, outcomes, impacts) are the actual changes that take place. Outputs and outcomes are usually
Constraints
Enablers Indicators Inputs
Activitiy
Output
Outcome
Impact
Fig 4. Components of the RBMS. 14
Results-Based Management Tools & Techniques
related to the what of a project, while impacts are related to the why – the development problem we face.
A Progression of Results - Typical Terms Impacts
Key variables that change as we move from inputs toward impacts are: •
Outcomes Performance Improved Institutional behaviour changes – End-user benefits – Large-scale policy & programming shifts – Social mobilization
Time – Impact statements are set
outside the timescale of the project. As we said in the definitions, outcomes
Outputs
represent changes sought within the
time frame of the programme or project. Outputs are immediate changes that emerge well within the project time
Control/Influence – At the output level, the project or programme implementing
Capacities Developed Institutional exchanges/collaborations – Applied knowledge – Functioning infrastructure – System improvements/adaptations – New/improved leadership – Aligned decision-making
Activities What We Do Research – Build – Train/Mentor – Promote – Procure – Convene
frame directly because of its activities. •
Conditions Changed Social – Economic – Civic – Cultural – Environmental
Inputs
What We Need Staff – Equipment – Supplies – Office Space – Services
team should claim results over which they exert “reasonable” control, given
Fig 5. A Progression of Results
time, resources and prevailing conditions
who feel some type of benefit from the
outcome level, this control diminishes to
spread even further out, to include whole
for the programme or project. At the
little more than direct influence. At the impact level, influence diminishes even
further. Your influence may be indirect or
difficult to distinguish from the influence
initiative. At the impact level, benefits population groups.
Do consider variables of time and control when defining expected results.
of others. •
Spread Effect – At the output level,
change usually centres around a defined group of people that have been directly involved in the activity, or around a
system or piece of infrastructure that
has been put into place. At the outcome
level, the change often “ripples” outward to include a larger group of people
Lake Victoria Basin Secretariat
15
Analytic Framework of the LVBC RBM System This section outlines the analytical framework
has been running and reporting on an array of
how RBM fits in to the organization.
provided by the above mentioned strategy
of the RBMS for LVBC. It addresses where and
LVBC Seen From Three Perspectives
projects more or less directly within the frame documents.
The diagram in Figure 6 sets out three scales
Applying RBM at multiple scales
Programme and Project. The institutional scale
of results at three levels, shown in Figure 6: the
of organisation within LVBC: Institutional,
Within the RBMS is an aligned and nested set
is the largest. Here we see LVBC as an entity
institutional level, the programme level and the
within the main body of the East African
Community, with its specific development role set out in the following documents (among others): •
EAC - The Vision and Strategy
Framework for the Management and
Development of the Lake Victoria Basin (2003); •
• •
project level/service area:
At the Institutional Level – illustrated by the
top wide ecosystem landscape - we see a results-
based planning and management perspective for the Commission itself.
The impact statement aligns closely with the
LVBC vision. Outcomes are closely aligned with the Institutional Framework found in Article
Protocol for the Sustainable
33 of the Protocol for Sustainable Development
(2003);
with the five policy areas identified in the LVBC
4th EAC Development Strategy (2011-
Management and Development of the Lake
Development of Lake Victoria Basin
16);
LVBC Strategic Plan (2011-16)
At the middle scale, we see LVBC programming setting out strategic themes, each shaped by the
of the LVB. These, and the outputs, also align
Shared Vision and Strategy Framework for the
Victoria Basin. The Scope of Service in Article 3
of the Protocol serves as a menu or filter to guide the kinds of activities that the Commission can support.
statutory documents. At the smallest scale, we see
The institutional level is the most appropriate
of these again operating within the programme
Council on Lake Victoria Basin and the
LVBC as a collection of discrete projects, each
themes and the legal framework of the EAC. In LVBC’s history up to now, the programmatic
scale to guide deliberations between the Sectoral LVBC. Working to its full potential, this scale
“middle” has been absent. The organisation 16
Results-Based Management Tools & Techniques
Thinking about Scale in Planning and Management – Three Images One way to think about scale perspectives is to imagine yourself flying high above a city like Kisumu as an airliner would. From the window, you see most or all
of the City in one view. This high, you cannot see bicycles, cars or people in the streets, but you can appreciate the city’s layout. As you fly closer to the ground,
you no longer have such a wide view of the city, but you would start to see cars, bicycles and people. You trade completeness for detail. When using RBM, you must decide what height is best for your management and reporting needs.
We see LVBC using an RBM system with three scales of analysis to match the
viewpoints we outline above. And we propose paying particular attention to the middle one. The diagram in Figure 6 makes the case for this.
Lake Victoria Basin Secretariat
17
Nested Scales
Acti viti es
uts Inp
INSTITUTIONAL SCALE Scope of Cooperation (Article 3, a-m)
LVBC Vision Institutional Framework
LVBC
(Article 33-2, 3)
acts Imp
Output s Programme Area
Outcomes
Other Institutions
Acti viti es
uts Inp
PROGRAMME AREA
Project
Output s
acts Imp
PROGRAMME SCALE
Programme Area
Outcomes Acti viti es
uts Inp
PROJECT/ SERVICE AREA
Output s
acts Imp
Project
PROJECT SCALE
Outcomes Fig 6. Nested Scales 18
Results-Based Management Tools & Techniques
should enable the Council to plan and manage using evidence of achievement that relates to Commission objectives.
Each activity at this institutional scale is equal to a programme or sector of focus.
•
Social Development
•
Economic Development
The first of these is Institutional Development.
It concentrates on enabling organs of the LVBC to work effectively together, and with the larger
At the Programme (or Sector) Level – the “bend
community of development partners. The other
see a collection of programme areas. Each has
which, taken together, cover the Scope of Service
closely tied to the institution’s strategic priorities.
At the Project or Service Area Level – pictured
in the river” in the middle image of Figure 6 – we
three define essential programming themes
a planning and management perspective that is
set out in the Protocol.
Programmes are usually bigger than projects,
in the bottom image as the bird standing in
Operating at their best, programmes are greater
collection of project and service area planning
programme level that LVBC decision-makers
principles.
and continue beyond donor funding cycles.
the water at the bend in the river – we see a
than the sum of their project parts. It is at the
and management plans. Each follows RBM
can concentrate on their niche role as part of a
Our review of LVBC’s current project portfolio
regional organisation “to facilitate, coordinate
and promote”. How to define programmes is an
important question for the Commission. For this, it is key that it remains within legal limits set out in the Protocol for Sustainable Development of the Lake Victoria Basin.
and donor requirements, shows no need here for fundamental changes to planning, monitoring and reporting practices. The planning and management systems that LVBC uses for
various donors are compatible with RBM. All seek logical connections between actions and
Four programmes were identified then validated
results, all require measurement systems that
Ministries from Partner States at the 2nd LVBC
risks, and all expect to gauge reporting against
in Nairobi, January 23rd – 25th, 2013.
capacities of project teams to use results-based
The four programmes are:
what they already have in place. This will help
by representatives of EAC and Focal Point
use indicators, all carry some examination of
NFPO & MEAC Affairs Coordination Forum
expected results. The focus is on building the
•
Institutional Development
•
Environmental Stewardship and Natural
planning and management methods to refine them to move donors toward accepting an
approach harmonized around LVBC’s RBMS and to lower administrative costs.
Resource Management
Lake Victoria Basin Secretariat
19
By contrast, LVBC service areas organised under
At the programme management level, we
Management are now accustomed to annual
strategy into a set of programme or sector
Finance and Administration and Knowledge activity-focused planning, with monthly and
quarterly reporting. Therefore most service area managers are less familiar with results-based management concepts.
The project/service scale is the basic level of
analysis in the proposed RBM system. For the
system as a whole to succeed, quality planning,
management and reporting at this scale is crucial. Developing the RBMS, it will be vital to ensure that at least some data from each project can be combined across all projects, to build a higherlevel programme or institutional picture of achievement.
Functions of the RBMS
see the institution converting the operational responses. These are more detailed five-year plans for each programme, describing the logic that
sets core areas of work and associated outputs,
outcomes and impacts. These give indicators by which to track progress and a methodology for doing so. There are opportunities here, also, to
understand programme risks – factors that could
get in the way of progress if left unchecked – and means for dealing with them. Within the five-
year-cycle, programme managers assemble annual plans and track and report on progress.
At the project level, we foresee LVBC staff
developing and implementing projects that fit within the Scope of Service set out in Article 3 of the Protocol, which match programme
The RBMS has a planning and management
outputs and outcomes. Projects will vary in
programming and ground level project levels, as
individual sponsors/donors. At this level, project
role to play at high level strategy, mid- level
size and length, and with the approaches of
shown in Figure 7.
and programme staff are accountable to both
At the strategy level, we see LVBC formulating
The challenge for the RBMS is to harmonize
a five year operational strategy for the
institution, that draws from social, economic
and environmental scans of the basin landscape,
individual project donors and to the LVBC itself. requirements in planning and reporting among these actors.
consultation with member states, an accumulated
The schematic in Figure 8 introduces all of these
continuing recognition of the mandate described
it LVBC’s RBMS cycle. It shows a sequence
into the operational strategy should be an end-
strategic, programme and project levels. The
understanding of LVBC results to date, and a
functions as a five-year cyclical flow. We call
in the statutory documents. One particular input
of eight distinct processes, spread across the
of-strategy evaluation from the previous cycle.
cycle begins and ends at the institutional level.
Strategic planning informs Programme Planning
which in turn shapes the design and management of projects on an annual basis. Programme level 20
Results-Based Management Tools & Techniques
monitoring and reporting continuously informs
RBMS Functions
progress as compared to expected programme
outputs and outcomes. This routine relies on the quality of project plans, data collection systems
and reporting. The cycle ends by examining the performance of each programme in relation to Strategy
Programme Management
the LVBC strategy and an understanding of changing conditions within the Basin.
Project Implementation
Project Development
Fig 7. RBMS Functions
Uses and Users RBM gives us a way of thinking and a set of
extension, results-based performance assessment
use this information to:
human resources management.
tools to track progress against the plan, and to
•
Make management decisions
•
Service accountability relationships; and
•
Learn on the go
As discussed above, RBM can be applied
consistently across multiple scales: at very large
scales, to sectors, institutions or programmes; at middle level scales to projects or departments;
and at the smallest scale, usually teams. And by
Lake Victoria Basin Secretariat
carries the results approach into the area of
At larger scales, the three most likely users of RBM approaches and tools are: •
Donor level officers. They seek concise, concrete results reporting that measures progress against planned achievement. They usually want to be assured that
the project is being implemented with due diligence to seek the best possible outcomes. RBM and the logic model
21
START
E
YEAR 5 of nd
luation y Eva g e t Stra
Strategy P lanni ng
g
Programm e Pl ann Annual in W an
ork Pl
Develop Project Proposal
Information is exchanged between levels throughout the cycle.
YEAR 1
Strategic Level Programme Level Project Level
Develop Project Concept
Implement Project
YEAR 4
YEAR 2 Mon
i t o r i n g & Re p o r t i n g
RBMS 5-Year Cycle
YEAR 3 Fig 8. RBMS 5-Year Cycle
(or related tools like the LFA) are core
risk frameworks and performance
contracts. As public managers, officers
the data these tools produce. The better
features in most programme/project
measurement frameworks but they use
and their agencies are champions of
they know the principles that guide
RBM. •
Implementing/coordinating agency
executive leaders or governance bodies. They seek proof that the programmes/ projects they oversee: a) align with the agency’s vision, mission and mandate;
and b) achieve relevant results in a timely way. This type of insight helps them to make strategic management decisions for their agencies. They may be less
likely themselves to craft logic models,
22
RBM, the better they can use the system to make wise executive decisions. •
Programme/project or departmental
managers. These are individuals or teams with a budget and a mandate to spend
it. RBM tools and approaches help these groups to engage stakeholders and, with
them, to transform project concepts into workable plans and ways to document results. As the main practitioners in
Results-Based Management Tools & Techniques
RBM, managers at this level should
master its power as an analytical tool.
RBMS Users at LVBC As we mention above, RBM is a management
approach that focuses everyone in the institution on achieving outcomes. It demands that they
track their progress toward the destination but allows them flexibility to change the ways and means they use to arrive there (i.e. that they
achieve intended results). Use of the RBMS will permeate all of LVBC.
Managers and staff at all levels, and in all areas – services and project teams – will use the system
to plan, monitor, report and evaluate how LVBC implements its strategies, policies, programmes
and projects. The table in Figure 9 outlines these roles and responsibilities.
Programme Steering Committees and Sectoral
Committees. Neither group exists yet, although the Protocol provides for them.
Project Coordinators and Heads of Units are responsible for results-based planning and
reporting at the project or service area level. Here, planning and management cycles and reporting
requirements are negotiated with donors. Project cycles will likely be five years or less. Donor
needs will influence formats and descriptions
in planning and reporting documents. Project
coordinators will submit their plans and reports to several user groups, especially: their Project Steering Committee, Sectoral Committees, Donors and the DES.
Wherever you work within LVBC, to stay on track you will need:
The office of the Executive Secretary (ES) will
•
Clear objectives, activities, outputs,
the high-level “institutional” logic model and the
•
To measure and evaluate as you go.
results reports every half-year to the Sectoral
•
To adjust plans and approaches to
The ES will also sign off on the LVBC’s results
•
To report on results.
“champion” the RBMS at LVBC. It will see that PMF are reviewed every five years and will issue Council of Ministers and EAC Secretariat. reports to the general public.
The Deputy Executive Secretaries (DES)
Finance and Administration and Programming will take the lead for results-based planning
and reporting at the programme level. They will ensure that programme level logic models and
PMFs are renewed every five years, in step with
outcomes and measures.
account for what you have learned and;
The RMBS has tools to help you do all of these things. We discuss them in Section III. Section IV outlines how to use them as you work and
Section V provides additional tools for project
management that can help you stay on track and achieve results.
the Institution’s five- year strategic planning
cycles. Primary user groups at this level will be
Lake Victoria Basin Secretariat
23
System Accountabilities Scale
Responsible Body/ Staff Leads
Planning & Reporting Deliverables
Frequency
Primary User
Institutional
Executive Secretary (ES)
Logic Model & PMF
Review every 5 years
ES
Reports
Semi-annual
Sectoral Council of Ministers
Annual Report
EAC Council of Ministers General Public
Programme
Project / Service Area
Deputy Executive Secretary (DES)
Logic Model & PMF
5-Year with Annual Report
DES
Reports
Semi-Annual
Sectoral Committees
Regional Project Coordinators / Heads of Units
Logic Model & PMF
Dependent on Project ToRs
Prorramme Managers
Reports
Quarterly (narrative)
Donors
Monthly (financial)
Fig 9. System Accountabilities
Governance – Management Relationships The link between governance and management at
At the programme level: the programme
Sustainable Development of the LVBC (2003).
Committees. As of now, neither programme
LVBC is well established in the Protocol for the Figure 10 shows parallel levels of governance
for management. At the institutional/strategic level, the Executive Secretary represents the
Commission to the EAC Sectoral Council of
Ministers. Article 40 elaborates on this. Article
35 of the Protocol states that the Council’s role is, among other things, to direct overall policy
and guide implementation of programmes in the
Lake Victoria Basin, and to consider and approve Commission work programmes.
24
manager role would be in contact with Sectoral manager positions nor Sectoral Committees
exist in practice. Article 37 states that Sectoral Committees are to include senior officials of partners states, heads of public institutions, representatives of regional institutions, and representatives from sectors covered under
Article 3 of this Protocol, including those in
business, industry and Civil Society. Further, each of these bodies is to take a national
focus that anchors it in each partner state. As
Results-Based Management Tools & Techniques
sector-focused multi- stakeholder platforms,
At the project level: the project coordinator
implementation of programmes in the Basin.
committee, at least where a project’s size or
they are expected to oversee preparation and
has historically related to a project steering
complexity warranted. It happens that this multistakeholder entity is not explicitly mentioned in the Protocol.
RBMS Management Structure Results-Based Management practice is not
The PMET works directly with project
integrated approach to programme and project
management perspective, PMET would shift
explicitly listed as a Commission function. This planning and management was only just being recognized when the Protocol was written in
coordinators. As it develops a programme
from analysing projects to analysing programmes.
2003. However, Clause “d” of Article 33 does
name as a function, “monitoring, evaluation and compliance with policies and agreed actions”.
LVBC Governance Relationships
The Commission has developed a Planning, Monitoring and Evaluation Team (PMET)
within the Office of the ES. The PMET is to be
Sectoral Council of Ministers
Executive Secretary
Sectoral Committees
Programme Managers
Project Steering Committees
Project Coordination
LVBC’s main “home” for the RBMS. The PMET brings together the following staff roles: •
Internal Auditor
•
Monitoring and Evaluation Officer
•
Planning Officer
•
Senior Accountant
•
Budget Officer
*dashed outline indicates “not yet established” Fig 10. LVBC Governance Relationships
Lake Victoria Basin Secretariat
25
LVBC - Monitoring and Reporting Arrangements Executive Secretary
PMET Internal Auditor M & E Officer Planning Officer Sr. Accountant Budget Officer
DES - FAD PAO, SA, HIRO
Institutional Dev Programme
Service Areas (Internal Audit)
DES - PP PDO, RM, MSO
Economic Programme
Environment & Natural Resources Program
Social Programme
Projects (M & E)
Projects (M & E)
Projects (M & E)
*dashed outline indicates “not yet established” Fig 11. Monitoring & Reporting Arrangements
As Figure 11 suggests, the PMET relies heavily
they’ve agreed with project donors. Internally,
functions at the level of the project and service
on a quarterly, semi annual and annual basis.
on inputs from the M&E and internal audit
area. Without good data at this level, RBMS integrity is compromised.
Current status of Reporting within LVBC
the Secretariat reports at an institutional level
Externally the Commission reports semi annually to the Sectoral Council of Ministers for LVBC
and Council of Ministers. The Commission also
prepares an Annual Report for the general public
As a regional intergovernmental institution
to read. Efforts have been made in the last year
has complex accountability and reporting
date has covered activities.
projects prepare monthly, quarterly and semi-
A New Emphasis on Project Development
level. Projects report on progress and finances
project development. Like the above-mentioned
with multiple donors and stakeholders, LVBC relationships. Internally departments and/or
or so to refocus on results, but most reporting to
annual reports for the Secretariat’s executive
The LVBC has created a tool to encourage
quarterly or semi-annually, based on what
structure, the Project Development Team (PDT)
26
Results-Based Management Tools & Techniques
should bring together service area and technical expertise case-by-case, to create winning
LVBC - Project Development Arrangements
development projects that fit LVBC’s scope of service.
As with the PMET, the PDT resides in the office of the ES, in this instance led by the Programmes Officer (Project Development and Partnerships). As LVBC develops its programme management
Executive Secretary DES Program & Projects PDO
planning, programme officers would also lead the project development process, as seen in Figure 12.
Program Managers Environment & Nat. Resource Economic Development Social Development
Project Development Team (PDT) (RM/PDO & others to be determined by expertise requirements)
*dashed outline indicates “not yet established”
Fig 12. Project Development Arrangements
Lake Victoria Basin Secretariat
27
28
Results-Based Management Tools & Techniques
III. RESULTS-BASED MANAGEMENT SYSTEM DESIGN TOOLS This section introduces four tools to help
for LVBC. These tools have been distributed
and programmes and tell their stories as you
Appendices to this Guide. They are the:
you design realistic, results-focused projects implement them.
Four core tools The RBMS uses four core planning and
management tools at all levels. Templates and
tip sheets for all of these have been developed
to LVBC managers. Templates are in the
•
Logic Model (Appendix A) – a flow diagram that describes the project
or programme’s “theory of change”,
expressed as inputs and activities and
the way they are converted into results - short term outputs, medium term outcomes and long term impacts.
Lake Victoria Basin Secretariat
29
•
Risk Analysis and Management Table
large amount of information that always
could hamper progress towards desired
and to use the most useful. The tool
(Appendix B) – to identify factors that
masses up between reporting periods,
results, tracking their influence over time
focuses attention on accounts of progress
and planning appropriate management
toward planned results, rather than on
responses. •
activities completed.
Performance Measurement Framework (Appendix C) – to set up monitoring and evaluation activities. This tool
focuses on indicators – predetermined evidence to look for – to confirm each
planned result. It helps users to identify
what information to gather, from where, how and when. •
Results Reporting Template (Appendix D) – helps managers to sift through the
Results Logic
Impact Change in state or conditions.
Outcome Behaviour change; adoption of new practices; performance improved.
Outputs Capacity developed; change in knowledge, awareness, skills, abilities; change in engabling environment
Activities Project and programme activities undertaken, e.g. target beneficiaries trained; studies produced; draft policies circulated; harmonized laws drafted
Inputs Financial resources (budget); time (schedule); human resources; support services; equipment and facilties
Fig 13. Results Logic 30
Results-Based Management Tools & Techniques
Logic Model How
Inputs
Activities
What
Outputs
Outcomes
Why
Impact
Fig 14. Logic Model
Logic Model (LM) At the centre of most results-based management
We often present results logic as a pyramid with
we state above, the Logic Model shows the
at the top. Figure 13 shows that direction. It
systems is the logic model, or logic chain. As underlying or logical relationships between
inputs, activities, outputs, and the outcomes of a given policy, programme or investment.
The LM should describe the work of the
project, programme or service area in one or two pages. This means that the higher the planning perspective, the more each statement will summarise.
inputs and activities at the bottom and impacts
illustrates the content you would expect to find in many LVBC project logic models.
However, you can draw an LM either vertically or horizontally. What is important is the causal links between each step. At LVBC we will
use a horizontal table for the LM as shown in Figure 13 (see also Appendix A). This is easy
to understand and to fill in using any standard
text processor. The LM tool designed for LVBC
provides a map to lead anyone who reads it from the inputs and activities to the final impact or Lake Victoria Basin Secretariat
31
“destination”. It describes the work of the project,
The logic chain is used as an analytical tool that
descriptions of the project/programme/service
resources and activities on one side connect with
programme or service area. It includes key
area, including the title, key contacts, donor and budget information, project timeline, vision/
goal statement(s) and a short rationale. We’ve
simplified the table in Figure 14 (and others in this section) for illustration purposes.
helps planners and managers to describe how results desired on the other. In particular, the
logic model enables us to adjust resources to get
the best results using an understanding of context and what time and resources are available. We describe this in more detail in Section IV.
Risk Analysis and Management Table The EAC has a Risk Management Framework,
and systematic process to help understand,
EAC, LVBC has adopted an integrated approach
Commission.
Policy and Strategy, and as an institution of the
to risk management. It is a continuous, proactive,
manage, and communicate risk throughout the
Risk Analysis and Management Table Description of Risk
Likelihood of Occurrence
Effect on Planned Results
Risk Mitigation Strategy
Person responsible to Manage Risk
Operational Risks 1 2
Financial Risks 1 2
Developmental Risks 1 2
Fig 15. Risk Analysis & Management Table 32
Results-Based Management Tools & Techniques
The Risk Analysis and Management Table
under each category. The next column shows
it to identify what could go wrong on the way
designates a person as a “champion” to manage
(Appendix B) is part of that process. Teams use toward the planned results. This tool helps: •
Provide an ongoing scan of key risks for
each programme, project and service area
•
Develop a systematic approach to risk
•
Contribute to a risk-aware culture.
risk-response or mitigation strategies. The last the risk.
There are three key areas of risk: •
management; and
Managers use the Risk Analysis and
Management Table to identify factors that
Operational – factors that could
interrupt activities (e.g. procurement problems, failure to receive vital
information on time, security problems, bad weather); •
Financial – factors that could interrupt
could hamper progress toward desired results,
how money flows to and through a
best to manage them. The table lists main risks
budget, misappropriation, etc); and
project/programme (e.g. failure to meet
track their influence over time, and plan how
(shown in the first column), analyses their impact and how likely they are to occur (the second
and third columns show the results of the risk
analysis). Those risks most likely to occur and to damage success are ranked highest on the table
Lake Victoria Basin Secretariat
•
Developmental – factors that could weaken results (e.g. substandard
infrastructure, loss of trained talent through “brain drain”, inability to
33
sustain commitment to a shared policy framework following a project).
Use risk analysis throughout the project cycle to prepare for implementation in uncertain environments.
Performance Measurement Framework We use the Performance Measurement
results statements from the logic model. The
systematically plan how to collect the right
and qualitative) – pieces of evidence that allow
Framework (PMF) (Appendix C) to
data to assess and demonstrate progress toward
achieving expected results. This tool ensures that we collect performance information regularly to enable real-time, evidence-based management decisions. PMF summarizes the way a project
or programme tracks progress. In our simplified illustration, the first column brings forward
second column holds indicators (quantitative us to measure capacity and performance. The
third column shows the status at the beginning of the project, compared against the indicator (or the results statement if you need a more
descriptive baseline measure). The fourth column names midstream and/or end-of-project targets, compared with the indicator. The fifth and sixth
Performance Measurement Framework Results Levels
Indicators
Baseline
Target
Data Source/ Method
Report Freq. & Resp.
Impact
Outcome
Output
Fig 16. Performance Measurement Framework 34
Results-Based Management Tools & Techniques
Tip #2: Examples of Quantitative and Qualitative Measures Quantitative
Qualitative
Number of
Change in patterns of
Frequency of
Perceptions of
Ratio of
Quality of
Percentage of
Beneficiary opinion or attitudes
Volume of
Level of understanding or skills
Results Reporting Template Planned Result
Indicators
Targets
Baseline
Actuals
Variance
Priorities for next period
Impact
Outcomes
Outputs
Fig 17. Results Reporting Template Lake Victoria Basin Secretariat
35
columns specify, for each indicator: information
us what to measure to show progress – can be
methods (surveys, document reviews,
IV. Performance indicators can be either
sources (documents, people/organisations);
participatory appraisal, key informant interview); how often they’re used (quarterly, annually, mid-
term); and the person charged with collecting the data.
Filling column two is one of the most difficult tasks in this table: finding the right indicators – setting the units of measurement that tell
challenging. We will discuss how in Section
qualitative (based on experience or perception)
or quantitative (discrete) as long as they provide a way to measure an output or outcome, so that we can gauge how a project, programme or
investment performs. We use quantitative and qualitative measures in different contexts, for different purposes.
Reporting (narrative) The reporting template focuses attention on
As such, it links directly to the final column
should mention activities, but only in relation
set priorities for the next reporting period.
results, rather than on completed activities. You to outputs. The fact is, managers at higher levels should be interested mainly in progress toward outputs and outcomes. For them to become
involved in activities and tasks is needless “micromanaging”.
The first four columns carry forward information
from the Performance Measurement Framework. The fifth column reports progress by output, and
when possible, by outcome and impact. Outcome
which uses the analysis of the immediate past to The complexity of LVBC’s accountability
relationships means that choices of formatting
results-based reports must be flexible. Templates like the one we’ve shown convey the reporting
requirements, but can be difficult to read when
they are filled out. Narrative formats can be much more readable, so long as they are structured (as in Figure 17) to focus on results.
data may not be available until partway into a
programme or project cycle. Impact will only be
a prognosis based on a current understanding of the operating environment and of progress to date.
The sixth column describes why any difference between actual and planned results occurred.
36
Results-Based Management Tools & Techniques
IV. HOW TO USE THE RBMS RBMS Approach LVBC has a clear vision for its RBMS: it wants
The RBM tools outlined in previous chapters can
improve organisational performance (results
a response; monitoring and reporting on
strategic planning and management tools to help management); and to generate substantive,
compelling reports (results measurement). The RBMS toolbox contains tools for strategic
planning, risk management, progress monitoring and reporting. It is designed to help make
evidence-based decisions and steer development efforts toward the objectives defined in its governing documents.
help at all stages: identifying a need; formulating implementation; and evaluating and planning
for a next round. These cycles are present with varying formality throughout LVBC, in both programme/project and in service areas.
RBM and LVBC’s Planning & Management Cycle Figure 18 outlines key areas where the various tools and techniques should be embedded in the Commission planning and management
Lake Victoria Basin Secretariat
37
Using RBM Tools in the Planning & Management Cycle Protocol for Sustainable Development of LVB
Align Plans to LVBC Objectives
1 Evaluate: Effectiveness, Efficiency, Relevance, and Sustainability
DOCUMENT
6 Adjust design and vimplementation on the basis of what is learned
Including: the Five Year Strategy, Annual Work Plans, the Rolling Medium Term Expenditure Framework, and Institutional and Programme Results Frameworks
Build Project and Service Area Concepts
2 For use in introductory discussions with donors or governance bodies
Monitor and Report on Activities and Results
Create or Adjust Designs
5
3
Using information to guide management and satisfy accountability relationships
Start Project or Launch Another Round of Work Activities
DEVELOP
Detailing results logic and the ways to manage risk and measure progress
4
IMPLEMENT
Work planning with an Organization Chart, Work Breakdown Structure, Output/Activity Matrix, Schedule, and Budget
Fig 18. Using RBM Tools in the Planning & Management Cycle 38
Results-Based Management Tools & Techniques
cycle. On paper, applying RBM tools and
LVBC’s institutional culture. In this section, we
straightforward. However, doing so in real life
planning and management cycle.
techniques throughout the planning cycle looks means embedding RBM as standard practice in
will discuss how to use the RBMS around the
Develop Initiatives Using RBMS into work packages that can be attached to
people and budgeted for; creating positions and DOCUMENT
DEVELOP
teams with roles, responsibilities and lines of
accountability; and scheduling activities so that they build toward expected results. IMPLEMENT
In a planning and management cycle, we develop, implement and document the work of LVBC - our projects and our service areas. At the
beginning of a cycle we are guided by mandate as defined in the Protocol for Sustainable
Development. We are also guided by knowledge of our ever changing working context and by what we have learned from the experience of
previous cycles. Here evaluations can tell us a lot. We develop concepts to address project and
service area needs. Some concepts gain enough support from decision-makers and donors to
warrant the development of full designs. These are appraised and some are funded. As the
project or service area ideas move closer to being a funded reality, we become more specific. We
go beyond the high level mapping of logic to the detailing of risk management and performance measurement and eventually to the details of
We learn as we go, with eyes and ears on how our teams are managing for results. Internal
monitoring generates information that typically goes into reports. Periodic evaluations usually provide a more independent and strategic
appraisal. Both kinds of documentation give managers insight to guide decision-making.
They help us to maintain accountabilities with those bodies that provide support.
All activities at LVBC should focus on achieving
the objectives set out in its governing documents. Five year strategic plans, annual work plans and
the rolling medium-term expenditure framework are all based on these governing documents. The
logic model (Fig 19) captures results expected of LVBC as an institution.
In Section II, we stated that four programmes
support LVBC in achieving its overall objectives.
Each aligns with institutional-level outcomes and outputs:
implementation. This includes breaking activities Lake Victoria Basin Secretariat
39
Institution: Lake Victoria Basin Commission Timeframe
On-going current phase five years (2011 - 2016 Strategic Plan)
Programs
Environmental Stewardship and Natural Resources Management, Institutional Development, Social Development, Economic Development.
Indicative Budget
S207.216.372
Mandate & Objectives
Activities
Article 33(2) of Protocol: Promote, facilitate and coordinate activities aimed at sustainable development and poverty eradication on the Lake Victoria Basin, specifically promote: - Equitable economic growth - Measures aimed at eradicating poverty - Protection of the environment within the Lake Victoria Basin - Sustainable utilisation and management of natural resources - Compliance on safety of navigation - Integration of cross-cutting themes gender, research, capacity building, information exchange, and public participation.
Outputs
Outcomes
Impact
1.1 EAC Partner States use a harmonized approach in their implementation of sustainable management, development, and conservation of the environment
1. Degradation and depletion of Basin environment is reversed, natural resources are used and managed sustainably (Policy area 1, Objectives c and d)
1. Environmental Stewardship and Natural Resources Management 1.1.1 Facilitate the development of and promote the adoption of harmonized laws, standards, policies for Transboundary Natural Resources (TBNR) management and bio-diversity of land, water, forests, fisheries, wildlife and minerals in EAC. 1.1.2 Promote and facilitate partner states adoption of ecosystems management approaches including clean production, climate change adaptation and conflict resolution. 1.2.1 Promote and facilitate partner states adoption of sustainable land use and natural resource management and conservation practices.
1.2 Regional trans-boundary ecosystems (watershed biodiversity) are protected, in practice
1.2.2 Promote and facilitate EAC partner states use of Lake Victoria Basin environmental health and natural resources data to support policy decisions and planning of sustainable management of TBNR
2. Institutional Development 2.1.1 Develop and implement effective administrative and financial management systems at LVBC 2.1.2 Strengthen strategic planning and programme/project management systems 2.1.3 Strengthen knowledge management systems at LVBC 2.1.4 Develop LVBC financial, human, and physical resources base to deliver on mandate 2.2.1 Promote and facilitate operational partnerships with EAC organs and institutions as well as key stakeholder groups 2.3.1 Promote and facilitate the involvement of government and non-government Basin stakeholders (public, private, and civil society).
2.1 LVBC has the operational, strategic, financial and knowledge resources to fulfill its catalytic roles and responsibilities
2. Appropriate institutions, policies and governance guide development planning and management in the basin (Policy area 5, Objectives 1 and 3)
2.2 LVBC is promoting, facilitating and coordinating actors in the sustainable development of the Basin 2.3 LVBC has established grender sensitive practices for civil socity participation in policy and programming
3. Social Development 3.1.1 Promote and facilitate the use of harmonized, gender-sensitive policy, legal and regulatory standards to design and deliver social services with EAC partner states
3.1 Social services in the Basin are more widely available and of higher quality
3.2.1 Promote and facilitate the use of services to control HIV/AIDS, reproductive health, malaria and other communicable diseases
3.2 Prevalence of HIV/AIDS, malaria, and other communicable diseases declines among the people of the Basin
3.3.1 Promote and facilitate the development of water supply, sewage treatment, public sanitary services, and waste management infrastructure 3.3.2 Promote and facilitate functional governance and management of water, sanitation and waste
3. Fewer households in the basin are living in poverty, and an increasing proportion are food secure and increasing their incomes (Policy areas 2 and 3, Objective 2 a and b)
A prosperous population living in a healthy and sustainably managed environment providing equitable opportunities and benefits (LVBC Vision)
3.3 The people of the Basin have greater access to water and sanitation systems that meet or exceed quality standards
4. Economic Development 4.1.1 Promote and facilitate the availability of navigation aids, meteorological and search and rescue services across the lake
4.1 There is improved transport, safety and security on Lake Victoria
4.1.2 Promote the widespread Lake Victoria community recognition and use of emergency rescue and weather alert services
4.2 There is increased economic productivity and improved access to markets across the Basin
4.1.3 Promote recognition of the provisions of the Transport Act and its regulations to Lake transport operators
4. Population growth within the Lake Victoria Basin remains within the carrying capacity of the land, with reduced morbidity and mortality (Policy area 4, Objective 2 b)
4.2.1 Promote and facilitate Partner states to maintain harmonized, reliable monitoring of identified security issues (e.g. piracy, robbery and trade in illicit goods) 4.2.2 Promote and facilitate agreements and protocols that will position the Basin competitively for investment 4.2.3 Promote and facilitate exploration and extraction activities, eco-tourism, agriculture and agri-business among other relevant sectors in the Basin to investors 4.2.4 Promote and facilitate the gainful participation of below subsistence and low income households in the informal economy 4.2.5 Promote and facilitate scalable technologies in fishing, farming and in farm/lake to market processing to the public and private sector
Fig 19. LVBC Logic Model 40
Results-Based Management Tools & Techniques
•
Institutional Development – covers
services and operations of the Secretariat
and enables it to produce other outcomes •
Environmental Stewardship and Natural Resource Management
•
Social Development
•
Economic Development
Good programme-level results rely on the
quality of project and service area level planning and reporting. Without the project and service
insights into outputs and outcomes, management cannot give strategic guidance, nor can it pass good data about programme outputs and
outcomes on to the institutional level. In turn, this cripples high level decision- making in
pursuit of the Commission’s objectives. Therefore
3. Build results-based plans and proposals.
Align Plans with LVBC’s Key Objectives LVBC’s projects are temporary activities set
up to reach particular objectives and then end.
Each project must produce unique deliverables or results, yet must also contribute to LVBC’s
overall objectives. The current LVBC Strategic Plan (2011-2016) clearly reflects this. The way LVBC activities nest and align at each of the above levels means that each new project or initiative must: •
Work within the activity areas set out in
•
Produce one of the outputs set out in the
•
Contribute to one of the outcomes set
the project/service scale is the basic focus of
analysis for the RBMS. For the system as a whole to work, quality planning, management and reporting here is vital.
the LVBC LM;
LVBC LM; and
out in the LVBC LM.
The LVBC’s LM, and those for each of the four
For this reason, this section focuses on how
programmes, are key documents for all project
based planning and management methodologies.
Managers need them to set the boundaries
project and service area teams can use results-
and service area managers who plan activities.
But the principles apply equally to the
within which they design all initiatives.
programme and institutional levels. We will show you how to:
1. Plan to achieve key objectives of the LVBC using the institutional and programme level Logic Models and PMFs. 2. Plan and present new project and programme concepts to donors and other stakeholder groups by using the logic model as an analytical tool.
Lake Victoria Basin Secretariat
How does this work in practice? As a manager,
you and your team start by identifying a problem. This may come from monitoring and evaluating past activities, from EAC priorities, or from
Partner States, Focal Points, donors or other stakeholders. There are many problems, and
LVBC cannot address all of them. So, first ask yourself:
41
Tip: On Logic Modelling When designing a project using a results-based planning approach, it is not wise to begin with a discussion on inputs – the first point on the logic chain. Rather it makes more sense to explore activities and impacts first – you might get the practical types in the
design group to think of the activities and the big picture thinkers to think about impact, then work toward the middle of the logic chain and end at the beginning (i.e. inputs). Developing the logic chain should be like putting together a puzzle.
•
Does this problem fit the mandate of
•
Can it be addressed within our areas of
•
Does it cross national boundaries? (The
LVBC (see LVBC LM in Figure 19)? cooperation?
regulatory framework. It shows how it could link to the LVBC LM.
Note: LVBC’s LM template does not include inputs. It starts at the activity level.
principle of subsidiarity means that
In the table, notice how results statements
issues; those are for nations themselves.)
Output level, the results relate specifically to the
LVBC must not address purely national •
project, namely harmonizing the legal and
Does it fall mainly within the Lake Victoria Basin? (If not, perhaps a
different organ of the EAC should be involved.)
If you can answer “Yes” to all of these questions,
then move to the first stage in the design process. This uses the Logic Model (Template attached in Appendix A) as a tool to assemble all the
information you collect about your initiative.
proceed from Outputs to Impacts. At the
activities that produced them; they are concrete and immediate. (There could be a lot of sub-
activities under each, such as “conduct a study”,
which would itself hold sub-sub-activities such as “develop TORs for the study”, “hire consultant”, “review and finalize study”, and “present and
validate findings”. You would develop these in the narrative part of your concept paper. You
don’t need to include them in the LM, but you
might want to put them in the Work Breakdown
The example table in Figure 20 sets out the kinds
Structure: See Section V). At the Outcome
that may be relevant to one component of an
programme. They point to changes that no
of Activities, Outputs, Outcomes and Impacts
level, the results focus on the purpose of the
environment and natural resources management
one activity could bring about. At the Impact
42
Results-Based Management Tools & Techniques
Aligning Your Initiative with LVBC’s Key Objectives & Mandate How?
What?
Why?
Activities
Outputs
Outcomes
Impact
1.1.1 Facilitate the development of and promote the adoption of harmonized laws, standards, policies for Transboundary Natural Resources (TBNR) management and bio-diversity of land, water, forests, fisheries, wildlife and minerals in EAC
1.1 EAC Partner States use a harmonized approach in their implementation of sustainable management, development and conservation of the environment.
1 Degredation and depletion of Basin environment is reversed, natural resources are used and managed sustainably (Policy area 1, Objectives c and d)
A prosperous population living in a healthy and sustainably managed environment providing equitable opportunities and benefits (LVBC Vision)
1.1.2 Promote and facilitate partner states adoption of ecosystems management approaches including clean production, climate change adaptation and conflict resolution.
Proposed Initiative Recommend between 5 & 8 action statements under headings like those set out below.
Immediate results – one statement for each activity – these will identify potential that flows from each activity.
Later results – the major changes the project can bring about within its life time.
A description of a preferred future – a vision to which the project will contribute.
Develop EAC standards for management of land and water resources.
Harmonized laws, standards, policies for trans-boundary management of land and water resources adopted in Partner States.
Partner states use a harmonized approach in their implementing sustainable management and development of land and water resources.
Degradation and depletion of Basin land and water resources reversed and they are used and managed sustainably.
Develop policies for management of land and water resources in EAC. Develop EAC laws for management of land and water resources in EAC. Promote adoption of EAC harmonized laws, standards, policies for management of land and water resources in EAC Partner States.
Fig 20. Aligning Initiatives with LVBC’s Key Objectives & Mandate Lake Victoria Basin Secretariat
43
level, the results extend beyond performance to
Figure 20 describes a large-scale programme
communities.
frameworks for a programme in a specific
include some material change for people and
Use the Logic Model to Formulate and Present New Project Concepts
scenario. You might be designing results
country or specific regions of a country. In this instance, your activities and results resemble the large scale programme framework. You
The Logic Model (Appendix A) is designed to
simply apply the programme within a specific
also make it a useful aid when building an idea
adjust your programme results logic to account
seeking financial support. This makes it useful
Or, you might design a specific project to nest
tell the story of your project in summary. This can
geographic area – say, a province. Here, you
with others, or a helpful marketing tool when
for the specific conditions of that locale.
for many tasks in the planning and management cycle.
within a programme of this nature. Let us
say it is a training initiative that only relates
Tables like the one in Figure 20 often make
to one activity in the programme. At this
think, “We will simply apply this formula to
somewhat. Your activities describe all that must
is deceptive. Yes, results logic applies certain
Outputs flow directly from these activities. For
ever “fits all”. Your LM must be custom-tailored
training materials” as an output of a curriculum
and resources you have available.
never show up at a programme level, but it
You must ground results logic to the time and the
outcome might be demonstrating competence
RBM look easy enough that you may be led to
smaller scale, the results perspective changes
our initiative!”. Sadly, this easy-looking shortcut
be done to design and deliver a training process.
patterns to development work. But no one size
example, you might seek “relevant, user friendly
to fit the specific problem, beneficiaries, context
development activity. This is a detail that should
resources you have. If you must complete your
activities in three months and you have $100,000, you should claim three-month, $100,000 results at the output, outcome and impact levels. If you have a three-year, $5- million dollar initiative, then your results should reflect this instead.
is important to a project. In this scenario, an
and confidence among participants after training. A good training design, skilled trainers, the right training materials, an accessible and suitably
equipped training site, and the right participants are all outputs that are needed to make this
outcome possible. Lastly, for the training project, improved understanding of the draft land and
water management policy might be an impact. So what we see is that, as we reduce the scale from
44
Results-Based Management Tools & Techniques
programme to project, we must also adjust the
results logic. Often, a project impact is equivalent to a programme outcome, while a project
outcome may be equivalent to a programme output.
Build Results-Based Designs Here are nine steps to prepare a Logic Model. As you see, you must do a lot of advance work
Tip: Increments Projects are often developed in
incremental steps as the project team develops a better and more complete understanding of the objectives and deliverables
(design discussions and research) before you assemble this document:
1. Start by identifying the problem you want
your investment to address, and the changes
you want to see to solve that problem. Make sure to analyse the context (cultural, sociopolitical, economic, and environmental) surrounding the problem.
2. Identify ultimate beneficiaries,
intermediaries, and stakeholders.
3. Ensure that the right people (e.g. technical specialists; implementing agency;
stakeholders; and beneficiaries) come to
the table. RBM is a participatory process. The process and methodology to select
RBM Core Design Principles - Logic Modelling Vantage point - clarify “who” is doing the work, and “who”/”what” stands to benefit Activity count and definition – Describe the work in five to eight activity areas, sharply differentiated from each other
Progression along the results chain – Formulate outputs, outcomes and impact on the
basis of an analysis of a) level of control – influence, b) spread effect, c) time required to bring about
Proportionality – make sure outputs and outcomes (especially) are realistic given the time and resources available; adjust if necessary
Communication value – use active language and present tense to write outputs, outcomes and impacts. Include a clear reference to the subject of the change/benefit
Lake Victoria Basin Secretariat
45
results and draft result statements should
should have one or more contributing
be as participatory as possible across a wide number of key stakeholders. Make sure to involve leaders, decision makers, women
and men, minorities and direct beneficiaries. Ensure that results statements address the
outputs.
8. Validate with stakeholders/partners. 9. Where required, write a narrative text to
concerns, priorities and needs of women and men, and girls and boys.
illustrate linkages. Explain how the proposed activities lead to the expected changes.
4. Identify your strategy to address the problem and the main activities that LVBC and its partners will undertake.
5. Identify outputs for each activity package. 6. Identify logical outcomes from the outputs. 7. Identify linkages. Check back and forth from activities to impact and back again to make sure everything flows logically. Activities
should support outputs. All activities should contribute to the outputs listed. Outcomes
What is a Result Statement? A result is a describable or measurable change that is derived from a cause-and effect relationship. It could be at an output, outcome or impact level.
A result statement outlines what a policy, programme, or investment is expected to
achieve or contribute to. It describes the change stemming from LVBC’s contribution to a development activity in cooperation with others.
The project will end when its specified termination date is reached, its objectives are
achieved, or the need for the project no longer exists, but it should produce lasting results.
46
Results-Based Management Tools & Techniques
Strong Results Statements Checklist Does it specify what, who, or where?
NO
YES
Is it measurable?
NO
YES
Is it clear what the change actually is?
NO
YES
Is it achieveable given the resources (time, money, etc.)?
NO
YES
Is it simple (contains one idea)?
NO
YES
Fig 21. Strong Results Statements Checklist
On Writing “Good” Results Statements To complete an LM you must write clear, concise result statements. The Results statement should:
1. Describe the change we want to see, not the work to do to get the change
2. Be simply worded. Avoid, for example, “how” details that use words such as “through” or “by means of ”
3. Where possible, put the subject - the actors
who will benefit from the change - up front
4. Be measurable 5. Be realistic and achievable within the budget
6. Apply to the development problem (or opportunity) identified.
7. Avoid tentative wordings like: “able to”,
“have increased capacity to...”, “contribute
toward”. These make results vague and hard to measure.
8. Be checked against the LVBC and
Programme Logic Models for examples of results statements that may apply to your
project. Make sure your results align with higher level results.
The Strong Results Statements Checklist (Fig 21) may help you to formulate your results.
and the time allowed
Lake Victoria Basin Secretariat
47
On “Connecting” Your Results Logically To test the logic that connects activities,
outputs, outcomes and impact statements, use
the “how-why” test. Starting with activities, ask the question “why?” The answers should help
you check connections to outputs. Do the same with outputs and outcomes. Then move in the opposite direction, and ask “how?” Again, the
answers should help you check connections back toward activities. Identify loose connections.
Either eliminate these statements or revise them.
This amounts to an important strategy discussion that could bring about changes in the proposed project design and budget.
On the Question of Control, Influence, Contribution and Attribution in the Logic Model 1. At the level of activities and outputs, the
implementing partner(s) has a high level of control. Use output statements – products,
– that you can confidently predict you will achieve as you conclude each activity.
2. At the outcome level, the implementing
partner(s) does not have control. Rather they have direct influence. Results statements should reflect the achievements that you
judge possible by the close of a planning
cycle, given the time and resources available to you as implementer (unit, project,
programme or institution as a whole). 3. At the impact level, the results statement
reflects your vision for the future regarding
the development problem you are addressing. Impact is long-term, so it will not be
achievable within the planning cycle. You
should be able to show you contributed as
one of many actors working for this preferred future.
services, knowledge, skills, awareness, etc.
Examples of Risk Responses
48
•
Accept the risk (for very low level risks, or risks with very high opportunity costs).
•
Refuse the risk (i.e. do things a different way to avoid the risk).
•
Reduce impact (i.e. buy insurance to mitigate financial loss).
•
Reduce likelihood (i.e. intensify outreach to build stakeholder engagement).
•
Research how to manage the risk.
•
Share the risk (i.e. distribute risk among several partners).
Results-Based Management Tools & Techniques
On keeping Activities, Results, Indicators and Targets Separate
Tip: On Stating Risks
Activities, results, targets and indicators are often confused with each other. When you develop
Look at a few results statements
your RBM tools, keep in mind: •
from the Logic Model. Risks
are threats or opportunities that
Activities are what you do. They describe
influence a result.
how you will transform inputs into results.
•
Risk statements should be clear, short and simple.
Results describe what will be different. Results statements identify what is to
Be careful that it is a risk (i.e. an
be accomplished at the output, outcome,
uncertainty) and not a problem (an
and impact levels. •
existing situation).
Performance indicators specify exactly
Ensure that it will effect your results.
what is to be measured along a scale or
dimension. They do not presuppose the direction of change.
•
Targets specify a particular value for
an indicator, to be accomplished by a specific date in the future.
RBM Core Design Principles - Risk Management Scope – consider: operational, financial and developmental risk areas. Specificity – detail the risk; avoid generalities. Priority focus – focus on risk areas where “likelihood” is higher and “effect” on success is greater.
Appropriate response – monitor where there the risk is imminent; mitigate where the risk is compromising performance.
Ownership – assign responsibilities for tracking and managing risk.
Lake Victoria Basin Secretariat
49
1. Risk Identification
The Risk Management Process - environment scanning - partner collaboration - selecting key risk areas
FEEDBACK
5. Evaluation
- improving the process - organizational learning - performance reporting
2. Risk Assessment
Understanding Context - communications strategy - gender considerations - stakeholder consultation - accountabilities
- decicion-making - adjusting - performance reporting
- impact & likelihood - risk ranking - risk tolerance
- determining options - mitigation options - implementing
3. Risk Response
4. Monitoring Fig 22. The Risk Management Process
Risk Assessment Matrix EFFECT Insignificant
Minor
Moderate
Major
Critical
Rare
LIKELIHOOD
Unlikely
Moderate
Likely
Common Fig 23. Risk Assessment Matrix 50
Results-Based Management Tools & Techniques
Performance indicators and targets are associated with the Performance Measurement Framework
Tip: Review Risk Analyses
(PMF) tool in the RBMS, which we discuss
below. But before we do that, the RBM design
Remember to review the EAC,
process requires us to identify risks.
LVBC and programme-level risk analyses: many of their risk and
Identify and Mitigate Risks
mitigation strategies may also apply
The East African Community has a Risk
to your project.
Management Framework, Policy and Strategy. LVBC has adopted an integrated approach to
risk management. It is a continuous, proactive,
implementing agencies and beneficiaries
and communicate risk across the Commission.
state what risks it sees from its unique point of
and systematic process to understand, manage, The Risk Analysis and Management Table
(Appendix B) is part of that process. This tool is designed to: •
Supply a continuous scan of key risks for
each programme, project and service area
(including men and women). Each group can view.
Ask this group to brainstorm as many risks as it
can – everything it can imagine that may hold up work toward planned results. Have each person
write the risks they identify on large sticky notes, one per note. Encourage them to be as specific as possible. Sweeping statements like, “lack of
•
Develop a systematic approach to risk management; and
resources” are far too general, and lead to fuzzy
•
Contribute to a risk-aware culture
mitigation strategies. An example of a specific
The risk management process is shown in
Figure 22. It shows a cycle that begins with risk identification (step 1), assessment (step 2), and the formulation of response strategies (step 3).
The results of these steps are documented in the Risk Analysis Management Table which is then used as a reference for teams when monitoring (step 4) and evaluating (step 5).
Conduct a Risk Analysis Session It is best to identify risks using a participatory
understandings and imprecise monitoring or
risk might be, “Profit seeking stands in the way
of (fishers or farmers) adopting (health and safety or environmental stewardship) practices”.
Each person (or group) should come to the front, read out their risks, and fix their sticky notes to
their preferred place on the risk matrix (Fig 23). The group as a whole should discuss each risk
and try to agree on its “likelihood” and “effect”. Then it should discuss responses – whether to monitor (and how), or to mitigate (and how).
model. This brings together a cross-section of stakeholders, including technical specialists, Lake Victoria Basin Secretariat
51
1. Under Risk Definition, write down key risks to the project
2. For each one, note the current risk level, its “likelihood” and its “effect”.
3. Give a summary of response strategies to manage the risk.
4. State the person responsible for managing the risk.
5. Monitor risks regularly and re-rate them
6. Incorporate handling of risk and learning
it generates as areas of inquiry in periodic evaluation.
Use the Risk Analysis and Management Table The Risk Analysis and Management Table is an RBM tool we use to identify factors that could
hamper progress toward desired results, to track their impact over time and to plan appropriate
management responses. The table lists the most important risks, with analyses of their impacts
when appropriate. A risk profile may change
and the probability of each occurring. It also
risk definitions and risk level. Track the use
gain in seeing risk as something that we look at
strategies; change Risk Responses where you
must be a constant process: we must review and
over time. As risks arise or fade, change the
summarises risk-response strategies. There is no
and the effectiveness of your risk response
only during the design phase and then forget. It
need to.
update the Risk Analysis and Management Table continuously, revising probabilities and new risks as they arise or change.
Make a Plan to Monitor and Evaluate Progress Using RBMS RBM is at its base repetitive: you focus on your
One way to ensure that you use data monitoring
there. You rate how you are doing to see if you
into meeting agendas. You can stay focused on
destination and track progress in order to get need to change your approach, your strategy,
your schedule or your inputs to reach the desired results, but the goal stays the same. Performance measure is thus vital to RBM: it is important to use monitoring and evaluation to collect
information on your progress and it is vital to
feed this information back into the planning and management process, so that LVBC can adjust plans and approaches at need. 52
to improve performance is by embedding RBM results by setting up the agenda around the key
project/service area outputs rather than activities. RBM gives us a way to align job descriptions of managers and staff working in particular areas to achieve the desired project/ service area outputs. This increases accountability.
The same results and monitoring data can be used in annual performance assessments and
Results-Based Management Tools & Techniques
appraisals. Assigning accountability for results at
•
frequency of reporting within the Commission,
This tool summarizes how a project or
the Secretariat allows us to tune the detail and so that achieved results are reported “up the
line” not activities completed. This enables all
managers to manage at scales appropriate to their positions.
Allow managers to make decisions based on evidence, in real time.
programme tracks progress. For each result,
it lists indicators that we will use to measure
progress, who measures that progress, how, and how often. The PMF also sets a baseline for
The key RBM tool for all this is the Performance
each indicator, and the target for some point
We use the PMF to:
are key markers for checking progress. You
•
partners, other donors, local stakeholders and
Measurement Framework (PMF) (Appendix C).
Systematically plan how to collect the right data to assess and demonstrate progress toward expected results.
•
in time toward the end of the project. These
should develop the PMF by consulting with
sometimes beneficiaries. All of these will use the information.
Ensure we collect regular performance information.
RBM Core Design Principles - Performance Measurement •
Reasonable precision – link indicators to planned results on the basis of the
following criteria: accuracy, reliability, cost effectiveness, disaggregation based on gender and other demographic variables, benefit to management.
•
Mixed methods – use a variety of data collection tools and techniques (formal and
•
Validation – where possible check back with key informants before using findings.
•
Ongoing monitoring – to determine progress towards the objectives and identify
informal) to capture insight; look for corroboration.
potential risks and problems early and take corrective action.
Lake Victoria Basin Secretariat
53
Performance Indicator Checklist Validity - Does it measure the result?
NO
YES
Is it cost-effective to collect the information?
NO
YES
Does it help understand how the programme or project affects men and women, and/or specific sub-groups of people differently?
NO
YES
Does it provide useful information for management decisions?
NO
YES
Does it communicate well to stakeholders (public, policy-maker, donors)?
NO
YES
Fig 24. Performance Indicator Checklist
Tip: Consistency To ensure consistency, indicators, baselines and targets should not normally be changed retroactively, and should only be changed by consent of all partners.
Assumptions and risks are critical elements of the project. They are first identified and then refined during project development but they should be monitored.
The validity of risks and assumptions should be checked regularly during implementation, and the information generated fed back to management.
54
Results-Based Management Tools & Techniques
Tip: On Using RBM to Adjust Under RBM you monitor constantly to ensure you are making progress toward the agreed
destination. This helps you to identify problems and design corrective actions. RBM allows you to: •
introduce a new activity
•
realign the budget
•
change the target (i.e. take more time to reach the destination)
But you don’t change the expected result: the “destination” remains the same. We can only take corrective action...we can only MANAGE, when we know where we are going.
Make a Performance Measurement Framework
3. Remember to look at the indicators used in the PMFs, for LVBC and the programme:
Preparing a Framework has twelve main steps.
your project should contribute to these
work (design discussions and research) before
framework will help to establish baselines,
You will need to do a fair amount of advance
and align with them. A robust indicator
you assemble this document.
identify trends and data gaps, and highlight
1. Place your output, outcome and impact
statements from the logic model in Column One (Use the “At-a-glance” guide in Appendix D).
2. Prepare a list of candidate indicators for each output and outcome and from the
impact statements in your logic model. We recommend strongly that you work with
partners to develop indicators and that you
build on what already exists where such data are available and of acceptable quality.
Lake Victoria Basin Secretariat
constraints.
4. Pay particular attention to breaking down
data to expose issues, and to the situations and views of people who suffer from discrimination.
5. Test your candidate indicators against the checklist in Figure 24; try to refine your statements to match the model.
6. Insert the indicators you’ve chosen in
Column Two, aligning each with its related output, outcome or impact statement.
55
7. Where you know baseline data, insert it in
Column Three, again aligned with its related indicator in the previous column.
8. In Column Four, insert target information that can be compared to the baseline data;
of information – is a highly challenging task
that will take trial and error. Indicators must give accurate and reliable evidence; be easy
to understand; and be useful to those making management decisions.
make sure to state both the value and the
Indicators point to the information you must
and data sources for each indicator should
difference. They cover the most useful identifiers
time frame (e.g. 30% by year three). Targets be decided by consensus among the key
stakeholders who participate in preparing the PMF.
9. In Column Five insert both the data source (e.g. documents, ministries, population
groups) and the methods you will use to
capture it (e.g. survey, focus group, document review, participatory rapid appraisal)
10. State the person(s) in the project team who will be responsible to collect the data and how often to collect it.
11. Validate your draft with key stakeholders before using it.
Working with Indicators These are the evidence, or proof, that you
need to show progress toward outputs and outcomes. Imagine the dials and displays
in an airplane cockpit. These give the pilots
gather to know if the project is making a of changes that a project causes. Writing
meaningful indicator statements is challenging. Indicators can be quantitative or qualitative. 1. For every output, outcome and impact
statement – ask yourself, “What piece of evidence will tell me that we are making
progress?”. Be as specific as possible – avoid
vague phrases like, “evidence of...” or “extent to which...” that will not aim you at the
precise data that you need to collect. Focus on evidence that will show you “progress
toward”, instead of only the moment a result is achieved.
2. Write your indicators to be “neutral” or
“non-leading” – keep the direction or the
target that you seek separate. (e.g. number of hectares reforested; before-after comparison of policy statements related to ‘x’ transboundary issue).
important information about the how well
3. Use the checklist (Fig 24) to help you refine
to its destination. Without these indicators,
4. Judge which indicators, if used at the
the plane is operating, and where it is relative the pilots would have little to guide them.
Drafting indicators – deciding which dials and
displays will give the project just the right kind
56
your indicator statements.
beginning of a project, will yield useful data. You will find that some work and some
don’t. A short narrative description about the
Results-Based Management Tools & Techniques
output, outcome or impact statement may be enough in some cases.
5. Make sure to make specific people
responsible for collecting specific data; ensure that they own that responsibility.
Figure 24 holds a checklist to help you choose the best indicators from a list of possibilities.
Techniques for Data Collection Here are a few common data collection
techniques. Some work better for quantitative
indicators and others for qualitative. You must use both to present a full picture of a project. You can even combine methods, for example by collecting stories from some users from a
Guided Interviews Where your questions are fixed – These help
you collect comparable information – especially important if different people are conducting
interviews. You might want to ask people who
have participated in the same activity to assess its merits. If interviewees answered differing
questions your audience will have less confidence in the results’ validity.
Closed, Fixed Choice Questions This is where you offer statements and the
respondent chooses from a list of responses to
show how much they agree or disagree with the statement. Written or online surveys often use these. Example:
specific user group. This is anecdotal, but taken
Question: “I feel confident that the local water
see the data to know how representative it is.
mind.” Answer one of: strongly agree, agree,
with quantitative insight helps persons who
authority has the community’s best interests in
Quantitative or qualitative, it is key to collect
data that will help you understand the status of your project or service area.
Informal Interviews, Surveys and Questionnaires This is the most popular form of data collection in evaluation. You create a list of questions or
inquiry areas to guide the conversation. You can
organise such interviews ahead of time, although
disagree, strongly disagree.
You can be creative in the way you ask
questions. Just make sure your method matches the language skills or culture of the group
you are surveying. Here are examples of less conversational answer options: •
sometimes opportunities may arise with little
notice. Either way, for informal conversations to be a valid way to collect data, you must inform
the person you speak with about your interest in the conversation and its purpose.
Lake Victoria Basin Secretariat
Use happy/sad faces to show a
continuum of satisfaction with aspects of a service:
•
Use a socio-gram in which participants physically place themselves in a room
according to how they want to answer the question (quantitative).
57
•
A variation on the above method places
very quickly as people exchange ideas. Make sure
example on a questionnaire as response
that is said.
the happy/sad faces from the first categories.
Response rates for written and telephone
questionnaires are usually not as high as with face-to-face methods.
Observation You ask no direct question here; just collect data or stories (e.g., checklists or frequency counts
done by project staff persons or volunteers, oral histories or anecdotal information). You can
observe things that are countable or you can observe patterns of behaviour.
the “note-taker” is fully capable of recording all You can collect very useful information by
inviting participants, before the programme begins and after the programme ends, to
demonstrate their experience with an issue
through a creative art. They may show the change they experienced through a video, diagram, song, or play.
Data Sampling It is not always possible to run a survey or a set of focus group discussions with a whole population. Where numbers of people involved are too large,
Often you will compare observations over time,
you may opt for a “sample”, a smaller selection
to similar activities or results from another
statistically.
or compare activities or results from your project
of individuals to represent that population
place. The term often used to describe this is
Collecting from a sample does not require as
“benchmarking”.
Documents/records Gather the information you need from
much staff or volunteer time, but can still yield
reliable information. The bigger your sample, the more reliable your information. For example:
existing documents such as participant diaries,
Three people’s responses may give you a skewed
meeting minutes and statistics from government
from 25 of them are likely to yield a more
organisational logs, test scores, literature searches, departments and reports.
Focus Groups Here, you bring together a group of participants for a discussion. You do not try to build a
consensus; you simply try to understand the
range and depth of opinion on a few selected
questions. In groups such as these, data can flow
impression of how 100 people feel; responses representative, more accurate picture.
Sampling can be used for most data collection. For example:
Instead of asking all stakeholders to comment on their area’s water resource management plan,
the evaluator asks a sample of 100 people from
across the watershed area. It may include urban
users, industrial users, agri-business owners and
58
Results-Based Management Tools & Techniques
farmers. The aim is to represent men and women
on its website at: http://ccnmtl.columbia.edu/
backgrounds and ethno-cultural groups.
sampling.html
of all ages and individuals from different
projects/qmss/samples_and_sampling/types_of_
There are many different sampling techniques.
Available at another website is an on-line
appropriate for your needs. For information
optimally-sized sample from identified
It is important to find the method that is most about sampling techniques and tools, visit the
Columbia Center for New Media Teaching and Learning (CCNMTL). This group carries easy-
to-access learning modules on sampling methods
calculator designed to help you draw an population groups:
http://www.surveysystem.com/sscalc.htm Figure 25 outlines some key performance
measurement activities that managers may implement, commission and/or supervise.
Use the PMF to Document Progress – Monitor and Report on Results or comprehensive reviews, all monitoring and DOCUMENT
DEVELOP
evaluations are systematic. One must plan how to collect data for each indicator, with most
methods using a “data sample,” a “baseline,” and IMPLEMENT
The Performance Measurement Framework (PMF) is the key tool for monitoring and reporting. It outlines the monitoring and
evaluation plan for a project: who is responsible, and how and when to measure performance.
In RBM, you must track results at all levels on an on-going basis. You must use a manageable
set of key performance indicators that compare
“information gathering”. Remember that data
collection is often long-term, and can be costly.
LVBC has limited resources, so look for the most
cost- effective, rapid assessment methodologies to collect data and consider the costs of monitoring when building project budgets.
Monitoring and reporting relies upon three key steps: •
Collecting data: After setting a baseline,
actual achievement to date against the baseline
collect data that shows how each
change early in the project’s development sets
measurement framework is changing.
and to the target. Identifying indicators of
indicator identified in the outcomes
the stage for usable project monitoring and/
Choose from a range of data collection
or evaluation. Whether they are small exercises Lake Victoria Basin Secretariat
59
methods such as those we have described in the past few pages. •
Analyzing: Compare pre- and post-data
to identify changes. Look for meaningful patterns. For instance, in a watershed
you might seek patterns in forest cover; or in a district it might be patterns in
•
Reporting: keep your audience’s
information needs firmly in mind as
you compile and present findings. It is
important to write clearly and concisely. Use your own voice. Make sure to show the depth of experience and knowledge gained through the project.
local media coverage of water supply
Projects are monitored internally as well as
revenue capture changes among users
and staff, you will conduct internal monitoring,
system, or community leaders’ pre- and
These are all part of performance measurement,
about water safety on Lake Victoria.
and data collection methods identified in the
services. Other examples might be how
“externally” by the PMET. As LVBC managers
of a community waste management
and commission surveys, studies and evaluations.
post-responses to interview questions
and all would draw on the results, targets, sources
You may review the data with a group of
PMF.
stakeholders and other staff to explore what the findings mean to them.
Performance Measurement Activities Activity
What it is
When
Baseline Surveys and Studies
Investigation of a problem or assessment of the conditions of a specified population group to define baseline indicators. Usually commissioned by project management to understand the starting point and use it to measure progress.
Start of project or annual plan.
Monitoring Systems
Usually developed at the project start, based on the PMF. Includes support to local information systems. The monitoring system involves collectin information at regular intervals and reporting on data related to project results fairly frequently so that management can make corrective actions. It supports project management, helping it to meet accountability requirements.
Ongoing throughout project or annual plan.
Evaluations
Evaluations attempt to determine objectively the worth or significance of a development activity, policy or programme. While project managers are responsible for implementing evaluation recommendations and learning from their findings, higher management or donors usually sponsor evaluations.
Usually mid-term and end-of-project or strategic plan.
Reviews
Reviews generally draw upon agency and partners’ monitoring systems, as well as the findings of survey, studies and evaluations.
As needed or desired.
Fig 25. Performance Measurement Activities 60
Results-Based Management Tools & Techniques
RBM Core Design Principles – Reporting Relevance – write to audience information needs. Results Focus – describe what is different more than what was done. Analysis – look back, think forward
The core performance measurement activities are summarized in Figure 25.
Establishing a baseline Once you decide what your data sample will be,
on needed actions. During these visits, try to
identify any capacity limitations, emerging risks,
lessons or good practices, and reflect on how well the results address gender equality and other
cross-cutting priorities. This information is vital
set a baseline. The baseline gives information
for your progress reports.
that you can compare later results against it to
Reporting
selection process that you will use to collect and
monitoring and evaluation plan, can ease the
about conditions before the project begins, so show change. The baseline sets up the sample
The RBMS PMF, which outlines your
compare data throughout the project. This will
way for you to write concise, compelling, results-
enable valid comparisons.
If it is not possible to set a baseline, a second-best option is to ask “before-after” questions when
you meet people who have participated in the
project. That way you can capture differences the project may have made.
Internal Monitoring
focused reports. Historically, project managers
have tended to write reports that described what was done – activities. Now, more donors would rather know how the work you have done has
changed things. The key RBM tool for reporting is set out in Appendix D.
All LVBC projects report regularly to funders
and stakeholders on progress, bringing lessons
For internal monitoring, meet regularly with
and good practices to their attention. New
agreed results. It is important to visit project sites
on results, using the PMF as a template. This
You might wish to do this jointly with the
should feed into preparing new annual work
your partners to assess progress towards the
reporting should include at least yearly updates
regularly to gauge achievements and constraints.
annual review of overall progress toward results
national focal point in order to build a consensus
plans, and in some cases policies.
Lake Victoria Basin Secretariat
61
In a results-based reporting system, the results
might be titled “Priorities for the Next Reporting
under which you can describe your progress.
to address failings or build on positive results.
statements listed in the PMF become headings Likewise, indicators will suggest what specific
pieces of information you will report under each result.
You may also want to include:
Period.” Use it to name actions you have planned In most reports, you will probably report against
all of the outputs and outcomes in your outcome measurement framework. As you write the
report, be guided by your indicators and what your data collection activities tell you about
•
What you learned
•
What you would do differently next time
activity you think about as you write the report,
•
Unexpected results
question: “So what?”
Each funding organisation has specific reporting requirements, but the table in Figure 26 shows
one common reporting format. Reporting hinges on expected results, set out in your PMF. The
Actual Results column lets you describe what is completed, and what is different (outputs and outcomes).
If, as in the following example, you are describing training progress, you would focus reporting on
trainees at the outcome level – what new things they know since being trained, or what they do differently since they’ve gained the knowledge. You can divide information between noting
progress over the current reporting period, and progress over the life of the project.
the project’s progress. For every completed
ask yourself or your colleagues the vital next
For example, you might say, “We designed and tested the training courses and materials – “So
what?” Your answer should lead you to the results of that activity (For this example, look at the first output in Figure 25).
And when you write a report, draw on the other reports prepared by Project Monitoring and
Evaluation Teams and by external evaluators. External monitors and evaluators are a key
support to project and service area managers. Think of them as an important source of
information that can help you manage and
stay focused on your targets. Monitoring and evaluation are related but slightly different
disciplines that have valuable information and
A report that spans project life describes progress
insights for improving project performance.
the Variance column to explain why a project is
What to expect from PMET
more broadly against outputs and outcomes. Use
progressing differently than planned. Finally, you may see a column that focuses on the future. It
•
Validation of conclusions, including
lessons and best practices, held in the narrative report that the project team prepares;
62
Results-Based Management Tools & Techniques
A SAMPLE REPORT Expected Results
Actual in this Reporting Period
Project to Date (Cumulative)
Variance
Priorities for Next Reporting Period
Activities
e.g. 70% of activities scheduled for this reporting period were completed as planned.
e.g. 60% of the project schedule is elapsed and we have completed 58% of planned activities on time.
e.g. 30% of activities planned for this quarter could not be completed due to flooding in area X.
e.g. With floods receding we plan to catch up on the activities delayed in the last period.
Outputs
e.g. 35 training programmes on HIV / AIDS for high risk populations completed, reaching 875 (525 men, 350 women) target beneficiaries in the Lake Victoria Basin.
e.g. Curriculum developed and tested in the first period, and training rolled out to target communities in each period thereafter, to date 185 training programmes have reached 4,625 (2,775 men, 1,850 women).
e.g. 15 training programmes planned for 375 persons (60% men, 40% women) were delayed by flooding.
e.g. 50 new training programmes will be delivered to 1,250 (750 men, 500 women) beneficiaries in area Y. In addition the 15 programmes for 375 (225 men, 150 women) beneficiarries in area Y will be completed.
Outcomes
e.g. Training evaluation reports show that 80% of trainees are aware of using condoms to avoid infection.
e.g. Toral for all training evaluation reports show that 76% of trainees are aware of condom use, alumni follow up studies show 48% are using them.
e.g. 2% below the planned 50% condom use for trainees: mid-term evaluation shows cost and availability of condoms is a constraint.
e.g. We plan to link trainees with NGOs working in the area for a source of cheap condoms.
Impacts
Not usually reporting on since they are post project. But you can note important progress toward achieving impacts. e.g. “Mid-term evaluation showed that use of condoms in target population increased by 20% and rates of new infections are down by 0.7%�.
Fig 26. A Sample Report Lake Victoria Basin Secretariat
63
•
Regular assessments of progress towards
•
Assurance that programme
the results;
•
Results and lessons that will be useful to those in a position to plan and
implement activities in the next cycle
implementation respects priorities and
•
•
Continued identification of partners’
The key RBM tools discussed in sections III
capacity development needs;
and IV will help you to define your projects,
•
Continued identification of changes to
some additional tools to help you implement
principles;
risk environment and mitigation plans;
•
Improved results-based reporting on
•
Better teamwork and project ownership
Improvements in effectiveness, efficiency, relevance, and sustainability.
programmes and initiatives. Section V provides them and keep them on track.
achievements; and
by implementing partners.
What to expect from evaluations: •
A careful judgement of the worth of
results and strategies; and of alternatives that could have used resources or addressed risks more effectively;
64
Results-Based Management Tools & Techniques
V. RESULTS-BASED PROJECT IMPLEMENTATION To manage a project that delivers the desired DOCUMENT
results, you need a detailed, accurate plan. It has DEVELOP
to tell you who is responsible for what, what will happen, when it will happen, and how much it
will cost. An implementation plan captures all of IMPLEMENT
these details. It makes the links between planned activities and the expected results of your project/ programme. The tools introduced here help you
This section introduces five additional tools to
help you implement complex projects. As such,
to ensure that the plan is realistic, affordable, and focuses on achieving its objectives4.
the tools fill in the middle between “develop” and “document”. They help managers make
operational decisions with results in mind, and they complement the RBM tools discussed in Sections III and IV.
Lake Victoria Basin Secretariat
4
These plans go by various names: Plan of Operations, Management Plan, Project Implementation Plan, etc. The name of the plan is less important than clearly documenting the plan. 65
The Organisation Chart visually depicts your
activities in a results-based framework. You
Output/Activity Matrix is a key tool that helps
Breakdown Structure (WBS), the schedule and
approach to project management. The Project you organise details of project/programme
can then use the matrix to generate the Work the budget.
The Organisation Chart The organisation chart is a tool that shows all
The organisation chart usually places project/
a project and how they relate to each other. It
in the middle, and implementation at the base.
the partners who govern, manage and implement states who is accountable to whom, who reports to whom, and how project communications
flow. To develop the chart you must identify the partners, committees and teams that operate at
each level, and make clear all of their basic lines
of accountability, reporting and communication.
Cooperation projects are usually partnerships between government or multi-government
organisations. The donor is often an official
development assistance organisation which is
part of the government of a developed country. That assistance organisation usually partners
The Organization Chart
Donor(s) e.g. NORAD for Norway
LVBC Sec (for EAC)
Sectoral Council of Ministers
Project Steering Committee National Focal Points / Ministries
Project Management Unit Project Management Committee
Legend Communication Ministries and Agencies
Technical Advisors/Service Providers
CBOs
programme governance at the top, management
Reporting
District / Municipal Agencies
CBOs
Participation
CBOs
CBOs Fig 27. The Organization Chart
66
Results-Based Management Tools & Techniques
with a recipient organisation like LVBC, which
Who will help develop the capacity
organisation chart in Figure 27.
decision-making?
is an organ of the EAC. We show a sample Accompany the organisation chart with a
short description of the project management
of local partners for management and •
approach. This description should cover the
mandates of each of these committees
the partners will work together to get results.
and groups in managing for results?
This complements the organisation chart and
Who will participate as members of
describes the reasons for your management
these committees and groups? What
approach. If appropriate, you should state how
steps will be taken to ensure equitable
you plan to strengthen local management
participation by women and men in
capacity through participation, support, and other Ask and answer these questions as you put together an organisation chart: •
What is the basic approach for
managing the project/programme? How does the approach reflect the
way partners expect to participate and make decisions? How does it add to
accountability for results? How will it
strengthen the RBM capacity of local partners? How will it help to manage
risks? How will it promote progress on
gender equality and other cross cutting objectives? •
Who will participate in managing for results? Who will govern the project?
be created to support participatory decision-making? What are the
vision and values of the partnership, and the way
capacity-building acts.
What committees and groups must
decision-making? •
What are the basic processes among the partners? What are the lines
of communication and reporting?
What processes will partners follow
to ensure effective links between the
levels of governance, management and implementation?
Roles and Responsibilities The organisation chart shows that various
partners will play leadership, management,
technical, operational and learning roles in the project/programme. In these roles, they are
responsible for any number of functions, activities and results. Every partner and committee the
organisation chart identifies has a corresponding list of responsibilities.
Who will manage the strategy? Who
will manage quality? Who will manage people, finances and operations? Who
will manage and implement activities?
Lake Victoria Basin Secretariat
67
It is useful to list the responsibilities of all
partners expect to play? What roles
throughout the entire project/programme, which
capacity? What roles will they have to
partners and state what responsibilities endure
are they capable of playing with their
occur in the annual cycle, and which are tied to
play to sustain the results? What capacity
specific components.
Clearly-defined roles and responsibilities help create a working environment where partners
building is needed? •
can comfortably commit to achieving results.
planning, implementing, monitoring
and correspond to their capacities. Descriptions
and reporting on results? What other
should also show accountability. Accountability
responsibilities are evident in the annual
agreements work best when they name
cycle, the project components, and the
individuals, give support, have clear indicators,
activities? What responsibilities do
and all concerned have agreed on the positive and As local partners develop technical and
management capacities during implementation, it is possible (and desirable) that their roles will evolve. As they take on responsibilities, the role
of the PMU may diminish. This transfer, which might be central to sustaining expected results, should be planned for.
Questions to consider in developing the
description of roles and responsibilities include: •
68
partners expect? •
What accountability is involved in
taking on these responsibilities? What agreements are in place to identify
accountabilities? What support will there be? How will you measure performance? What are the consequences of not performing?
Programme/Project Committees The sample Organisation Chart includes two committees. Project/programme committees
What basic roles must be performed
are a means to ensure that partners participate
results? What are the needs in
meaningful roles in planning, implementation,
technical assistance and learning and
improve their management abilities.
for the project/programme to achieve
in decision-making. They help partners play
leadership, management, operations,
and monitoring. They also allow local partners to
development? How are these roles linked
Different types of committees meet the needs
to achieving results? •
involved in playing these roles? What are the responsibilities involved in
Roles should align with partners’ expectations
negative consequences of performance.
What are the basic responsibilities
What are the capacities of partners
of different projects/programmes. The most common are:
to play these roles? What roles do
Results-Based Management Tools & Techniques
•
Steering committees for policy direction and general project or programme oversight.
•
will it foster shared accountability for
Management committees for
results among members? •
implementation and resource
What information will it receive? How often will the committee meet? What
Advisory committees to provide
financial resources will the committee be
technical advice and options.
assigned? How will it work to produce
Many projects/programmes will also establish
management or implementation teams to work in particular components. These teams often
the committee responsible? How
will the committee make decisions?
management. •
For what decision-making areas is
expected results and deliverables? •
Who will be the members of the
report to management committees.
committee? Which organisations should
When deciding on what committees to include
such as other donors, should be invited
in a programme/project, consider these questions: •
What is the committee’s purpose? What is its mandate? What results
are expected? What deliverables is the committee expected to produce? How
be represented? What organisations, as observers? What capacities are
required to participate effectively in the
committee? How will these capacities be
strengthened to ensure full participation? What steps should be taken to ensure
that the committee addresses the needs of women and men?
Work Breakdown Structure The Work Breakdown Structure (WBS) is a
the same information on outcomes and outputs
simple, effective way to picture links between the
model you must ensure that your WBS mirrors
snapshot of the plan you want to execute. It is a
activities you do and the outcomes you expect. It
summarises links between outcomes, outputs and activities, so it builds on the LM, but provides
more detail on the activities and sub-activities. The WBS organises all programming and
management activities into basic work packages,
in your logic model. So, if you update the logic these updates also.
Many implementing partners begin planning with the WBS. From there they develop the output/activity matrix, schedule and budget
(discussed below). Others start with an outline
called WBS categories. It is a short digest of Lake Victoria Basin Secretariat
69
Work Breakdown Structure Impact
Outcomes
Outcomes
Outcomes
WBS 100
WBS 200
WBS 300
WBS 400
WBS 500
(Output)
(Output)
(Output)
(Output)
(Output)
110 - activity package
210 - activity package
310 - activity package
410 - activity package
510 - activity package
120 - activity package
220 - activity package
320 - activity package
420 - activity package
520 - activity package
130 - activity package
230 - activity package
330 - activity package
430 - activity package
530 - activity package
140 - activity package
240 - activity package
340 - activity package
440 - activity package
540 - activity package
Fig 28. Work Breakdown Structure
of the WBS, then from that develop a detailed
and the WBS may not align. For this reason, we
detailed WBS, schedule and budget.
and the Project Output/Activity Matrix, moving
output/activity matrix to underpin a more
The WBS will also outline management and
administrative activities. In Figure 28, the WBS categories for project management can refer
to the organisations that manage the project/
programme, while the sub-categories describe their respective management activities and
tasks. This simplifies budgeting and financial procedures.
Some project/programmes will organise their WBS activities by other categories such as
components or themes. This way, the components may represent logical subdivisions of the project/ programme, but it also means the Logic Model 70
recommend aligning the WBS, the Logic Model from a very high level of activity groups in the
Logic Model broken down into activity packages
in the WBS, and in complex project into detailed descriptions in the Project Output/Activity Matrix.
The WBS should look something like Figure 28. Note how the numbering scheme allows easy
cross referencing to other key project documents. Outputs are numbered in a 100 series (i.e. WBS 100, WBS 200, etc.). Activities are linked to the outputs in a 10 series (i.e. WBS 110, WBS 120, etc. which would be the lowest level reflected in the Logic Model), and the tasks linked to the
Results-Based Management Tools & Techniques
activities in a 1 series (i.e. WBS 111, WBS 112,
Output/Activity Matrix? Is it easy to
etc.). Keeping the Logic Model and WBS at a
see the expected results and reach of the
higher level allows them to be captured on a few
WBS sub-categories?
pages and gives readers a chance to understand
•
the architecture of the project.
For presentation, some groups will write a
produce the project/programme
outputs? Do the work packages and
short narrative on the main work packages to
activities shown in the WBS make
complement the WBS. In this case, they might
sense? Does it require a budget? Are
choose to present the WBS in an annex.
these roughly in proportion to other
work packages? Is each activity or step
Key questions to ask are: •
What activities will you apply to
distinct? Does each activity advance the
Does the WBS present the organisation
project/programme toward a successful
accurately? Does it highlight the logical
the gender equality strategy? Should you
Can you link the structure to the Project
plan?
of project/programme activities
finish? Do the WBS and activities reflect
sub-divisions of the project/programme?
add, change or remove activities from the
Activity Package Example Outcome Output/Activity
Inputs Specific Actions
Responsibility
Required Resources
Outcome 1: Degredation and depletion of Basin environment is reversed, natural resources are used and managed sustainably. Output 100: EAC Partner States use a harmonized approach to implement sustainable management, development and conservation of the environment. Activity 110
Task 111
Facilitate the development of, and promote the adoption of, harmonized laws, standards, policies for Transboundary Natural Resources (TBNR) management and bio-diversity of land, water, forests, fisheries, wildlife and minerals in EAC.
Develop EAC standards for management of land and water resources. - Draft Terms of Reference (TORs) for comparative study of land and water resources management.
3 days for LVBC specialists to prepare TORs 1 day for LVBC procurement specialists
Environmental Stewardship and Natural Resources Management Programme
- Circulate TORs for review and comments. - Finalize TORs. - Etc (tasks are broken down as far as needed to allow managers to assign resources and responsibilities) Fig 29. Activity Package Example
Lake Victoria Basin Secretariat
71
Outputs and Activities
project/programme budget. Inputs include
Some organizations like to create very details
people, materials, equipment, travel, conference
level so that managers can see the big picture
The matrix also supports writing work plans and
you take this approach, then the Project Output/
plans on the matrix, and update the matrix at the
WBS. Our preference is to keep the WBS higher
rooms, translators, per diem, and other needs.
and understand how the pieces fit together. If
progress reports. You can base your annual work
Activity Matrix is a useful planning tool that
output level in all progress reports.
allows you to detail your implementation
strategy. This detailed matrix helps to ensure that you identify all the work packages and activities
that you need to turn inputs into results. It draws links between outcomes, outputs, activities,
You may find that your project/programme is
too large to capture in the matrix. If this is the
case, you must present your lists of activities in an appendix.
resources and responsibilities. The matrix shows
If you have a preliminary WBS, you have already
with the results described in your logic model.
programme. Developing the Project Output/
how the work outlined in the WBS connects You use the matrix to: divide the project/
programme into programming and management work packages; divide work packages into
activities and tasks, and assign resources and
responsibility to each activity and task. Preparing
outlined the work packages in your project/
Activity Matrix is a way to review the outline and delve into the details.
The key questions to ask are: •
the matrix reduces the chance you might
accurately? Can you link the structure
project/programme.
to the Project Output/Activity Matrix to
Figure 29 is an example showing how the activity
the WBS? Can you link it to the project/
packages are broken down into discrete tasks:
for outputs, activities and tasks you describe in
programme budget? •
the matrix. That way, you can cross reference
activities you need to gain each output (and the tasks involved in each activity). It also shows inputs for each task, to help you prepare the
72
What main tasks make up these
activities? Is each task discrete? Is each needed? Should any tasks be added,
easily (see the WBS).
The matrix has uses beyond identifying the
matrix present the organisation
of project/programme activities
overlook any step essential to completing the
It is best to use a WBS-type numbering system
Does the Project Output/Activity
changed or removed? •
What inputs are needed for each activity and task? What human
resources are needed to deliver the
activities? What materials, equipment,
Results-Based Management Tools & Techniques
travel, conference rooms, translators, per
partners are responsible for completing
diem, etc., are needed? •
each programming activity and task?
Which partners will participate in each
Which institutional partners are
programming activity and task?
producing which outputs? Which
Schedule For a project manager, the schedule is a key tool.
and sequence of activities. You can also highlight
and the order they should occur. The schedule
would look something like Figure 30.
You need to know when activities should occur
the milestones along the way. A Gantt Chart
sets the duration and order of activities, and key
Preparing a schedule on a Gantt chart is quite
milestones in implementation.
straightforward. Work from the activities and
The Gantt chart is a popular tool for picturing
tasks outlined in the Project Output/Activity
the time dimension of a project/programme.
Matrix to identify: (1) how long activities will
With it, you can describe the scheduling, length
last; (2) the order of the activities, and (3) the
Project Implementation Schedule Fiscal Year 1 Q1
Q2
Q3
Fiscal Year 2 Q4
Q1
Q2
Q3
Fiscal Year 3 Q4
Q1
Q2
Q3
Q4
WBS 100 110 Activity Package 120 Activity Package 130 Activity Package 140 Activity Package WBS 200 210 Activity Package 220 Activity Package 230 Activity Package WBS 300 310 Activity Package 320 Activity Package 330 Activity Package 340 Activity Package Fig 30. Project Implementation Schedule Lake Victoria Basin Secretariat
73
earliest time they can begin. (Some activities
depend on a prior activity, while others can be
•
done independent of sequence.) When you
Will you indicate the schedule for
you see the minimum time needed to complete
reaching milestones? Will you include
the project/programme. As you add in workload, usable schedule begins to emerge.
Scheduling becomes more complex when you
recognise that some activities occur once, some repeat periodically, and some are ongoing.
management activities?
Once you make these basic choices, use
these questions to guide how you prepare the horizontal bar chart: •
For example, in the management activities,
is your list of activities to include in the
distinct cycles – the programme cycle, the annual
occur only once and those that repeat in cycles.
Gantt chart? •
time do you actually estimate for each
appropriate, you should show the schedule for
activity? •
The updated WBS and Project Output/Activity
what you want the chart to show. •
each activity? Which activities flow
completed first? Which can happen at
The Gantt chart should correspond to these tools. format, you must make basic decisions about
What is the earliest time you can begin in a sequence and so need others to be
Matrix will provide an updated list of activities.
Before you begin to develop a schedule in Gantt
What is the minimum time needed to
maximum time? Within this range, what
each project/programme activity and output. If
a very good tool for these needs.
What is the duration of the activities? complete each activity? What is the
The schedule should show the duration of
reaching other key milestones. The Gantt chart is
project/programme? Which activities
activities will occur more than once, what
prepare work plans every year. LVBC has three
schedule, you need to identify activities that
What are the main cycles in the
are included in each cycle? Since some
you prepare a project plan only once, but you
cycle, and the project cycle. To prepare a usable
Will you schedule all activities and
tasks, or higher level activities only?
display the activities in a horizontal bar chart,
holidays, slack time, and partners’ schedules, a
Decide the level of detail to show.
the same time? •
What special factors should you
consider in preparing the schedule?
Are there holidays when you will not
Decide if the chart will show the
work? What realities exist in the partners’
of interventions or components that are
programme schedule? What “slack” do
schedule for achieving outputs, or a series
schedules that will affect the project/
more clear to you and your partners. 74
Results-Based Management Tools & Techniques
you need to add, to make up time if you fall behind schedule? •
When will the activities take place?
schedule? What is the timing of activities and milestones in terms of fiscal years and quarters?
What milestones will you achieve in the
Budget Budgets are powerful tools to use to manage for
resources, levels of effort and scheduling. As
communication inside a project/programme.
clearly the consequences of non-compliance.
results. They empower managers and support
They also are a concrete expression of your formal agreement as partners. The budget shows cost
breakdowns by output and input, and the cash flow estimates for implementing a schedule of activities.
LVBC will need to negotiate budgets to carry out its assigned work with its implementing partners. This negotiation should consider a full range of
management and implementation details. These should include responsibilities, expectations,
accountabilities, levels of effort and the material support each particular partner needs to reach
expected results. It should also consider the cash and in-kind contributions that partners will make.
In the project design process, budget negotiation is time-consuming, and you should approach it
in a business-like way. Come prepared with all of
the information you need to justify all allocations. This will include costs of staff, overhead and inputs. Formalise in writing all agreements among partners regarding responsibilities,
Lake Victoria Basin Secretariat
importantly, make sure that agreements state Donors may want budgets that break down costs
either by output or by input. The former links the budget to the Project Output/Activity Matrix,
and describes the costs of performing tasks. The
latter states the costs by input categories, such as
salaries, travel or equipment. Whichever method
you choose, for management purposes you should prepare cash-flow estimates quarterly or semi-
annually for the first year, and by year for the rest of the programme or project.
If your project/programme has a major
component that calls for the purchase of capital equipment or goods, you should also prepare a Procurement Plan. The Procurement Plan will
describe the goods you will procure, how you will
identify and contract suppliers, how goods will be insured, and who will own them at the project’s
end – or more formally, how their ownership will be vested. The Procurement Plan ensures that this process is transparent.
For many projects, you will need to monitor cash
and in-kind contributions that your partners give to the project/programme. In some cases, donors
75
look for a contribution that shows partners have • committed to maintaining the results. They state
What cost information do you need to
find out? What are the specifications
the planned contributions in their agreements. In
for the activity inputs? What travel,
such cases, LVBC must track their contributions
accommodations, equipment, salaries
to present this information in financial reports. Basic questions for developing the project/ programme budget are: •
What are the fixed costs? What costs for staff, rent and other overhead will
you assign to the project/programme?
Many donors are very concerned about overheads so try to capture and bill
and fees will be linked with each planned activity? •
What will these inputs cost? What are the prices of the inputs that you list?
Will these input costs stay stable over the life of your project/programme?
Will you need to factor inflation into the budget?
directly for as many costs as possible –
e.g. by using time sheets to allocate the time that finance and administration
staff spend on your project rather than lumping them into overhead. •
What activities will you cost? Which
activities or tasks of those listed in the
Project Output/Activity Matrix will you actually cost?
76
Results-Based Management Tools & Techniques
Some Additional Resources As LVBC moves to implement its RBMS,
the related learning process will need to be
continuous. Additional sources of information could include, but are not restricted to:
http://www.un.org/esa/coordination/pdf/
rbm_report_10_july.pdf http://search.pmi. org/?q=RBM
http://www.prince2.com http://bigpushforward.net/resources http://mande.co.uk http://www.rbmtraining.com http://www.mango.org.uk/About http://www.tbs-sct.gc.ca/ http://www.oecd.org/dac/evaluation/ http:// www.bond.org.uk/pages/value-for-moneyresources.html http://www.mfdr.org
http://www.outcomemapping.ca http://www. adb.org/
Lake Victoria Basin Secretariat
77
78
Results-Based Management Tools & Techniques
APPENDICES
Lake Victoria Basin Secretariat
79
80
Results-Based Management Tools & Techniques
Donor Partners
Implementing Partner(s) Outputs
Budget
Project Timeframe
Activities
Goal / Objective
Project Name
Appendix A: Logic Model Template Appendix A: Logic Model Template
Outcomes
Impact
82
Results-Based Management Tools & Techniques
Output
Outcome
Impact
Baseline
Target
Donor Partners
Implementing Partner(s)
Indicators
Budget
Project Timeframe
Results
Goal / Objective
Project Name
Appendix B: Performance Measurement Framework Template Appendix B: Performance Measurement Framework Template
Data Source/Method
Frequency & Responsibility
84
Results-Based Management Tools & Techniques
Developmental Risks
Financial Risks
Operational Risks
Effect on Planned Results (low, medium, high)
Donor Partners
Implementing Partner(s) Likelihood of Occurrence (low, medium, high)
Budget
Project Timeframe
Description of Risk (ranked in order of priority)
Goal / Objective
Project Name
Appendix C: Risk Analysis & Management Template Appendix C: Risk Analysis and Management Template
Risk Mitigation Strategy
Person(s) Responsible to Manage Risk
86
Results-Based Management Tools & Techniques
Impact
Outcomes
Outputs
Activities
Inputs
... describes a scenario that is close enough to show a real contribution to the desired change, yet big enough and far enough out on the horizon to be visionary
... is the picture of a preferred future; the reason why it is important to do the initiative
... are fewer in number; each flows naturally from several outputs
... describe key changes you want to make through the initiative
... are more than one step removed from Activities
... create the potential for Outcomes to occur
... are the most direct and immediate results. Each Output relates to one Activity
... are to be summarized in 5 - 8 statements
... describe the essential work of the initiative – how the Inputs are to be combined
... give you essential information for budgeting
... are the human and physical resources that make the initiative possible – people, equipment, supplies, and other ingredients
Defining Characteristic
... may expand into yet wider settings to include societies, districts, communities, professions, whole organizations
... may also include those people or workplaces who use or are influenced by products created in the initiative
... expand into wider settings – includes those who are directly involved in Activities, but goes beyond such direct participants to include families, organisations, or neighbourhoods around them
... may simply refer to a product (i.e. a functioning well), though often refers to an immediate benefit among those people directly participating in an activity (i.e. trainees)
... are written from the outlook of those instigating the work (i.e. managers, staff facilitators, volunteers)
Who is Affected
Appendix D: Project Results Logic at Glance Appendix D: Project Results Logic at a Glance
... are observable well after the initative is complete
... are observable at, or shortly after, completing the initiative
... can be observed as you complete the activity
When Observable
... allow you only indrect influence – the initiative will not achieve the vision by itself, only contribute toward it
caution: you should be reasonably confident in your claims here, since you ay be responsible for bringing them about
... allow you direct influence – you must keep your sights set on the Outcomes; then manage activities for the best chance of success – learn by doing, adapt as you go
... allow you substantial control – not total though; there are often surprises during implementation
Degree of Management Control
... are like: THE OUTER RIPPLES
... are like: THE MOST IMMEDIATE RIPPLES
... are like: CREATING THE SPLASH
... are like: DROPPING THE ROCK
... are like: THE ROCK
Splash & Ripple Analogy