Results-Based Management (RBM) training 9-11 April 2014 Anders Pettersson anders.pettersson@swedevelop.com
Training rules Everyone can participate Value each other’s ideas No mobile devices Respect each other by being on time Questions any time (answers may come later) Other?
Presentation • Your name • One expectation • One fear • A secret about yourself
Purpose and content • To familiarise participants with the concepts and ideas behind the RBM-methodology – – – – – – –
Results chain Problem analysis Setting goals (RAF) Indicators and baseline Risk analysis Budgeting Monitoring and reporting results
What is RBM? Background: Aid Effectiveness Agenda RBM is a shift in focus from activity-based to resultsbased. - What do we want to change? - How to we plan to do it? - Are we making progress? RBM is a concept just as much as a number of tools!
Meier (2003): “Results-Based Management (RBM) is a management strategy aimed at achieving important changes in the way organisations operate, with improving performance in terms of results as the central orientation. RBM provides the management framework with tools for strategic planning, risk management, performance monitoring and evaluation. It’s primary purpose is to improve efficiency and effectiveness through organisational learning, and secondly to fulfil accountability obligations through performance reporting.“
Results chain – one result will lead to another! Increased usage of clean water in the village Increased access to clean water in the village
Improved health in the village
Intermediate outcome
Ultimate outcome Impact
Immediate outcome
A well with hand pump Output
Input Activities
Construction of a well using money, people and material
Tiger prawns are produced through cutting down mangroves and overfishing, which damages environment. Your job is to design a project /campaign within Ukraine to address this problem.
Problem analysis • To achieve results, you must know the problem! • Problem analysis is a crucial tool to design a good project/programme and part of the RBM approach! – A well designed project/programme is more likely to achive results!
Simplified model People get sick
No clean water available
No wells in the village
Effects
Core problem
Causes
Task • Work in groups and design a ”Problem tree” based on the information in the summary of the Ukrainian energy policy problems.
Results chain
Input
• Controlled by you •Internal risks •Few assumptions
Output
Immidiate outcome
• Controlled by you • Internal risks •Few assumptions
• Likely to happen if outputs are produced • External risks •Basic assumptions
Intermediate outcome • You are making a valuable contribution • External risks •Assumptions required
Ultimate outcome/Impact • You are hopefully making a contribution •External risks •Other factors involved
Achieving institutional change Changes in behavior of people
Changes in behavior of institutions
Changes in behavior of system
RAF/LFA/RBM-matrix • ”A dear child has many names” • Tool for planning and follow-up of a project. • Important to understand principles rather than details!
Link between problem tree and LFA/RBM Effects
Overall objective or ultimate outcome and intermediate outcome
Problem
Project purpose or Immediate outcome
Causes
Expected outputs or Outputs Activities or Inputs
Link example
Effects
Problem
Causes
People get sick
Reduced cases of diarrhoea by 80 %
No clean water available
Water contain 50 % less bacteria
No wells in the village
A well with a hand pump in the village
Overall objective or ultimate outcome and intermediate outcome Project purpose or Immediate outcome
Expected outputs or Outputs
Group 1
Effects
Problem
Causes
Overspending energy of government and business
Bad state regulation in energy sphere
Gov. does not have strategy in energy sector
Effective system in energy sector
1. Int. standards ratified by gov. 2. Ukr. Join at least 2 networks 3. Adoption new legislation 1. A new strategy developed. 2. New legislation developed 3. Int. standards are advocated to join int. networks
Overall objective or ultimate outcome and intermediate outcome Project purpose or Immediate outcome
Expected outputs or Outputs
Group 2
Effects
Problem
1. Reduced energy dependency 2. Reduced corruption in energy sector
Energy dependency
Corruption in energy sector
1. New energy policy adopted which includes recommendations. 2. Implementation of new policy 1.
Causes
Bad legislation Lack of public control
2.
3.
Developed PP “Reducing depency in energy sector” Developed PP “How to improve investment climate and anticorrputon” Conducted lobbying campaign
Overall objective or ultimate outcome and intermediate outcome Project purpose or Immediate outcome
Expected outputs or Outputs
Group 3 Poor environment
Effects
Problem
Causes
Contribute to reducing pollution Promote use of green tech. by business Decrease of energy consumption
High energy intensity causes environmental damage No use of nature friendly technolody
Overall objective or ultimate outcome and intermediate outcome
1. Public pressure on business and politicians to cut energy Nobody cares about consumption energy use/consumption 1. Research and information
Lack of knowledge Lack of awareness Not much NGO action
2. 3. 4. 5.
materials produced on level of energy. Built advocacy coalition of NGOs Awareness and advocacy campaign conducted on‌. Draft of new legialstion Annual medals awarded to business (best and worst)
Project purpose or Immediate outcome
Expected outputs or Outputs
Common mistakes • No established results chain • Results not specific and not possible to measure • Attribution problem • To ambitious and lack of outcomes • Only established outputs and activities
Are your goals SMART? S – Specific M – Measurable A – Achievable R – Relevant T – Time-bound
Task • Work in the same groups and scrutinize the results chain. What is good and what is not good?
Link between problem tree and LFA/RBM Effects
Problem
Causes
Overall objective or ultimate outcome and intermediate outcome
Project purpose or Immediate outcome
Expected outputs or Outputs Activities or Inputs
Task • Work in the same groups and develop – Overall objective (Impact) – Project purpose (Outcomes) – Expected outputs (Outputs) You should base it on your problem tree. (Adjust if needed!) If in doubt, be SMART! 1) Ukrainian think tank 2) Ukrainian environmental rights NGO 3) Bi-lateral donor
Round up day 1 • Each participant has maximum 15 seconds: – “Today, the most valuable learning for me was….” – “Anders, tomorrow you must really improve by…” – “I’m a bit confused, I hope XXX is clarified better tomorrow” – “I’m expecting more….” – “Before I go to bed, I’ll be thinking about….”
Key points day 1 • • • •
Shift in focus from activities to results Results chain – outcome results Define your problem Link results to your problem – be specific
Agenda day 2 • Indicators • Risks • Budget (day 3 Reporting and Monitoring)
Example of framework Intervention logic
Overall objective (Impact)
Project purpose (Outcome)
Expected outputs (Outputs)
Target group
Indicators
Baseline values
Risks
Risk mitigation
Target groups • • • • •
Often omitted/forgotten not specific Should be as specific as possible Primary target group Secondary target group (Ultimate target group)
Task • Identify target groups for your established goals. Do you need to re-phrase them?
Setting indicators • • • • • •
Few and carefully selected Not contain targets Objectively verifiable (preferably) Not necessary require a numerical (Linked to a source of verification – method) Would normally require a starting point/baseline to measure against
Types of indicators Intermediate outcome
Ultimate outcome Impact
Immediate outcome Output
Input Activities
Measures numbers (trainings, participants, activities etc.)
Measure changes in knowledge and behaviour
Measure changes in status
Common mistakes when setting indicators • • • • • •
To many Not relevant to the goals Contain targets Not possible to measure Not possible to establish baseline Not objective
Task • Work in the same groups and scrutinize the indicators. What is good and what is not good?
Task • Work in the same groups and develop indicators with sources of verification to your Intervention logic in your LFA. Test questions: 1) Does it measure your goal/objective? 2) Is it possible to measure? 3) Can a baseline be established?
What is Risk Management? • Important component in RBM • To identify and manage potential problems that could undermine your organisation or your projects • To integrate the risks into planning, implementation and learning processes.
Why is Risk Management important? • All projects will encouter problems and have risks! – Be open with them! • Make decisions on how to move forward • Avoid failiures • Improve results • Handle changes in your environment
Results chain
Input
• Controlled by you •Internal risks •Few assumptions
Output
Immidiate outcome
• Controlled by you • Internal risks •Few assumptions
• Likely to happen if outputs are produced • External risks •Basic assumptions
Intermediate outcome • You are making a valuable contribution • External risks •Assumptions required
Ultimate outcome/Impact • You are hopefully making a contribution •External risks •Other factors involved
Risk mitigation strategies • Two general strategies a) Reduce probability b) Reduce impact – – – –
•
Preventive Corrective Directive Detective
Be concrete! – What can you do!?
Risk mitigation strategies - example Risk: Corruption within the organization (internal strategic risk)
• Preventive control Develop Anti-corruption policy that all employees will have to sign. Payments above 10 000 UAH must be signed by Executive Director AND Board Chairman. • Corrective control Immediately contact the police. Suspend person from handling money. • Directive controls Have guidelines in place that describes which procedures that should be taken by whom in case of suspicion of corruption. • Detective control Carry out an external financial audit every year.
Task • Identify risks in your project – internal or external risks? • Identify mitigation strategies – do they reduce probability or impact?
“There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – there are things we do not know we don't know.” Donald Rumsfeld
Rumsfeld Matrix
Pettersson Matrix Side effects
Positive
Include in project Known planning/results chain
Unknown
Document success stories
Negative
Include in project planning or risk mitigation actions
Document and/or adapt project
Budgeting • RBM is a shift in focus from activity-based to results-based.... • ...budgeting should then follow the same logic!
Demands on a budget • Support achievement of results • Allow for follow-up on how funds have been spent • Disallow misuse of funds (level of detail?)
Task • Study the two selected budgets – Are they supporting: – Support achievement of results – Allow for follow-up on how funds have been spent – Disallow misuse of funds
Highlight good and bad things!
Task • Design a budget structure for the project on Water and Sanitation – Make sure it: – Support achievement of results – Allow for follow-up on how funds have been spent – Disallow misuse of funds
Round up day 2 • Each participant has maximum 15 seconds: – “Today, the most valuable learning for me was….” – “Anders, tomorrow you must really improve by…” – “I’m a bit confused, I hope XXX is clarified better tomorrow” – “I’m expecting more….” – “Before I go to bed, I’ll be thinking about….”
Key points day 2 • Linking indicators of success with results – Neutral (no targets) – Be possible to measure
• Linking internal and external risks to results • Risk mitigating actions to reduce impact or probability • Budget that should support achievement of results
Baseline values • Always required for an indicator – Where do you start? • Can often be 0 or “does not exist”. • Should be established in planning process or beginning of a project.
Creating a monitoring framework • Monitoring of results is a key component of RBM • Follow-ups on the developments of the indicators • Sets frequency, methods and responsibilities • Meta-plan – monitor and act on the plan…
Task • Create a monitoring framework for a selected project. • Framework will be given to another group for improvement. • Framework will be sent to a thrid group to ask questions.
Reporting • RBM is a shift in focus from activity-based to resultsbased.... • ...reporting should then follow the same logic! – Information on progress on the goals/objectivs/results achieved using indicators – Analyse intended and unintedend results – Monitoring spending of resources – Projection if goals/objectivs/results will be met or not – Follow-up on risks and assumptions – Learning lessons and integrating them into decisions
Task • Discuss in your group for possible headings for a Progress Report to donor for a project.
Link between projects and program Program
Project
Overall objective or ultimate outcome and intermediate outcome
Overall objective or ultimate outcome and intermediate outcome
Program purpose or Immediate outcome
Project purpose or Immediate outcome
Expected outputs or Outputs
Expected outputs or Outputs Activities or Inputs
Link example Program
Project
Overall objective or ultimate outcome and intermediate outcome
NGOs influence government decisions
Examples of change
Program purpose or Immediate outcome
Increased legitimacy and effectiveness of NGOs in the country
NGO identifies area of influence and implements activities
Project purpose or Immediate outcome
Expected outputs or Outputs
Support to organizational development of NGOs
Development of strategic plan
Expected outputs or Outputs
Overall objective or ultimate outcome and intermediate outcome
Discuss in groups • In what way is this model and theory realistic? • How can IRF benefit from this model for its programs? • What needs to be done in that case?
• Reflections? – Was expectations met? • Fill in evaluation! • ... RBM guide will come… • THANK YOU!