Brief Guide to Results-Based Management (RBM) for IRF Think Tank Development Initiative (TTDI)
Date: 2 June 2014
By: Martin Schmidt, SPM Consultants for
1.
Background and definition
Results-based management (RBM) is one of several ways to describe a results-oriented management perspective. RBM is separate from similar approaches not in its fundamental notions, but in its holistic approach to organisational behaviour and, in development cooperation, a global agreement manifested in the Paris Declaration on Aid Effectiveness (2005). The following definition is from Meier1: “Results-Based Management (RBM) is a management strategy aimed at achieving important changes in the way organisations operate, with improving performance in terms of results as the central orientation. RBM provides the management framework with tools for strategic planning, risk management, performance monitoring and evaluation. It’s primary purpose is to improve efficiency and effectiveness through organisational learning, and secondly to fulfil accountability obligations through performance reporting.“
A general results-management perspective is since the 1980s part of the modern OECD concept of good governance. The notion is integral to almost any management standard public or private available. Its basic idea is that we shall allow real performance to influence policy, decision-making, and learning. In turn, this means RBM is intimately associated with the planning process — the revolving procedure of plan/report/plan/report where the next plan is based on an assessment of past performance — of every organisation. Another general term often employed for this aspect of RBM is evidence-based planning.
2.
Approach to management
RBM is distinctly oriented towards strategic planning, organisational learning and results for the beneficiaries. What makes it “results-oriented” is the idea that from regular monitoring of results we learn about what works and what doesn’t, and how one may proceed and improve in the future. In RBM, what really matters are changes for the beneficiaries; which is called outcomes. It means that we are a little less concerned with how many doctors, medicines and heath stations we have, than with what they provide in terms of improved health status for the target population. Is infant mortality reduced? Is the individual doctor improving his or her diagnosis? Is the spread of the HIV virus prevented? 1
Meier, Werner; Results-Based Management: Towards a common understanding among development cooperation agencies, for consideration by the DAC Working Party on Aid Effectiveness and Harmonisation October 2003.
In turn, the approach is based on the idea that environments and organisations are evolving, and the expected impact of a given set of actions is depending on these changes. No outcome is a given and we need to monitor progress in order to understand how the target of our efforts is responding to what we do. The approach seeks to contextualise and adapt to changing circumstances as a fundamental principle. The RBM perspective can be summarised in a few bullets;
3.
The analytical, contextual, and operational point of departure is in beneficiary level outcomes. Actual results should be allowed to influence policy making, decision-making and dialogue agendas during, not after, implementation. Performance is monitored to create a basis for planning, which means in practice that planning and reporting come together in a common operational framework.
The working tool
The basic working tool in RBM is the results-chain. It is a representation of reality in causal and temporal terms.
The standard definitions for these levels follow the OECD/DAC2: Input (Activity Output Outcome Impact
The financial, human, and material resources used for an operation. The activity is the work performed in which inputs are mobilized to produce outputs.) 3 The products, capital goods and services which result from an operations activities. The short- or medium-term effects of an operation’s outputs. 4 The long term and societal beneficiary level effects intended or unintended by specific operations or policy.
In the planning process, result managers begin by expressing objectives for beneficiary level 2
OECD/DAC; Glossary of Key Terms in Evaluation and Results-Based Management. OECD, Paris 2002. ”Activities” are often put within brackets because indicators to capture them are conceptually different (momentary) from indicators on the other levels. Sometimes they are called process indicators, as in the new policy will be published in May. 4 Their short- or medium-term nature is what motivates the outcome box to be broader than the others. For analytical purposes, this aspect has wide-ranging consequences. This is further explored in section 4. 3
outcomes. After that, the complete set of expectations along the chain are spelled out in what is sometimes referred to as a strategic framework; we do A to achieve B and expect this to happen in the following manner. In the course of operations, progress is monitored through the use of indicators. In the image below some of the key aspects of the chain and its use are illustrated:
Indicators of results are neutral descriptions of reality – separated from objectives that are always normative – and found on the levels or output, outcome and impact. An indicator is always measurable and consists of empirical information; such as the number of hospitals, proportion of children in school, literacy in the age group 15-24. Implementation is the part of the chain over which the responsible actors have direct control; resources, activities and their immediate consequences. Primarily, but not exclusively, factors internal to the operation affect the relationships of the chain. Development results describe change patterns on beneficiary level. In RBM, such patterns displayed as trends over time are central to any type of policy-, strategic- or operational analysis. Is the Maternal Mortality Rate (MMR) dropping or not, is democratic representation improving? Outcome trends matter excessively because depending on direction, often completely different causes of action are warranted. The further one travels from left to right in the chain, the more numerous are the potential external factors that affect the relationships between the boxes. This is why it is rarely a good idea to describe or seek impacts as a consequence of a particular action programme. Although impacts are conceptually the same as outcomes (beneficiary level changes), their societal and long-term nature make them virtually impossible to describe as a clear consequence of just about anything.
This problem, equally apparent to impacts as to longer-term outcomes, is often referred to as the “attribution problem�. Part of the reason for the outcome box being wider than the others has to do specifically with dealing with it.
4.
Theory of change
In practice, RBM is a learning approach to management. One observation it is based upon, is that managers seem to be significantly more successful when approaching their operations (and staff performance) from the point of view of objectives and results, rather than from the point of view of budget turn-outs and activities. For the approach to be valid, however, the manager needs both an idea of what he or she wants to achieve, and a theory of how to get there. This theory is often referred to as a theory of change. Most changes occur when people or organisations change behaviour and do things differently. In RBM, as we have seen, the key to understanding change is to observe outcomes on beneficiary level. The key is obviously not how many laws or medicines there are if they are neither observed nor put to use. The following image (from the results chain) summarises the key relationship in RBM; where outputs are converted into outcomes. Because most changes in modern society happens within some form of institutional context, this is illustrated in the text - institutional change, institutional effectiveness etc. - but the same reasoning can be applied equally to individuals or larger structures in society such as the democratic or legal systems.
With this image as a point of departure, almost any type of change in the behaviour of people, organisations or society can be illustrated. 1. Activities result in immediate consequences called output, such as people trained in a new method, a new law drafted or recommendations delivered.
2. If a person or organisation starts behaving differently because of the said output, change is occurring in the form of a short-term outcome; such as a new method or law being used in practice. 3. In turn, if the change is positive and improves on the effectiveness of an organisation, a medium-term outcome occurs; such as the application of a new method improving production efficiency, or a new law leading to lower crime-rates. 4. Finally, the image identifies a long-term outcome occurring when improved efficiency spills over into outcomes of a higher order, often manifested in public goals such as better health status or learning outcomes in schools; in this case perhaps lower public spending for a certain good or a safer environment for a crimes-exposed group in society. Observe the split-up of the outcome box of the results-chain. It serves many purposes of which two are noteworthy. a) First, it improves the level of precision of the theory of change so that planning and reporting, based on evidence from what is happening on each level, is greatly facilitated. b) Secondly, by reducing the conceptual gap between the steps of the results chain, each link is more credible (than if the gap is greater and producing the “attribution problem� encountered in section 3) and it becomes safer to suggest that one result is connected with the next.
5.
Planning and reporting in practice
5.1
Planning framework
As already suggested, planning is the central process of RBM thinking and any attempt to monitor performance should be designed to support it; i.e. to answer questions important for planning decisions. To arrive at this stage, the first step is to identify a credible and operational objective on beneficiary level. What is the change for the beneficiaries that a given set of actions should be designed to promote? The recommendation is to start the identification process with the highest - most overarching, most long term - possible objective in existence, and then to work ones’ way down to an objective that is immediately relevant and measurable for a operation. In the image of section 4, this objective is normally found on the levels of medium- or long-term outcomes. Four general steps characterises the identification process: A first step is to start with policy, either the existing Government policy or the policy one would prefer if existing policy is inappropriate or non-existing for the subject area. Policy objectives are always normative and often general: All children should go to school, all
citizens should have access to an impartial legal system, all citizens should have the right to basic health care. Step two is to agree on what is hindering the fulfilment of the overall objective, what is known as a problem analysis. If the problem is that not all groups in society have access to democratic institutions, and overall policy says democratic governance is a national objective, a development actor in society (an NGO for example) could decide that political participation is the preferred objective to support the fulfilment of policy. Step three is to make the objective operational, meaning to turn it into something observable and measurable. Participation, which is used in this example, is often observed as the proportional representation of different social groups in political assemblies or government institutions. Making this objective operational then means to say: The representation of group X in this political body should increase from the current level to a desired level Y. A final fourth step is to design the operations - resources (input), activities and outputs - necessary to achieve the objective. The path from input-to-objective is explained in a theory of change (see above and below), which describes how implementation shall turn into the desired outcome for the beneficiaries. When this is done, it is time to develop the flip-side of the equation plan/report, and design the reporting framework that will allow the implementer (the NGO in our case) or external party to monitor progress towards achievement of the objective.
5.2
Reporting framework
To design a reporting framework, the following general steps are required once the outcome objective, and the explanation of how it shall be achieved, has been established: 1. Identify results indicators by which the path through the results-chain towards the outcome objective can be monitored. 2. For each indicator, develop a baseline, a target and an expected timeline and group them together in a monitoring framework to be displayed both in the plan and the report. 3. For the monitoring framework as a whole, identify key risks (internal and external to the operation) and associate them with the relevant results achievement. 4. In reporting, make a performance assessment based on results turn-out. 5. In planning, make explicit reference to the performance assessment and its consequences for continued operation. In this hypothetical example, an NGO seeks to support political participation by involving more youth and women in the political discussion on European integration. A key objective
identified is to improve the political representation of youth and women in local political bodies. The theory of change reads: By involving the target group (youth and women) in seminars, debates and trainings organised by the NGO, the NGO and its objectives will come into focus and provoke the press, political bodies and society to recognise the need to engage youth and women in the debate on European integration. In turn, public recognition and an increasingly aware and empowered target group is expected to spill over in a higher degree of political participation, and election to political bodies, of representatives of the target group. Based on this theory, the following hypothetical monitoring and reporting framework is created:
It should be noted that the outcomes are presented in an order were longer-term outcome (impact) are presented at the top (outcome 3), medium-term outcomes follows (outcome 2) and short-term outcome (outcome 1) is presented further down. These are linked in a results chain. As is evident, the framework identifies a few key terms such as baselines and targets. These are all there for the purpose of assisting the planning process. Recall that this is a primary purpose in RBM.
Baselines are used to define a starting point and the level of progress, and also to provide analytical input to the planning process. For instance, if the 25% female representation of body X had been 5% in the previous election, it has other consequences for the NGOs approach than if the same number in the previous election had been 40%. Targets are equally there to provide managers with an operational focus, and also when monitored, to provide input to the performance assessment. Frequency defines how often a particular set of indicators should be followed-up. It is important to highlight that indicators on outcome level requires as frequent monitoring as many of the output indicators. That is to be able to use the results as lessons learned for a project/programme and adapt the implementation process. As the NGOs programme evolves over time, it is monitored with reference to the indicators identified, and discussed in light of the risk assessment. This assessment in not visible in the framework above but is of critical importance: The risk assessment is developed by asking the question “how?� when results on one level leads to the next. It is the internal and external factors that are either favourable or hampering the realisation of a result that is the subject of attention. either favourable or hampering the realisation of a result that is the subject of attention.