Big Results Now! in Tanzanian education: Has the Delivery Approach delivered? March, 2017

In recent years there has been growing interest across Governments and education Development Partners in looking beyond the formulation of best practice and policies, to focus on implementation and ‘getting things done’ using a set of ideas and structures called the ‘Delivery Approach’. This approach encompasses the set-up and operations of centralised ‘Delivery Units’ and the application of best practice principles. Initially popularised in the early 2000s by the UK Government, they have been applied in diverse contexts such as Tanzania, Punjab and Malaysia.

Tanzania’s Big Results Now! (BRN) Education was a transformational government led programme initiated in 2013 and which sought to apply the Delivery Approach to adopt learning orientated education reforms, to bring about dramatic improvements in examinations results across Tanzania’s 20,000 public schools. Complex, politically sensitive education reforms were proposed, particularly around enabling and motivating teachers and the use of accountability measures derived from high stake examinations and learning assessments.

This paper, written by practitioners directly involved in BRN, provides reflections on the political economy, achievements and challenges of education system reforms, particularly surrounding teachers. It looks beyond public proclamations of success to consider whether BRN was genuinely effective in overcoming political obstacles to education reform and delivering sustainable improvements in learning.

The paper considers the extent to which BRN Education adhered to the principles of successful delivery and its sustainable impact on quality and access at national scale. In so doing we identify important lessons regarding the application of education system performance management through country-led (rather than donor-driven) reform programmes. We note limitations in wholesale export of prescriptive educational reforms, and the need for local inclusive and politically grounded solutions that tackle the main barriers to equitable teaching and learning.

These lessons may enhance global efforts to deliver inclusive and quality education for sustainable development. As set out in The Learning Generation’s call to action (Education Commission, 2016), performance management delivery approaches are one of four pillars required to deliver education reform to scale.

Authors: Robin Todd, Cambridge Education: robin-todd@camb-ed.com Ian Attfield, Dept. for international Development: ian-attfield@dfid.gov.uk

Note that this paper expresses the views and perspectives of the authors alone and not in any way the official position or views of their respective institutions. Any mistakes or omissions within this paper are the faults of the authors alone

Introduction

In recent years there has been growing interest across Governments, Multilateral and Bilateral Development Agencies in looking beyond the formulation of best practice policies and focusing on implementation and ‘getting things done’. At the heart of this interest has been a set of ideas and structures which can be termed the ‘Delivery Approach’. The Delivery Approach encompasses the set-up and operations of Centre of Government Delivery Units (at either Presidential, Prime Ministerial or Ministerial level) as well as the application of a set of best practice principles initially popularised in the early 2000s by the UK Government’s Prime Ministerial Delivery Unit (PMDU). The World Bank under the leadership of President Jim Yong Kim has played a key role in advancing thinking on the Delivery Approach or what it terms the ‘science of delivery’.1 Although as Dan Hymowitz from the Africa Governance Initiative (AGI) quite rightly points out achieving results through the Delivery Approach is as much of an ‘art’ as it is a ‘science’, requiring a shrewd understanding of politics and incentives. 2

In March 2014 Cambridge Education wrote a paper entitled “Delivery Units: Can they catalyse sustained improvements in education service delivery?”3. This paper provided a summary of the defining features, principles and practices of the Delivery Approach and examined how the approach was being applied within the education sector in Tanzania and Pakistan. At that time the application of the Delivery Approach in Tanzania, called the ‘Big Results Now! (BRN)’ Programme, was still in its infancy having been launched by President Kikwete in early 2013.

Tanzania’s BRN was a broad, transformational government programme which sought to adapt the successful Malaysian approach to economic development and improving service delivery. This involved a switch to ‘business unusual’, operating in a new, more open and accountable way and focusing on delivering tangible results in priority sectors. The BRN programme launched in 2013 focused on achieving a set of specific objectives by 2015/16 in six priority sectors: Education, Energy, Agriculture, Water, Transport and Resource Mobilisation.

1 In his inaugural address to the World Bank’s Annual Meeting in Tokyo in October 2012, President Kim said: “This is the next frontier for the World Bank Group- helping to advance a ‘science of delivery’. Because we know that delivery isn’t easy- it’s not as simple as just saying ‘this works, this doesn’t’. Effective delivery demands context-specific knowledge. It requires constant adjustments, a willingness to take smart risks, and a relentless focus on the details of implementation.”

2 Hymowitz, D. (2016)’Too much science, not enough art’, Tony Blair Africa Governance Initiative 3 Todd, R.; Martin, J. & Brock, A. (2014) “Delivery Units: can they catalyse sustained improvements in education service delivery?”

This paper is written as a follow up to Cambridge Education’s March 2014 paper. The two authors of this paper- Robin Todd and Ian Attfield- were both heavily involved in BRN Education from its inception in 2013 up to mid 2016. Robin provided embedded technical assistance within the Ministry of Education, Science, Technology & Vocational Training (MoEST) from 2013 through until 2016- initially in the Ministerial Delivery Unit (MDU) and subsequently, from mid-2015, within the Education Program For Results (EPforR) Coordination Unit.4 Ian Attfield was DFID Tanzania’s lead Education Advisor from 2012 until June 2016. In this role Ian was responsible for overseeing DFID’s financial support to Tanzania’s education sector, which included significant funds in support of BRN education activities. Ian also played a lead role in designing the BRN Education Programme for Results (EPforR) financing instrument which, co-funded by DFID, the World Bank and the Swedish Government, made over $250 million available to the Government of Tanzania’s education sector.

This paper provides a personal reflection on the achievement and challenges relating to the Tanzanian education sector and BRN from 2013 to 2016. It attempts to move beyond the rhetoric and public proclamations of success to examine the evidence and consider whether BRN was genuinely effective in delivering results and change within Tanzanian education.

The paper begins by summarising the key principles of successful delivery set out in the Delivery Approach literature; these principles then serve as the framework for analysing the successes and challenges of BRN education in Tanzania.

4 Prior to October 2015 the Ministry of Education, Science, Technology and Vocational Training (MoEST) was called the Ministry of Education and Vocational Training (MoEVT). For the sake of consistency MoEST is used as the acronym for the Ministry throughout this paper regardless of the time period referred to. Likewise the Prime Minister’s Office for Regional Administration and Local Government (PMO-RALG) was renamed the President’s Office for Regional Administration and Local Government (PO-RALG) in October 2015. PO-RALG is used throughout this paper regardless of the period referred to.

Principles of Successful Delivery

Cambridge Education’s March 2014 paper identified 11 ‘lessons learnt and key principles’ from the growing body of literature on the Delivery Approach and its application in different country contexts. It should be noted up front that this is an inherently government led approach, which does not explicitly consider some of the political challenges, constraints and interests of elite groups that may have vested interests in the status quo or resisting technocratic reforms.

For the purposes of this retrospective review of BRN Education in Tanzania we have grouped these 11 lessons and principles under four broad headings namely:

- Prioritisation and Resourcing

- Data, Information and Routines

- Analysis and Understanding of Delivery Issues

- Communications and Culture Change

The relationship between these four headings and the 11 lessons learnt and key principles is set out in Figure 1 below.

Figure 1: Principles of Effective Delivery Prioritisation and Resourcing

 Focus on a limited number of key priorities which are clearly understood across the delivery system

 Ensure that there is a strong link between priorities and resources so that adequate budgets are available to support each priority.

 Develop a clear understanding of citizen centred outcomes so that key priorities are viewed from the perspective of what is achieved at the level of individual citizens rather than what government spends to deliver services.

Data, Information and Routines

 Use regular data as the basis for establishing effective performance management routines

 Develop good quality data and metrics to measure what matters. Collect reliable data for a small number of priorities and then ensure that data is analysed and used regularly to inform decision-making

Analysis and Understanding of Delivery issues

 Stakeholders are actively engaged in analysing delivery issues and owning outcomes through ‘Delivery Labs’, publishing data and feedback mechanisms.

 Clear understanding of delivery systems to identify the drivers of successful outcomes and the motivations and perceptions of actors throughout the system

 Understand and involve front-line workers in analysis and developing solutions

 Develop an effective support and challenge function at national and local levels.

Communication and Culture Change

 Develop an effective communications strategy to assist in rapidly engendering change and reform to ‘turn around’ a perceived decline or deficit in standards of service delivery

 Strike the right balance between planning and delivery, recognising which areas can achieve rapid results and others where it may take a longer time.

Prioritisation & Resourcing

The first key success area, ‘Prioritisation and Resourcing’, requires government to develop a clear idea of what they want to achieve. Too often governments will state that almost everything connected with education service delivery is a priority, making it difficult to focus resources in making a tangible difference. Where multiple constraints exist and resources (financial, human and physical) are finite, trying to address a multitude of problems simultaneously is not likely to lead to success. Even within a relatively ‘resource rich’ delivery system such as the UK, the PMDU found that relentless prioritisation (on, for example, early grade literacy or results in Inner London schools) was necessary, if rapid improvements were to be brought about.

When adopting the Delivery Approach governments must be clear as to what they want to achieve, selecting a limited number of key areas which are a priority. They must then ensure that these priorities are clearly understood: from national level, through regional and local government structures, down to individual schools, colleges or training centres. If there are dis-connects or conflicts in any part of the delivery system then this will undermine the chances of success. Actors throughout the system need to know what the government wants to achieve and how their individual actions will contribute towards these goals.

Prioritisation is one thing, but it must then be backed up by resources. This does not necessarily mean utilising more resources or launching new projects. Within the UK PMDU when teams conducted problem solving ‘Priority Reviews’ with stakeholders the question of finance was specifically addressed. Stakeholders were told that they could not suggest increased resources as a solution to delivery issues unless they specified which things they would stop financing to make the additional resources available. The focus of the PMDU therefore was very much on achieving better results with the resources which were currently available.

This is not to say that the Delivery Approach cannot be a means of securing additional funding for priority areas, indeed in many cases it serves this purpose. The problem arises when governments produce and communicate aspirational delivery plans which contain unfunded commitments. In many countries there is often a sense of weariness or policy fatigue when governments announce new plans and objectives. Citizens often see this as purely political - having heard many promises of better results over the years without seeing tangible change. Announcing ambitious delivery plans which contain aspirational and unfunded commitments runs the risk of adding to this sense of cynicism.

The final aspect of prioritisation is that of citizen centred outcomes. This involves viewing policies, interventions and targets from the perspective of the citizens who will benefit from improved services. Too often, at a national level, success is viewed in terms of the amount of money spent or beneficiaries reached, rather than the actual

quality of delivery or actual impact on peoples’ lives. Looking at service delivery from the perspective of the end users is an important shift in thinking, which can then permeate the delivery system. Taking a citizen centred approach enables governments to then start thinking about how they can achieve ‘better for less’.

Data, Information & Routines

The second key success area, Data, Information and Routines, requires governments to establish mechanisms to collect accurate and timely data related to their priorities and then use this data to hold the delivery system to account. Timely and accurate data is the lifeblood of the Delivery Approach. Without it there is no means for the centre of government to hold actors in the system to account for their progress in achieving results. Regular, monthly, independent collection of school specific data on teacher attendance and a set of other indicators was a key feature of the Punjab Roadmap reform programme, that were used to generate heat maps for senior level performance monitoring and course correction meetings5

It is important to develop good quality data and metrics to measure what matters. ‘What matters’ is important. Too often traditional reporting systems gather masses of data infrequently (for example, a very long annual school census questionnaire). Collecting masses of data increases the length of time and complexity of analysis, by the time information is available it is then often too late to take remedial action to get delivery back on track.

The quality and reliability of the data which is being collected is another important issue. If there is significant doubt then it will lose its effectiveness as a means of holding actors to account for performance. Time will then be spent disputing the validity of the situation portrayed by the data rather than focusing on addressing delivery issues.

The Delivery Approach relies on collecting reliable data for a small number of priorities and then ensuring that data is rapidly analysed and used regularly to inform decisionmaking. In some cases data collection can be as frequent as daily, weekly or monthly whilst for assessments of achievement it may be quarterly or annually. It is important not to overburden the system by collecting too many metrics or data which will not be analysed and used for decision-making. Use existing systems wherever possible to reduce cost and enhance the potential for sustainability and introduce rapid, adaptive feedback loops so that the people collecting data know that it is being analysed and used.

Data collection and analysis by itself is insufficient to drive change, it must be used as the basis for establishing effective performance management routines. These routines

5 M. Barber, 2013, The good news from Pakistan How a revolutionary new approach to education reform in Punjab shows the way forward for Pakistan and development aid everywhere

rely on regular meetings with system leaders where information is reviewed, analysed and used to course correct and agree remedial actions.

Initially governments should work with stakeholders to develop delivery plans with actions and named responsibilities. These plans should then be used to plot evidence based performance trajectories which will be used to track progress over time. Forums then need to be established to review progress and ensure that the appropriate people are held accountable. A typical performance management routine would involve regular meetings within the education ministry that cascade bi-directionally. Accountability should not just stop at ministerial level, consequences for performance right throughout the system is needed to develop a results culture which is everybody’s business.

Analysis and Understanding of Delivery Issues

The third key success area, Analysis and Understanding of Delivery Issues, requires government to develop an understanding of what exactly is required to achieve results. In some countries writing policies and agreeing targets is technically out-sourced to external consultants, with little consideration of the political economy. Problems arise when it comes to turning these policies and targets into tangible delivery. Prioritisation and improved data flows linked to performance management routines can all play a role in addressing this core problem. By themselves these actions are not going to succeed unless government has a clear idea as to how services are actually delivered (in reality rather than set out in guidelines) and the incentives to adjust them by stakeholders throughout the delivery system.

One way of attempting to address this issue is by actively engaging stakeholders in analysing delivery issues and owning outcomes. One means of doing this is through ‘Delivery Labs’, first introduced by PEMANDU in Malaysia, where a range of stakeholders (senior government officials, academics, union representatives, local government workers, teachers and civil society) worked together to identify problems, analyse issues and develop a mutually agreed delivery plan.

Whichever method is chosen to identify and analyse issues it is important that stakeholders develop a clear understanding of the education delivery systems and identify the drivers of successful outcomes and the motivations and perceptions of actors throughout the system. Without this understanding interventions designed at a national level will often fail because they don’t take account of the actual situation on the ground and make ungrounded or unrealistic assumptions. It is vital to involve frontline workers (including teachers, principals, local government officials and community leaders) in analysing problems and developing solutions.

At times government will be faced with a trade-off in terms of the desire to move rapidly and put in place an ambitious, nationally agreed set of targets and the need to consult

and gain local understanding and ownership. The two issues are not necessarily mutually exclusive. Representative stakeholders can be involved in national level discussions whilst flexibility should be built into the planning and target setting process so that national targets can then be disaggregated by locality or school. The same process of results accountability which has been put in place at a national level should then be reflected in regional, district or local level arrangements so that a ‘delivery culture’ can be promoted throughout the system.

Dedicated Delivery Units or their equivalent at both a national and local government level can play an important role in assisting stakeholders to analyse and understand delivery systems and instil a proactive culture of analysis and problem solving although care must be taken not to establish ‘parallel structures’ which undermine existing systems. Small units of talented staff can play an invaluable role as a support and challenge function, adding real value by working collaboratively to build capacity and solving problems. Often when governments adopt the Delivery Approach they focus initially at the national level, looking at the capacity and culture which exists within national ministries. This is a necessary first step to achieving results but strengthening the centre alone does not provide a sustainable basis for transformational change. The Delivery Approach needs to become embedded at sub-national and institution level, so that stakeholders are locally responsive and accountable.

Communication and Culture Change

The fourth and final key success area, Communication and Culture Change, requires governments to communicate their priorities to actors within the delivery system and the public at large in order to create a culture of shared responsibility for achieving results.

The Delivery Approach is often adopted by governments to rapidly engender change and reform to ‘turn around’ a perceived decline or deficit in standards of service delivery. This requires government to acknowledge that there is a problem, to be transparent and frank with their citizens. It is then important to attempt to inculcate a sense of shared responsibility for results transformation rather than seeing service delivery as an ‘us and them’ issue; whereby government delivers and citizens receive.

To do this entails engaging multiple stakeholders and creating in them both the belief that things can change and the willingness to engage in change. This in turn means that significant effort has to be put into publicising the work, and eventually the successes, of relevant actors including ministries, local governments and schools. In short government needs to develop an effective communications strategy.

In some systems (such as Malaysia) the unit responsible for the Delivery Approach is given a very high profile and takes much of the credit for the successes achieved. In others (such as the UK) the central unit was all but invisible to the general public and

facilitates the processes behind the scenes whilst ensuring that credit for success is taken by the relevant line ministries, local governments and schools.

To what Extent were the Principles of Successful Delivery Applied in Tanzanian education?

Overview of BRN Education

President Kikwete launched the Tanzanian government’s Big Results Now! (BRN) programme in early 2013. BRN was a transformational government programme which sought to adapt the successful Malaysian approach to economic development and improving service delivery. This involved a switch to ‘business unusual’, operating in a new, more open and accountable way and focusing on delivering tangible results in priority sectors. BRN initially focused on achieving a set of specific objectives by 2015 in six sectors: Education, Energy, Agriculture, Water, Transport and Resource Mobilisation.

A six week ‘Delivery Lab’ was carried out for each of the six priority BRN sectors –Education, Agriculture, Energy, Water, Transport and Resource Mobilisation6 . Facilitated by staff from Malaysia’s Performance Management and Delivery Unit (PEMANDU) and McKinsey, each Lab involved the participation of key Tanzanian experts and government officials. Each Lab identified a set of priority actions and targets along with a detailed delivery plan (called a ‘3 Feet Plan’) setting out exactly what needed to be done by whom and when to deliver the proposed activities.

BRN arrived at a time when the Tanzanian education system was at a critical juncture. The past few years had seen a significant and rapid expansion of enrolment at secondary level where the number of students almost tripled in seven years from 0.68 million in 2005 to 1.8 million in 2012. Primary level enrolments had also increased rapidly in the early 2000’s following universal primary education (UPE) campaigns, but the rate had since plateaued around 8 million in 2012. Unfortunately these increases had not been accompanied by improvements in the quality of education received by the majority of Tanzanian children. Results in both the Primary School Leaving Examination (PSLE) and Certificate of Secondary Education Examination (CSEE) had deteriorated markedly between 2005 and 2012 as Figure 2 illustrates. Exam results are an imperfect proxy for educational quality but they are the best national measure available in Tanzania which captures performance at the end of primary and secondary

6 The proposal to run the Delivery Labs was made following a Presidential visit to Malaysia, following the request of Tanzania, DFID agreed to cover most of the costs of running them in its initial support to BRN.

levels.7

This deterioration in pass rates sparked a national debate about the reasons for the declining standard of education in Tanzania. This debate dominated the national media and led to heated discussions in the Tanzanian Parliament (Bunge). The debate continued throughout the first half of 2013 and came to a head in the middle of the year with the announcement that the 2012 CSEE results were to be re-graded, a process which led to the national pass rate for 2012 increasing from 34% to 43%8. There were also a change of leadership at the national examination council.

Crisis events and ‘shocks’ are recognised in public policy theory as having the potential

7 The current system could be significantly improved and strengthening Tanzania’s capacity to conduct robust student learning assessments and examinations is one area where technical assistance was recommended, but not taken up.

8 Typical headlines of the time read “Massive examination failure must be stemmed” (Daily News, 20th February 2013 and “Spectre of poor quality education stalks Tanzania” (The East African, 18th May 2013).

Figure 2 Primary School Leaving Examination (PSLE) and Certificate of Secondary Education Examination (CSEE) pass rates in Tanzania between 2005 and 2012

to catalyse significant changes in practices which would not be considered in a period of equilibrium. 9 If there is widespread acceptance and acknowledgement that there is a problem then this creates opportunities to challenge the status quo. Arguably the poor results and subsequent national debate on the state of Tanzanian education in 2013 created just such a window of opportunity for change and also contributed to education being included with the first six priority BRN sectors.

BRN Education Initiatives & Targets

It should therefore come as no surprise that the plan developed by the Education Delivery Lab in the first quarter of 2013 focused on significantly improving the quality of primary and secondary education. It planned to do this through nine activity strands including improved delivery of school capitation grants; greater focus on the 3Rs (reading, writing and arithmetic) during the early years of primary school through teacher training and assessment; the mass roll-out of remedial classes: entitled Student Teacher Enrichment Programme (STEP) for students scheduled to sit primary and secondary examinations and measures to address the significant number of outstanding teacher claims in the system. The overall aim of the BRN Education plan was captured in 3 targets to be achieved by the end of 2015 or early 2016, namely:

 To create transparency on 3R skills levels in Standard II.

 To achieve 80%+ pass rates in Primary PSLE Exams.

 To achieve 80%+ pass rates in Secondary CSEE Exams.

These targets were known as the ‘National Key Results Area’ (NKRA) for Education. In order to oversee progress across the six NKRAs (one for each target sector) Tanzania established a comprehensive BRN delivery architecture and regular performance monitoring regime. At the centre of this architecture was the President’s Delivery Bureau (PDB) headed by a prominent Tanzanian from the private sector10. The PDB was an independent unit in the President’s Office responsible for ensuring delivery of BRN objectives. The PDB monitored and reported on BRN progress and was the secretariat to the Transformation Delivery Council (TDC). Chaired by the President the TDC was intended to meet on a monthly basis and incorporated the Vice President, Minister of Finance and all relevant line Ministers and Permanent Secretaries. The TDC monitored overall delivery progress and problem solves difficult issues.

Each of the 6 BRN ministries then had its own Ministerial Delivery Unit (MDU). The Education MDU reported to the Minister for Education and was responsible for progress monitoring and reporting on the Education BRN plan, problem solving and analysis, supporting delivery capacity and communicating. Composed of a small team

9 See for example Andrews, M., Pritchett, L. & Woolcock, M. (2012) ‘Escaping Capability Traps through Problem-Driven Iterative Adaptation (PDIA)’, Center for Global Development Working Paper 299 & Andrews, M. (2013) The Limits of Institutional Reform in Development

10 A former CEO of the Investment Climate Facility for Africa and Executive Director of a telecoms operator.

of five staff from MoEST the MDU was headed by an internal MoEST appointee in its first year, and then by an external appointee put in place by the PDB until the dissolution of the BRN programme. The MDU acted as the secretariat for the NKRA Education Steering Committee, chaired by the Minister for Education, which met every month to look in detail at progress and issues against the Education BRN plan.

To what extent did Tanzania’s BRN Education plan, structures and approach adhere to the best practice principles of the Delivery Approach identified earlier in this paper?

Prioritisation and Resourcing

The BRN education delivery plan demonstrated a degree of prioritisation in that it contained a set of nine priority initiatives each with a dedicated set of activities and targets (see Figure 3 below for details).

The nine BRN initiatives contained a broad range of activities - from school construction to eliminating the backlog in outstanding teachers’ salary and non-salary claims and from teacher training to the public ranking of schools and the provision of related rewards for improved exam results.

These initiatives were then intended to contribute to a set of three headline results-

Figure 3: Summary of 9 BRN Education Initiatives from Delivery Lab Slide Deck

improvement in the PSLE pass rate and CSEE pass rates to 60% in 2013, 70% in 2014 and 80% by 2015 and to create transparency on the 3R (reading, writing and mathematics) achievement of pupils in the second grade of primary schools. This latter result was important because, at the time of the Education Delivery Lab, there was no national achievement survey for the 3Rs in Tanzania. As we shall see, improvement in the 3Rs may be BRN’s most important and lasting achievement, however as a target it was often overlooked by the public, media (and, at times, the Ministry) as BRN came to be viewed as synonymous with improvements in high stake examination pass rate.

The broad range of BRN initiatives may have been considered necessary to bring about improved results but the focus across a relatively wide range of activities at the same time created several issues, particularly during the early days of BRN. These issues included budget (to prioritise funds across so many activities, and out of synchronisation with the government budget cycle), focus (limited capacity to focus on problem solving, data collection and getting the mechanics of delivery right) and understanding (limited awareness of all the specific BRN initiatives, in contrast the water sector BRN plan had only one headline- operational rural water points) There was excellent awareness of BRN’s headline objectives to improve exam pass rates amongst local stakeholders, but they were much less clear on the nine specific initiatives intended to achieve these11

The BRN Education delivery plan contained a number of unfunded and underfunded commitments when it was completed in April 2013. This caused significant concerns within MoEST, although it was able to reallocate existing resources to make a rapid start with a limited number of activities- including STEP training for secondary school teachers, clearing a proportion of teachers’ claims and producing the School Improvement Toolkit.

It was not until the approval of Global Partnership for Education (GPE) funds and Education Programme for Results (EPforR) funding in late 2014 that sufficient resources were available to make substantive progress against the majority of activities. The Government up to 2015 was unable to make sufficient resources available to meet the commitments in the BRN plan for capitation grants (enrolment based finance and books to schools). It was only following the election of President Magufuli and the introduction of the ‘Free Education’ policy, that schools began to receive their full allocation of capitation grant funds.

Despite these early funding difficulties one notable success of the BRN Education Plan was the way in which it provided a clear, Government-led focus for funding from development partners. Tanzania’s successful GPE application was structured around support for the BRN plan whilst DFID, the World Bank and SIDA collectively provided results-based financing support which was structured, at least initially to indirectly

11 Field survey reports by the Education MDU, late 2013

finance ex-post, BRN’s priority activities. This coordinated support from development partners (which in all likelihood would not have been as ‘joined-up’ without the clear set of Government-led priorities identified by BRN) meant that education did not suffer from the persistent financing gaps which afflicted progress in other BRN priority sectors12

Prioritisation and Resourcing- Adherence to Delivery Approach principles Principle

Limited

Priorities were identified but they had a wide scope. Overall results of programme (improvements in PSLE and CSEE results) clearly defined and understood.

Initial weaknesses with resourcing gaps and unfunded commitments addressed as programme continued. Provided a useful focus for development partner support.

Improvements in exam results seen as the number one priority by citizens as they provide a clear means for students to progress to higher levels of education.

Data, Information & Routines

BRN put considerable emphasis on establishing effective performance management and monitoring routines at the centre of government. This included the creation of the President’s Delivery Bureau (PDB) and Ministerial Delivery Units (MDUs) in each key Ministry. At Presidential level the Transformation & Delivery Council (TDC) was the key means for holding Ministers to account for progress against plans. Within the education sector the monthly NKRA Steering Committee was the main accountability forum.

Chaired by the Minister for Education this Committee involved senior representatives from a range of organisations including MoEST, the Prime Minister’s Office Regional Administration & Local Government (PMO-RALG), the Ministry of Finance, the PDB, the Teachers’ Union and the National Examinations Council of Tanzania (NECTA).

The Education Steering Committee met regularly (although unable to maintain the monthly frequency of meetings which was initially intended) and proved a useful forum for identifying and addressing performance issues. Where the ability to address these issues lay within the authority and remit of those people in the room the Committee proved effective and made a series of notable decisions (such as the decision to implement a ‘direct to schools’ capitation grant delivery system which was eventually operationalised nationally by President Magufuli in 2016).

12 However only a partial overlap was evident in the education sector plan that underpinned the GPE application and the BRN priority education sector plan. Soon afterwards, political pressure was applied on local government officials to improve secondary school infrastructure, through a national campaign for science laboratory construction that was in neither of the former priority costed plans.

There were sometimes occasions however where issues could not be unblocked. Often these involved the Ministry of Finance whose representatives on the Steering Committee were not of sufficient seniority to make resource allocation decisions. These issues had to be escalated to the TDC, chaired by the President, which met increasingly infrequently as BRN progressed.

Whilst performance management and monitoring routines were established at the centre of government this was not the case at district level. In 2013 the Local Government system in Tanzania consisted of 25 regions and 164 Districts. Whilst efforts were made to establish Regional Delivery Units (RDUs) these had limited impact on the implementation and monitoring of the BRN education plan. Districts were not subject to a locally equivalent performance management routine as the one which was in place nationally. The Education MDU did disaggregate the national examination pass rate targets into a series of regional and district specific targets to track headline progress and introduce a monthly reporting regime. Districts were then given autonomy as to how they oversaw implementation at a local level.

The lack of timely and accurate information was a major weakness when the BRN Education Plan was launched. Within the Tanzanian education system the President’s Office for Regional Administration and Local Government (PO-RALG)13 is responsible for basic education and thus had reporting lines to all primary and secondary schools through regions (Regional Education Office- REO) and districts (District Education Office- DEO). MoEST is responsible for setting education policy whereas PO-RALG is responsible for overseeing implementation. The weaknesses of this system and inconsistent reporting lines between MoEST and PO-RALG were quickly exposed when BRN implementation commenced.

The PDB insisted on weekly and monthly progress reporting from the Education MDU. Initially the only way to gather progress data on initiatives such as school construction and capitation grant delivery was to make ad hoc phone calls to PO-RALG officials. They in turn would make phone calls to REOs and DEOs. The result was confused and often conflicting performance and progress information. This undermined performance management particularly as PO-RALG initially played a limited role in the delivery structures. The PDB was trying to hold MoEST Directors of Primary and Secondary Education responsible for initiative delivery when these Directors had only a policy, not an implementation mandate. Attempts to bring PO-RALG more clearly into the performance monitoring and management structures were never entirely resolved and bureaucratic resistance was not overcome.

In 2013 the reporting structure between DEOs and PO-RALG was based on the submission of quarterly narrative reports on primary and secondary education. These

13 Prior to October 2015 PO-RALG was known as the Prime Minister’s Office for Regional Administration and Local Government (PMO-RALG)

reports were voluminous, running into tens and in some cases almost a hundred pages, with significant repetition. Being quarterly in nature also meant that they were far too infrequent to meet the reporting requirements of BRN. Districts stated that they rarely (if ever) received feedback from PO-RALG. This is not surprising given that at the time PO-RALG was woefully understaffed, dealing with all aspects of primary and secondary education delivery across 164 districts. Compounding the problem, there were clashes over the operation of the Education Management Information System (EMIS) between MoEST & PO-RALG while annual school census data was habitually late and inaccurate.

To address this, the Education MDU worked with PO-RALG to develop a simple onepage excel based reporting template specifically for BRN activities. The initial reporting template was then refined and training conducted for Statistical Liaison Officers (SLOs) in each district. These were submitted by email to the Education MDU and PO-RALG through REOs. The report was compiled centrally and submitted to the Minister for Education. Personalised feedback was sent to all Regions and Districts through dedicated email groups. This feedback was warmly welcomed by local officials who stated that they now truly felt part of a responsive national system.

The monthly reporting system proved very labour intensive for the MDU but proved highly successful in enabling the centre of government to access timely progress information with useful narrative descriptions of implementation challenges. Over 95% of districts submitted their reports on time each month and it worked effectively for a twelve month period. Unfortunately the MDU did not have sufficient numbers of staff to sustain the system and provide monthly feedback so monthly district level reporting was discontinued in late 2015 as the final BRN targets deadline drew closer.

Data, Information & Routines- Adherence to Delivery Approach principles Principle Assessment of Adherence

Establishing effective performance management routines

Quality data and metrics to measure what matters

High/ Medium

Effective routines established at national level but not within Districts where monthly reporting was used as main lever to drive performance.

Medium

Significant weaknesses with reporting initially but improved considerably from June 2014 with the introduction of excel-based monthly district and regional reports. MoEST and PO-RALG were unable to sustain the intensive effort required to analyse reports and provide feedback to districts.

Analysis and Understanding of Delivery Issues

The BRN Education programme was launched through a large, lengthy and high profile ‘Delivery Lab’ conducted in the first quarter of 2013. The Lab was a six week planning session with 34 participants from a wide range of organisations including MoEST,

PMO-RALG, NECTA, Tanzania Education Authority, various universities and teaching colleges, development partners and civil society. It can be argued that the costs of the Lab were too high and that it could have been shortened to a week or two (with subsequent planning and detailed preparation of activities carried out be dedicated working groups) but there is no doubt that the process was very successful in establishing a high profile consensus over the issues that needed to be addressed in education. It also engaged a wide range of relevant stakeholders in analysis and planning.

The requirement to present the final plan to the President and have a signing ceremony where the Minister of Education took responsibility for delivering the results and targets was a very good way of getting high level endorsement. For all of its undoubted flaws (which will be touched on shortly) the BRN Education Plan was authentically and definitively the Government of Tanzania’s own plan. Indeed there is anecdotal evidence that in-country development agencies, (who were accustomed to supporting longer-term plans developed over many months and drafted in large part by external consultants), found the whole government-led pace and momentum of the BRN process alarming.

While most of the proposed activities in the BRN education plan displayed a good understanding of the education delivery system, some:

i.) were always likely to have minimal impact on examination results, at least in the short term;

ii.) failed to address the underlying systemic causes of issues and instead dealt with the symptoms.

An example of the former would be the production and distribution of the School Improvement Toolkit. While it is undoubtedly positive that all school principals have a concise guide setting out their roles and responsibilities it would be naïve to think that the simple act of delivering this Toolkit would have a meaningful impact on the quality of school leadership. In hindsight perhaps the BRN plan could have focused on the Toolkit as the entry point and then designed a district-based programme of training, twinning and mentoring.

An example of a BRN initiative which failed to address the underlying systemic cause of the issue was teacher claims. While funds were made available to clear the immediate backlog of outstanding teachers’ claims, no action was taken to analyse and improve the teacher claims system. The inevitable consequence was that, over time, the number of outstanding claims increased again and required a significant investment of Education PforR funds in 2015 to clear again. It was not until April 2016, once BRN had ended, that a thorough analysis of the teacher claims system identified the significant flaws which were leading to the constant backlog of claims.

The presence of these somewhat flawed initiatives should not distract from the fact that there were BRN initiatives which had a major positive influence on education performance in Tanzania. Such initiatives include 3R Assessment and Training, STEP remedial classes and capitation grant delivery (where the original BRN plan did not address the underlying cause of the problem but where the scrutiny provided did give a very significant momentum and eventual action).

One aspect of BRN delivery that could be called into question was the level of prescription and detail in the original ‘3 Feet Plans’ produced in the BRN Education Lab. The logic of producing a very detailed and prescriptive implementation plan is clear in that it enforces accountability and ensures that activities do not stall. This approach undoubtedly works well in Malaysia, which has a strong central government. In Tanzania, which has a more decentralised and diffuse education delivery system, the approach was not as effective. The ‘3 Feet Plans’ produced at the BRN Education Lab were very prescriptive and rapidly became out of date.

The BRN approach as implemented did not allow for much flexibility in reviewing and amending plans. The plan focused almost entirely on actions taking place at the centre of government and neglected communication and interactions with districts and schools. It is not surprising therefore that the most successful activity (STEP training for secondary schools) was one of the few activities where MoEST was directly responsible for implementation. The views of teachers, principals and local government officials were not adequately represented in the original 3 Feet Plans. However the lack of local delivery structures did allow for creativity as to how individual schools and districts implemented activities.

The Education MDU and PDB Delivery Team provided a support and challenge function to MoEST and PO-RALG at a national level. The Education MDU also played a support and challenge role for Districts based upon i.) feedback on monthly performance reporting and ii.) field visits to Districts. The ability of the Education MDU to provide this function to Districts was constrained by the fact that the MDU never had more than 5 staff to visit 164 Districts. With hindsight greater efforts could have been made to provide a support and challenge function to regions and districts, perhaps through peer networking and analysis sessions.

Analysis & Understanding of Delivery Issues- Adherence to Delivery Approach principles Principle Assessment of Adherence

Stakeholders are actively engaged in analysing delivery issues and owning outcomes High

The BRN ‘Education Lab’ which developed the education plan was a high profile, six week session with senior representation from a range of stakeholders. It ensured that the BRN education plan was truly ‘owned’ by the Tanzanian government.

Clear understanding of delivery Medium Whilst most activities displayed a good

Analysis & Understanding of Delivery Issues- Adherence to Delivery Approach principles systems understanding of the education delivery system there were several BRN initiatives which failed to address the underlying systemic causes of issues (e.g. teacher claims and capitation grant delivery). The initial BRN education plan also did not fully account for some of the difficulties involved in implementation caused by the structural and spatial differences between MoEST and PORALG. Some initiatives (School Toolkit and School Construction for example) were always likely to have minimal impact on examination results.

Understand and involve frontline workers

Effective support and challenge function

Low

High/ Medium

The ‘3 feet plan’ produced at the BRN Education Lab was very prescriptive and rapidly out of date. The PDB/PEMANDU approach did not allow for much flexibility in reviewing and amending plans. The views of teachers, principals and local government officials were not adequately represented in the activity plans. The lack of local delivery structures did allow for creativity as to how individual schools and districts implemented activities.

The Education MDU and PDB Delivery Team provided a support and challenge function to MoEST and PO-RALG at a national level. The Education MDU also played a support and challenge role for Districts based upon i.) feedback on monthly performance reporting and ii.) field visits to Districts. Coverage was constrained by the fact that the MDU never had more than 5 staff to visit 164 Districts.

Communication & Culture Change

Communications and culture change was arguably the greatest strength of the BRN programme. BRN had a very high profile launch in 2013 across all sectors and featured prominently across the print and broadcast media. Within the education sector MoEST and PO-RALG made a concerted effort to rapidly disseminate information on BRN initiatives and targets through their formal dissemination channels. Official letters were written to all District Education Offices and, a two-day BRN education launch event was held in Dar es Salaam. This launch was attended by at least 3 officials from every District and Regional Education Office. Performance contracts were signed by national, regional and district education officials as a sign of their commitment to deliver BRN initiatives and targets.

These communication efforts clearly had an impact because when the MDU TA Team carried out district-based fieldwork in December 2013 education officials, principals and teachers in remote rural areas of the country had all heard of BRN and knew that it was intended to drive up pass rates in primary and secondary schools. There was a clear sense that teachers, head teachers, principals and district officials would be held accountable for these pass rates, with some officials stating that BRN should stand for ‘Better Resign Now’ because they did not believe it would be possible to achieve these ambitious targets with their current level of resourcing.

Despite the high levels of awareness of the overall BRN programme and objectives there was much more limited awareness of the detail of the specific BRN initiatives and accompanying targets. In part this was due to the top-down formalised nature of communications through the existing education delivery system which meant that messages reached key officials but did not always permeate to the wider community. A study showed that the stakeholder groups with the lowest levels of understanding of BRN were communities, parents and pupils (Integrity Research, 2016)

Communication & Culture Change- Adherence to Delivery Approach principles

Principle Assessment of Adherence

Effective communications strategy

Public awareness of BRN Education’s objective of improving pass rates was very high, penetrating down to even the most remote rural areas. The key message about collective responsibility for improving education was also well understood. There was much more limited understanding of the 9 BRN education initiatives (such as School Ranking) particularly at school and community level due to the ‘top down’ nature of communications and reliance on government reporting lines.

Right balance between planning & delivery

The 9 BRN education initiatives contained a mixture of activities which would be like to have a short-term impact (school ranking, STEP classes) and longer term impact (3Rs Assessments and Training) on educational performance. Some of the proposed initiatives (most notably teacher claims but also capitation grant delivery) failed to address the root structural cause of issues and instead tackled the symptoms.

Achievements and Results

Examination Results

When the BRN education plan ended in 2016, had it achieved the objectives and results set out when it was launched in April 2013? The headline aim of BRN education was to improve pass rates for the Primary School Leaving Examination (PSLE) and the Certificate of Secondary Education Examination (CSEE).

In 2012 national PSLE pass rates had fallen to 31%. Following the introduction of the BRN Education NKRA in 2013 there was a rapid improvement in PSLE pass rates to 50.6% in 2013, 57.0% in 2014 and 67.8% in 2015. This may be behind the BRN target of 80% PSLE pass rates in 2015 but it still represents significant progress since 2012. However, as the graph below shows, the undoubted progress in pass rates achieved under BRN is less impressive if looked at in wider historical context. If the starting point for analysis is taken as 2005 rather than 2012 progress appears steady rather than spectacular. Comparability between years in terms of examination difficulty is also questionable and may account for some of the large variations; it was determined not to be a robust enough measure of learning outcomes, to be linked to the Education Payment for Results financing triggers.

Pass rates themselves can also be misleading as they don’t give an indication of the number of students taking the examinations. Tanzania has a rapidly growing population so can we make the assumption that growing enrolments and the consequent strain on

Figure 4: National PSLE Pass Rates 2005-2014

the education system have made it difficult to maintain pass rates?This was a key aspect of the narrative formed during the BRN Education Lab. However looking at the data over the past few years this argument fails to hold up to scrutiny. Firstly, using data from the Education Lab itself primary enrolments in Tanzania peaked in 2009 (unlike secondary which continued to display rapid growth) as Figure 5 shows.

Analysing the number of candidates sitting PSLE annually between 2012 and 2015 there was a steady drop year on year- from 865,534 candidates in 2012 to 763,602 in 2015almost 100,000 fewer candidates in a period of just three years. Consequently whilst the absolute number of candidates passing PSLE increased each year over the BRN period the increase was not as spectacular as the percentage pass rate figures would indicate. Still, it is important to note that the 518,034 candidates who passed PSLE in 2015 was almost double the 265,873 who passed in 2012- undeniably this represents impressive progress.

Figure 5 – BRN Delivery Lab analysis of education trends

Figure 6- Annual number of PSLE Candidates

At secondary level national ‘Form 4’ CSEE pass rates had fallen to 43% in 2012 driven in large part by extremely rapid increases in enrolment since 2006. Progress in CSEE pass rates showed significant improvement since the introduction of BRN with it increasing to 57.1% in 2013 and then 69.8% in 2014, before falling slightly to 68.5% in 2015. This meant that the 2015 CSEE pass rate was 11 percentage points behind the aspirational BRN target of 80%. It is also very important to note that the methodology for marking CSEE changed in 2014, thus weakening the comparability of results year on year.

Figure 7- National CSEE Pass Rates 2005-2015

What about the actual number of candidates sitting and passing CSEE each year?

There was a significant drop in the number of candidates sitting the national CSEE at the end of Form Four in 2014, largely because of the introduction of the requirement to pass exams at the end of Form Two in order to progress to Form Three. This policy was introduced prior to BRN and had a significant impact on the 2014 figures. The actual number of candidates writing CSEE in 2014 was 240,300 compared with 352,600 in 2013. This means that, while the overall pass rate increased significantly (from 57% in 2013 to almost 70% in 2014) the actual number of candidates passing CSEE declined from just over 200,000 in 2013 to 168,000 in 2014. When viewed in terms of actual number of students passing CSEE 2015 was by far the most impressive year. The actual number of passes in 2015 was 243,200 as the overall number of candidates sitting the exam rebounded to 336,800. The total number of students passing CSEE annually between 2010 and 2015 is set out in Figure 8 below.

Interestingly both PSLE and CSEE pass rates ended up looking very similar in 201567.8% for PSLE and 68.5% for CSEE. Whilst these results are below the aspirational BRN targets of 80% in both cases they represent significantly more young people passing examinations than had been the case in 2012. Overall therefore BRN is thought to have had a positive impact on PSLE and CSEE results albeit a more nuanced success than a simple percentage pass rate snapshot from 2012 to 2015 would indicate.

Figure 8- No. of students passing CSEE annually 20102015

Early grades and the 3Rs (Reading wRiting and aRithmetic)

RTI (with funding from USAID) carried out a statistically robust Early Grade Reading Assessment (EGRA) across Tanzania in 2013 and repeated this exercise in 2016.14 These provide an objectively verifiable measure of the progress which Tanzania made on the 3Rs during the BRN period. The results of the EGRA survey were very encouraging. As Figure 9 demonstrates the proportion of Standard 3 students classed as ‘non-readers’ declined from 28% to just over 16% between 2013 and 2016 whilst the proportion of students classed as ‘progressing readers’ increased from 22% to almost 30% over the same period. These are very significant national level increases observed over a relatively short length of time, representing measurable improvements in reading achievements by hundreds of thousands of learners in Standard 3 alone15

In addition to the decrease in ‘non-readers’, the EGRA study also found that the mean reading speed for mainland Tanzania increased from 17.9 words per minute in 2013 to 23.6 words per minute in 2016.16 What this meant in practice is that a growing

14 USAID and RTI conducted a baseline Early Grade Reading Assessment (EGRA) in October 2013. A follow-up EGRA was carried out in February 2016. It was administered to a total of 7,765 Standard 3 students randomly selected from within 650 schools. The 650 participating schools were also randomly selected to create a sample that would provide data at the national level and for the 25 regions of Tanzania, as well as Zanzibar

15 It should be noted there was only very limited improvements in mathematics achievement levels over this period.

16 Reading speed is an important aspect of comprehension. The 2013 EGRA identified the low mean reading speed in Tanzania as a significant barrier to comprehension and an area which required attention. It is for this reason that it was selected as an EPforR disbursement linked indicator, the increase led to a major release of $36m of donor finance to the government. See blog posting (Attfield 2016).

Figure 9- Proportion of Standard 3 students classed by Kiswahili reading ability in 2013 and 2016 (RTI EGRA)

proportion of the 8 million pupils enrolled in government primary schools are able to read and understand Kiswahili - a critical foundation for their future life chances.

It is undeniable that BRN brought about much greater emphasis across Tanzania on early grade instruction and the 3Rs. Early grade reading, writing and mathematics achievement levels were measured for the first time, the curriculum was simplified (reduction of non-core subjects in early primary), and there were major rounds of training for teachers and release of new materials The 3Rs interventions were developed in collaboration with the Tanzania Institute of Education (TIE), including lowcost, school-based, teacher continuous professional development aligned to the new 3Rs syllabus and the provision of supplementary reading materials. The Government’s commitment to the 3Rs was demonstrated in its focused use of resources from the major Global Partnership for Education financed LANES programme, which was often branded as components of BRN.

The Government of Tanzania benefitted from the presence of two substantial donorfunded education projects which worked in 8 of the country’s 25 Regions and contributed to improvements in early grade reading in this period. The USAID funded TZ21 project and the DFID funded Education Quality Improvement ProgrammeTanzania (EQUIP-T: with an extensive coverage of around 4,500 schools and 2.3 million students) both initiated work on improved early grade pedagogy and reading materials These were extensively used and adapted in the other 17 LANES supported 17 regions, with the BRN ‘brand’ and strong government backing assisting to overcome inertia to changes at the local government and school level

Information Flows, Reporting and Transparency

The Education MDU worked closely with Regions and Districts to introduce a streamlined monthly reporting system focusing on BRN initiatives. This was a simple spreadsheet which was completed and submitted electronically each month. The MDU then compiled and analysed the data for monthly review meetings and also ensured that detailed, individualised feedback on reports was sent to all Districts and Regions.

The quality of reporting improved noticeably month by month and at least 90% of Districts regularly submitted their monthly reports on time. Most importantly this reporting system and the use of email groups meant that District and Regional officials came together to share ideas as to how to tackle common problems. Although popular, the reporting system ceased to operate effectively from mid-2015 onwards, as the perceived importance of the BRN processes fell as political elections approached

In addition to internal improvements in information flow within the education system BRN also had a positive impact on the quantity and quality of publicly available education information. Tanzania experienced significant difficulties in conducting and publishing an Annual School Census in 2014 and 2015, largely because of issues to do with the transition of responsibility from MoEST to PO-RALG which took effect from late

2013 onwards. However the 2016 Annual School Census was carried out in a timely manner and, for the first time, comprehensive school level data was collected and made publicly available on the Government of Tanzania’s Open Data Portalhttp://opendata.go.tz

Whilst these improvements in the Annual School Census took place after BRN had officially finished the impetus and stimulus for change came quite clearly from the EPforR programme and the combination of both (i) technical and data processing support and (ii) a financial disbursement linked indicators for online publishing of the main datasets. School ranking conducted by the examination council NECTA , with results for all schools made available on their website, provided a further level of transparency. Although only a small proportion of the Tanzanian population would be able to access these online results, Regions and Districts took their own initiative to publicise and publish lists of most and least improved schools locally, meaning that parents and communities began to receive more information on exam performance because of BRN.

Resource Mobilisation to Schools

The introduction of a ‘direct to school’ capitation grant system in December 2015 is perhaps the most significant systemic achievement of BRN Education. Whilst the groundwork and analysis for this change was laid in 2014 and 2015 it took the election of President Magufuli in late 2015 to unblock the system and make change happen.

The amount of capitation grant funds reaching schools within 14 days of receipt by Local Government was a key BRN indicator which was reported on a monthly basis. This indicator remained ‘red’ throughout the BRN period due to a combination of:

i.) Inadequate fund releases from the Ministry of Finance and

ii.) The practice of some Districts to ‘sit on’ funds received and not release them to schools promptly, with limited due process to deter alternative fund use.

This situation was reported on monthly basis to the NKRA Steering Committee from late 2013 onwards. Whilst the issue of the amount of funds released by MoF was never adequately addressed, a more efficient distribution system was developed where funds could be released directly into school bank accounts. This system was broadly endorsed ‘in principle’ in mid-2014 and a press release was issued to this effect. However institutional inertia, budgetary constraints and covert opposition meant that by the final quarter of 2015 the new system had yet to be implemented.

This all changed in late 2015 when President Magufuli came to power. ‘Free Education’ was a generic campaign priority and he had made a public commitment to provide adequate government financing and fee free access at primary and secondary level. A plan was presented to the President for the direct to school capitation grant system and by the end of the year all 20,000 government primary and secondary schools were

receiving their full allocation of funds in monthly transfers directly to their bank accounts. Although the amounts were relatively small, this represented a major break from the recent past, in which a sense of distrust and fear of local petty corruption was used to centralise resources and decision making on quality enhancing school inputs.

Enhanced focus on student attainment

One noticeable effect which was identified through a field survey was the increased focus on examination results and pass rates that BRN had brought about at school, district, regional and national levels (Integrity Research, 2016). Respondents felt that this has led to the proliferation of extra classes within primary and secondary schools (the nationally organised BRN ‘STEP’ classes but also many classes organised locally under the initiative of schools and districts) and a focus on the attainment levels of all students.

Whilst some respondents in the Integrity study felt that a pre-occupation with examination results and ‘teaching to the test’ had some negative implications, overall we would argue that this was a positive development considering the situation in 2013. Prior to BRN teacher absenteeism rates were very high (Tanzania’s 2014 SDI Survey (World Bank, 2015) showed encouraging signs that this issue was declining) and many teachers did not pay much attention to individual student performance. Whilst exam results are an imperfect proxy for learning achievement we would argue that: (i) the requirement for teachers to assess the individual performance levels of all students and initiate remedial measures to address gaps and improve pass rates was a positive development and (ii) the instrumental importance of passing examinations is undeniable the world over as a pathway to post-basic education and enhanced job opportunities.

Local Problem Solving and Innovation

One of the most encouraging aspects of BRN was the way in which individual regions, districts and schools displayed their own initiative in coming up with creative ways of addressing BRN priorities. These approaches came to national attention through the monthly BRN reporting system, which also acted as a transmission mechanism to make other districts and regions aware of local innovations which could be adopted. Examples of local initiatives and problem solving include:

- Mbinga District Council which used its own funds to provide tailored training on remedial classes and exam preparations to 1,100 teachers in 150 primary schools.

Mtwara Municipal Council which partnered with Schlumberger to provide training and support to secondary science teachers to boost CSEE performance.

Chato District Council which created its own Special Training Network for 3Rs, whereby each school has a 3Rs coordinator who operates as part of a learning network under the supervision of the Council.

Mbogwe District Council which collaborated with parents and used its own

resources to provide incentives to secondary subject teachers based on grade performance of their students in terminal and mock examinations.

Dodoma Municipal and Mpwapwa District Councils which arranged their own in-house training at school and ward level for primary teachers utilising Teacher Resource Centres (TRCs).

Katavi Regional Education Office which developed an effective means of overseeing and monitoring construction progress across its constituent districts.

The levels of initiative and innovation displayed by Districts and schools in using local resources to help achieve BRN priorities suggests that it may be more effective for national Government to set clear objectives and then allow some autonomy in deciding how best to achieve these, rather than setting tightly controlled activity based plans as per the ‘3 Feet Plan’ approach. Being clear about the destination and allowing more autonomy and initiative as to how Districts reach that destination is arguably a more effective approach in a complex and diverse educational system rather than centrally drafting a tight plan which can quickly become outdated.

Culture Change and Capacity Development

At a national level one of the most notable effects of BRN was to shift conversations between senior officials and Minister to focus on results and outcomes rather than focusing on process and protocols. In the initial months the Minister of Education used to have almost daily conversations with the MDU and relevant Directors to discuss performance data and try to identify why indicators for some BRN activities were not moving fast enough. This helped to expose inadequacies in data, reporting, performance management and national understanding of local delivery issues which would not otherwise have been discussed.

By 2015 it had become accepted practice in the Ministry to scrutinise performance data, look in detail at local variations and track progress against trajectories. This contributed to a shift in mind-sets of officials to focus on what was happening in schools and district offices rather than focusing on internal processes and protocols.

Another significant change brought about through BRN was the development of a group of progressive officials within MoEST and PO-RALG with a new mindset and approach. These officials had a genuine interest in bringing about school level changes to improve educational outcomes and were equipped with the tools and techniques to interrogate data and to influence change. They no longer accepted ‘business as usual’. Although officials with this mindset are definitely still in a minority it is noticeable that they were seized upon by the new Presidential administration as a force for positive change. In early 2017 many of them occupy far more senior and influential roles within Government than they had back in 2013.

Weaknesses and Challenges

The previous chapter has set out some of the undoubted achievements of BRN education but it is also important to identify the weaknesses and challenges which BRN failed to overcome, these are set out in the paragraphs below.

The structural weaknesses of education delivery system remained unaddressed

Perhaps the biggest obstacle to effective BRN implementation was the tension and lack of collaboration between the two leading government actors: the MoEST and PORALG. The separation of roles and responsibilities between a policy setting Ministry based in Dar es Salaam (MoEST) and an implementation Ministry based in Dodoma (PO-RALG) was an accountability vacuum waiting to happen. The ‘grey area’ between policy development and implementation provided an ideal opportunity for officials in both Ministries to blame the other for a lack of progress and results.

It was especially problematic that the initial Education Delivery Lab failed to adequately address the realities of this delivery system - with undue emphasis being placed on the role of the Minister for Education and policy-focused Directors in Dar es Salaam. Attempts were made to address this by seconding PO-RALG staff into the Education MDU and by assigning PO-RALG Directors with named responsibility for BRN initiatives but in many ways the damage had already been done.17 There was an abiding sense across PO-RALG that BRN Education was driven by MoEST and was therefore a MoEST responsibility. Requests for regular performance data exposed significant data and reporting inadequacies. As PO-RALG held the direct reporting relationships with Regions, Districts and Schools, MoEST Directors responded to data requests by passing them on to PO-RALG officials. These were handled in an ad hoc manner: by making phone calls to district officials, rather than through systematic and verifiable channels. The result was multiple and sometimes conflicting figures for BRN indicators which were then used to hold MoEST Directors to account.

The working relationship between MoEST and PO-RALG did improve over time as both Ministries established a more effective working relationship underpinned by improvements in monthly BRN reporting. By mid-2015, when the Education Program for Results (EPforR) TA Team began operating there was a noticeable improvement in the working relationship and understanding between the two Ministries. While this improvement generally continued into 2016 (after President Magufuli had reconfigured PO-RALG so that it reported to the President’s Office rather than the Prime Minister’s

17 Further confusion occurred in late 2013 when PO-RALG created their own multi-sector MDU whose role and relationship with the Education MDU was never formally clarified.

Office) the structural weaknesses of having two Ministries (each with their own set of Ministers and priorities) persisted.

In addition to the coordination issues between MoEST and PO-RALG, there were other systemic weaknesses which remained unaddressed by BRN. Most prominent amongst these was the issue of teacher claims. Clearing the backlog of teacher claims was a BRN priority initiative but the 3 Feet Plans failed to address the root structural causes of this backlog. As a result once funds had been released to clear the original backlog of claims, new outstanding claims continued to accumulate throughout the BRN reporting period. It was not until April 2016 (once BRN had officially ended) that MoEST and PO-RALG (supported by the EPforR TA Team) conducted an in-depth analysis of Tanzania’s teacher claims system and identified the root causes of the persistent backlogs (MoEST & PO-RALG, 2016)18

Consistency of Presidential engagement and willingness to tackle financing gaps

The personal interest and drive of President Kikwete was a major factor in the initial publicity and prioritisation of BRN within government and by stakeholders across the education system. The requirement to report progress on a weekly basis to the President’s Delivery Bureau (PDB) and for the Minister and PS to then be held to account by the President on a monthly basis at the Transformation and Delivery Council (TDC) was a major incentive to deliver results.

The initial TDC meeting had a very positive impact on the Ministry as it engendered a real sense that the President considered education to be a priority and took a personal interest - even to the extent of specifying the content and format of a high profile National Education Week. As BRN delivery extended into 2014 and 2015 it became noticeable that, whilst regular BRN Education meetings continued to take place, TDC meetings were regularly postponed and cancelled. This weakened the sense of downwards accountability between the President’s Office and MoEST.

18 The final report on the ‘Assessment of the Effectiveness of Teacher Claims Processing Systems and Practices in Tanzania’ (MOEST & PO-RALG, 2016) identified 4 main findings, namely:

1. The payment of non-salary claims is not prioritised in some Councils while others have inadequate funds to pay claims.

2. There is a lack of effective and timely sharing of information between Councils and PO-PSM with a weak linkage between the Teachers Service Department in Councils and PO-PSM in executing the latter’s delegated mandate of teacher discipline and promotion.

3. The absence of automated systems for processing leave, transfer and retirement claims generates bureaucracy and leads to delays and inaccuracies.

4. Limited communication with teachers as to the status of their claims means that the majority of them feel demotivated and frustrated

The cancellation and postponement of TDC meetings also contributed to a sense of frustration and powerlessness at the lack of traction on financial matters. In some ways education was in a better situation than other BRN priority sectors due to (i) the availability of funds from the Education Levy (a transaction tax placed on mobile data services and initiated as part of the BRN initiatives) and (ii) the presence of donor funded initiatives (initially GPE and subsequently EPforR). However there were still significant BRN activities which required the support and buy-in from the Ministry of Finance, in particular for fund release for capitation grants and textbook procurement.

The Ministry of Finance was a standing member of the Education NKRA Steering Committee but this failed to resolve issues such as the continued underfunding of the education system through low levels of capitation grant release to Districts (in the 2014/2015 financial year for example only 38% of the budgeted amount was released). While there were evident macro-fiscal constraints, relatively small education budget lines prioritised under BRN were not in reality a priority in the central decision making process. It took the election of President Magufuli in late 2015 to unblock these issues with the Ministry of Finance and ensure that education received the levels of government funding specified in guidelines and the annual budget.

Lack of connection between some BRN activities and improved educational outcomes and absence of focus on falling attendance levels

Whilst some BRN initiatives dealt with self-evidently important aspects of the education system such as ensuring adequate school level financing (capitation grant) and improving the quality of teaching and learning in the early grades (3Rs assessment and training) this was not universally the case.

As an example, evidence of the impact of the School Improvement Toolkit on the quality of leadership and management in primary and secondary schools is limited to say the least. The BRN Lab identified a critical issue- the quality of leadership and management in schools- but its proposed solution (the production and distribution of a simple guide to school leadership to every school in the country) is not supported by any evidence base as to its impact. To be effective such a guide could have been completed by a package of mentoring and support, overseen by District offices and utilise various community of practice approaches. A more nuanced solution was not considered during the Lab, perhaps because of the desire to come up with neat, easily measurable national ‘solutions’ which could be rolled out rapidly nationwide.

So while the BRN reporting system diligently measured the number of schools every month who had received a copy of the ‘School Improvement Toolkit’, this indicator was effectively meaningless in determining whether there had actually been any measurable improvements in leadership and management in government schools.

Limited community engagement and use of ‘bottom-up’ accountability mechanisms

Whilst BRN proved largely effective in harnessing ‘top down’ accountability measures to influence behaviour at school and district levels it did not adequately consider the role of ‘bottom up accountability’ as a driver of change. The impetus to conduct BRN activities and report on progress was transmitted from national government down to regions, districts and schools through formal communication channels. Whilst there was general publicity about BRN transmitted through newspapers, radio and television there was never a concerted effort made to engage communities in bringing about and monitoring school level activities.

The lack of community knowledge of the specifics of BRN activities at a local level was highlighted through field research ( Integrity Research, 2016). Community members could talk in general about BRN’s aim of improving exam pass rates but they were not able to talk in detail about activities which had taken place locally or how their community’s school was performing in relation to others. BRN did increase the amount of information which was publicly available on schools but as this was displayed in government websites it remained inaccessible to most Tanzanian citizens. The EPforR Programme is trying to address this issue, recognising the power of local communities to enforce accountability and bring about changes in behaviour in schools. Methods of achieving this include the involvement of community leaders and simple visual displays of school performance displayed within communities or on school premises.

Uneven willingness and inconsistent incentives to drive change combined with lack of dedicated capacity to drive necessary system improvements

Whilst some staff in Government wholeheartedly ’bought in’ to the BRN approach, this was not universally the case. Indeed there was covert and overt resistance from several staff, some in quite senior positions, within Ministries. On the whole Districtbased staff appeared to embrace BRN initiatives more enthusiastically than some of their national colleagues, perhaps because it increased the attention and resources dedicated to local delivery. Over the period in question it was noticeable that some staff who had been either overtly or covertly hostile to BRN were transferred away from positions of authority within the education sector.

The lack of willingness to embrace BRN was compounded by the fact that the most competent and capable staff within MoEST and PO-RALG were over-burdened with existing activities and were not re-assigned in sufficient numbers to deal with its initiatives.19 As a result some of the progress made during the BRN reporting period, such as the development of an effective monthly reporting system, was not sustained or embedded within existing staff responsibilities.

19 This lack of capacity was exacerbated by the earlier phased transfer of responsibility for basic education from MoEST to PO-RALG in the 2000s, without the consequent transfer of staff. There were not resources to relocate families the 500km to Dodoma. Large scale transfer of staff from MoEST in Dar es Salaam to PO-RALG in Dodoma mitigated this situation in 2016.

It can be argued that the Education PforR programme and its provision of financial rewards linked to the achievement of specific tasks by national and district staff proved a more effective incentive than BRN in addressing critical issues such as EMIS and equitable teacher deployment.

Key Lessons Learnt and Conclusions

Governance Theory and Practice in BRN

The delivery approach that emerged in Tanzania in 2013 - 2016; and the actors, institutions and financial instrument it encompasses have subsequently evolved with very different pathways to those originally prescribed in the convening BRN labs held in 2013. Formulaic, ‘3 feet’ detailed education sector delivery plans in Gantt chart format, a hierarchical set of Delivery Units and oversight structures reaching up to the presidential level and aligned state budgeting all struggled for traction in the government and political systems of Tanzania. Was the Malaysian tiger model agile enough and able to adapt to the Tanzanian context?

It is reflective to critique the BRN outcomes with contemporary work on flexible and adaptive programming. Broad calls for good governance or inclusive institutions may well miss the point and a more explicit recognition of political conditions is needed to enable development progress and the adaptability to sustain this progress (Booth & Unsworth 2014, Wild et al. 2015). Clear structural weakness and limitations of BRN’s actual approach in engaging Tanzania’s education system are evident. There was almost no attempt to adjust and course correct the ‘3 feet’ plans that rapidly become out of date, the MDU was emasculated in between two competing line ministries and a parallel reporting structure that was ignored as political elections loomed and a budget crisis limited resource allocation to priority programmes.

The World Development Report 2017: Governance and the Law (World Bank 2017) proposes three principles for rethinking governance for development (set out above) These will be expanded in the forthcoming WDR 2018, which will be themed: Realizing the Promise for Education for Development and seek to guide on how education systems can be reformed. At the heart of this philosophy is the premise that tangible and measurable student learning outcomes are the key, primary outcome of a successful education system and other secondary outcomes and structural factors

embedded in traditional hierarchical, centralised systems frequently distort reforms and inhibit progress. (Pritchett 2013)

The BRN education programme did indeed place student learning as the highest level targets and goals, seeking large and sustained short term improvement in primary and lower secondary school completion examinations and introducing a requirement for a national assessment of early primary grade literacy and numeracy skills. Whilst the examination pass rates were raised in successive years from the nadir reached in 2012, it can be argued that this was in part due to changes to the examination systems and the filtering out of less capable student who were entered to take the exams.

By contrast the early grade ‘3R’ assessment did provide impetus to reform early primary curricula and teaching practice, with additional financial incentives offered by the Education Programme for Results (EdPforR) disbursement linked indicator that rewarded improvements in average oral reading fluency from the 2013 baseline. The consequent measurable improvements in reading speed and comprehension stands as the most tangible achievement of the BRN period.

BRN and alignment with the WDR 2017 three governance principles

1/ Function over form

The dimensions of commitment, co-ordination and cooperation are cited as core functions of effective institutions, with form (structure) mattering less;- in particular if it is transplanted in from elsewhere or exhibits isomorphic mimicry (Andrews, 2012)

Despite some cynicism in the media, BRN was notably successful in generating a degree of commitment amongst key education actors including local government managers, principals and teachers towards improving learning outcomes. Whilst there was initially confusion as to what exactly should be done, commissioned in-depth research around the BRN school ranking initiative (based on exam performance) did generate a reaction. Stakeholders had limited understanding of the ranking nor agreed with its exam focus, but did express positive views about the concept. It did appear to have a ‘motivation effect’ particularly if performance had declined (Integrity Research, 2016).

Integrity stakeholder citation

In terms of co-ordination and co-operation, whilst there were strong attempt to impose these, through the creation of the Ministerial Delivery Unit, it was not able to operate effectively. In particular the overlapping roles of the Education Ministry (policy and planning, post basic education based Dar es Salaam) and PO-RALG (co-ordination and management of schools and resource allocation, based in Dodoma) led to numerous problems. The lack of co-operation between some senior staff largely thwarted the MDU from ever becoming functional in Dodoma. Co-operation on data was also difficult with parallel BRN specific reporting structures being established, whilst both existing and new data collection systems were not effectively used or rolled out. It was not until the changes of leadership following political elections in 2015 that some of these issues were resolved. It could be argued that these were systemic constraints on all aspects of basic education management, and not just the high priorities activities identified for BRN to take forward.

2/ Capacity and power asymmetries The high level political authority established by Presidential convening and executive power was an essential component in BRN’s drive to bring about meaningful change in the education system. The initial 6-week Lab process that blended government, private sector, donor, CSO and management consultancy approaches was effective in breaking down some of the invisible barriers associated with a highly bureaucratic public administration system.

The initial perceived influence of the MDU and the injection of fresh technical capacity overcame some of the institutional inertia within the education system and provided a challenge to the status quo. Years of donor budget support had evolved complex dialogue structures that were ineffective and disguised education’s systemic weaknesses - it had expanded so rapidly in the 2000’s it was unable to offer even a minimal level of quality in many schools.

The technical assistance (TA) provided to the government to enable the education BRN process was initially resisted by some actors and the decision was taken to embed it in a unit that supported the MoEST education planning department. This partly reflected a challenge to the power of the MDU and BRN structures from some senior officials. The TA team worked pro-actively to support and enable BRN reporting, rapidly diagnose implementation bottlenecks (for example unpaid teacher allowance claims) and to communicate effectively the BRN rationale, activities and incentives to local government officials. Over time they were able to build confidence and trust, especially during the election transition period when a real space for reform opened.

The issue of influential group elite capture and clientelism was evident in obstacles to providing government budget resources to BRN priorities, most notably direct financial transfer to schools. Over a decade earlier, a system to transfer funds for school managed quality inputs was established; ‘capitation grants’ using an enrolment formula

with a fixed unit cost. Despite support and regular budgeting from the Ministry of Education, other central level actors effectively blocked its operation through devaluation, opaque and non-transparent two tier fund flows and guidelines that ignored the principle of local grant determination with centralised book purchases that rarely actually happened. This was in direct contravention of local official, union, teacher and community wishes, with a weak justification around local fiduciary risk and macro-economic factors.

Sustained pressure and analytical work to break this impasse unexpectedly came about through direct mandate of the new leadership following the 2015 presidential elections. Within weeks of taking office, monthly direct fund transfers to almost 20,000 schools were established using existing school bank accounts.

3/ Rules and the Role of Law The BRN reform priorities did touch upon a number of areas in which legal or operational guidelines were identified as major constraints and faced considerable resistance and capacity gaps to adjust them. The example above of ‘capitation grants’ is pertinent, with highly regressive funding decisions being made through executive guidelines that circumvented the original policy intent of financing levels and local determination of resource use.

Inefficiencies related to a dysfunctional teacher payroll and management systems were also a major reform initiative that the BRN prioritised, seeking to raise motivation through the settlement of a very large backlog of claims for allowances and promotions. Analysis supported by the TA team was able to clearly identify potential reforms that were considered likely to address ineffective working practices: a survey found 47% of teachers were found to absent from classrooms. (World Bank, 2014).

While this was an area where additional BRN linked finance was made available, local government officials did not verify teacher claims effectively and inflated requests were so extreme that senior central government had to issue sanctions and order repeats of verification exercises. Whilst eventually most of the originally identified backlog were cleared, the legal and financial guidelines governing teacher remuneration have not to date been reformed: teacher transfers or promotions can be approved without funds being available to meet additional costs and the highly centralised payroll management systems is misaligned to the local government cadre needed to effectively implement HR change management.

Public sector reforms to a highly centralised and unionised teacher workforce remain perhaps the most critical element which still needs to be addressed in order to achieve the norm of motivated, trained teachers present and teaching effectively in classrooms (MoEST & PO-RALG, 2016). The technical steps to reform, evolve and improve staff management are largely identified and feasible. However the rules regarding actors including central, local government and school staff are opaque and contradictory and overlapping guidelines inhibit clear roles and responsibilities that the BRN initiative has

only started to address.

Following the political transition in late 2015, there has been a major push from the incoming President Magufuli on the rule of law, with major pushes in particular to eliminate ghost workers from the civil service and deal with tax evasion. It will be more challenging for these measures to bring about changes in the education system, not least tackling absenteeism rates from classrooms while teachers are present at the school campus.

Big Results Now! Education lessons for Africa

The importance of education system performance management with sovereign, country led reform programmes, driven by data and a sharp focus on measurable learning outcomes is at the heart of The Learning Generation’s call to action (Education Commission, 2016). However the potential to directly export and replicate such approaches is not a given: ‘Investing in what works is not as simple as finding a reform that has been proven to work elsewhere and importing it…. Decision-makers must carefully consider whether a given reform or intervention addresses the specific needs of a given system and whether it is implementable in the institutional context’ (Pritchett 2013).

The Education Commission have followed up this agenda with a search for ‘pioneer countries’ with the political leadership and drive to move forward down a similar road of education system and domestic financing reform, with external actors and finance to support in a form of compact. The BRN Tanzania education model has been highlighted as a possible example to learn from by the Education Commission, with former President Kikwete one of the Commissioners promoting the potential to perspective pioneer country leaders.

Conclusion

This paper has demonstrated the critical need for flexibility and awareness of the likely space and pace needed to deliver complex education sector reforms. Tanzania’s context of a developing African country, struggling with the competing demands for rapid improvements in both access to education and meaningful learning outcomes is not unusual.

Several successful attributes of the Delivery Approach were clearly in place: the political convening and innovative stakeholder lab process to identify priorities, action plans, results compacts and wide public focused communications on learning and performance. As this critique of common governance principles and adaptive approaches has shown it is very difficult to avoid the many existing political economy constraints that exist. Whilst a robust, short cycle iterative monitoring and course correction approach has only partially been followed, there have been some significant

adaptations: in particular around early primary curriculum and a reorientation of teaching practice that has seen significant gains in average kiSwahili oral reading fluency and a drop in complete non-readers in primary school students.

Despite the dropping of the BRN branding and disbandment of the MDU and other BRN structures by the incoming administration in 2016, the ethos and confidence of education sector practitioners within Tanzania has continued to build. The three donor, Education Programme for Results (EPforR) financing framework continues to offer a range of major financial incentives. A revised set of disbursement linked indicators were jointly agreed at the programme’s mid-term, partly to reflect the cessation of the formal BRN Education plan in 2016. These were negotiated with a much greater sense of government buy-in and ownership of the financing instrument than had previously existed, and a growing capacity to manage results based delivery in the Tanzanian education sector.

References

Andrews, M. (2013) The Limits of Institutional Reform in Development

Andrews, M., Pritchett, L. & Woolcock, M. (2012) ‘Escaping Capability Traps through Problem-Driven Iterative Adaptation (PDIA)’, Center for Global Development Working Paper 299

Attfield, I. (2016), ‘Incentivising national scale increases in reading skills in Tanzania’ http://www.heart-resources.org/blog/incentivising-national-scale-increases-readingskills-tanzania/

Barber, M. (2013) ‘The good news from Pakistan: How a revolutionary new approach to education reform in Punjab shows the way forward for Pakistan and development aid everywhere’, Reform

Booth, D. & Unsworth, S. (2014) ‘Politically smart, locally led development’, ODI Discussion Paper, London.

Hymowitz, D. (2016)’Too much science, not enough art’, Tony Blair Africa Governance Initiative

Integrity Research (2016) An assessment of attitudes towards the implementation of the School Ranking Initiative (SRI) as part of the Big Results Now in Education Programme.

International Commission on Financing Global Education Opportunity (2016) The Learning Generation: Investing in education for a changing world

MoEST & PO-RALG (2016) ‘Assessment of the Effectiveness of Teacher Claims Processing Systems and Practices in Tanzania’

Pritchett, L. (2013a) The rebirth of education: Schooling ain’t learning, Washington DC, Centre for Global Development.

Pritchett, L. (2013b) ‘The rebirth of education- why schooling in developing countries is failing; how the developed world is complicit; and what to do next’ Centre for Global Development Brief, 1-4.

RTI (2016) Early Grade Reading Assessment (EGRA) Report for Tanzania

Todd, R.; Martin, J. & Brock, A. (2014) ‘Delivery Units: can they catalyse sustained improvements in education service delivery?’

Wild, Booth, Cummings, Foresti & Wales (2015) ‘Adapting development: improving services to the poor’ ODI Politics and Governance Programme

World Bank (2015) Tanzania 2014 Service Delivery Indicators (SDI) Education Technical Report.

World Bank (2017) World Development Report 2017: Governance and the Law