Achievements and Results

Examination Results

When the BRN education plan ended in 2016, had it achieved the objectives and results set out when it was launched in April 2013? The headline aim of BRN education was to improve pass rates for the Primary School Leaving Examination (PSLE) and the Certificate of Secondary Education Examination (CSEE).

In 2012 national PSLE pass rates had fallen to 31%. Following the introduction of the BRN Education NKRA in 2013 there was a rapid improvement in PSLE pass rates to 50.6% in 2013, 57.0% in 2014 and 67.8% in 2015. This may be behind the BRN target of 80% PSLE pass rates in 2015 but it still represents significant progress since 2012. However, as the graph below shows, the undoubted progress in pass rates achieved under BRN is less impressive if looked at in wider historical context. If the starting point for analysis is taken as 2005 rather than 2012 progress appears steady rather than spectacular. Comparability between years in terms of examination difficulty is also questionable and may account for some of the large variations; it was determined not to be a robust enough measure of learning outcomes, to be linked to the Education Payment for Results financing triggers.

Pass rates themselves can also be misleading as they don’t give an indication of the number of students taking the examinations. Tanzania has a rapidly growing population so can we make the assumption that growing enrolments and the consequent strain on the education system have made it difficult to maintain pass rates?This was a key aspect of the narrative formed during the BRN Education Lab. However looking at the data over the past few years this argument fails to hold up to scrutiny. Firstly, using data from the Education Lab itself primary enrolments in Tanzania peaked in 2009 (unlike secondary which continued to display rapid growth) as Figure 5 shows.

Analysing the number of candidates sitting PSLE annually between 2012 and 2015 there was a steady drop year on year- from 865,534 candidates in 2012 to 763,602 in 2015almost 100,000 fewer candidates in a period of just three years. Consequently whilst the absolute number of candidates passing PSLE increased each year over the BRN period the increase was not as spectacular as the percentage pass rate figures would indicate. Still, it is important to note that the 518,034 candidates who passed PSLE in 2015 was almost double the 265,873 who passed in 2012- undeniably this represents impressive progress.

Figure 6- Annual number of PSLE Candidates

At secondary level national ‘Form 4’ CSEE pass rates had fallen to 43% in 2012 driven in large part by extremely rapid increases in enrolment since 2006. Progress in CSEE pass rates showed significant improvement since the introduction of BRN with it increasing to 57.1% in 2013 and then 69.8% in 2014, before falling slightly to 68.5% in 2015. This meant that the 2015 CSEE pass rate was 11 percentage points behind the aspirational BRN target of 80%. It is also very important to note that the methodology for marking CSEE changed in 2014, thus weakening the comparability of results year on year.

What about the actual number of candidates sitting and passing CSEE each year?

There was a significant drop in the number of candidates sitting the national CSEE at the end of Form Four in 2014, largely because of the introduction of the requirement to pass exams at the end of Form Two in order to progress to Form Three. This policy was introduced prior to BRN and had a significant impact on the 2014 figures. The actual number of candidates writing CSEE in 2014 was 240,300 compared with 352,600 in 2013. This means that, while the overall pass rate increased significantly (from 57% in 2013 to almost 70% in 2014) the actual number of candidates passing CSEE declined from just over 200,000 in 2013 to 168,000 in 2014. When viewed in terms of actual number of students passing CSEE 2015 was by far the most impressive year. The actual number of passes in 2015 was 243,200 as the overall number of candidates sitting the exam rebounded to 336,800. The total number of students passing CSEE annually between 2010 and 2015 is set out in Figure 8 below.

Interestingly both PSLE and CSEE pass rates ended up looking very similar in 201567.8% for PSLE and 68.5% for CSEE. Whilst these results are below the aspirational BRN targets of 80% in both cases they represent significantly more young people passing examinations than had been the case in 2012. Overall therefore BRN is thought to have had a positive impact on PSLE and CSEE results albeit a more nuanced success than a simple percentage pass rate snapshot from 2012 to 2015 would indicate.

Early grades and the 3Rs (Reading wRiting and aRithmetic)

RTI (with funding from USAID) carried out a statistically robust Early Grade Reading Assessment (EGRA) across Tanzania in 2013 and repeated this exercise in 2016.14 These provide an objectively verifiable measure of the progress which Tanzania made on the 3Rs during the BRN period. The results of the EGRA survey were very encouraging. As Figure 9 demonstrates the proportion of Standard 3 students classed as ‘non-readers’ declined from 28% to just over 16% between 2013 and 2016 whilst the proportion of students classed as ‘progressing readers’ increased from 22% to almost 30% over the same period. These are very significant national level increases observed over a relatively short length of time, representing measurable improvements in reading achievements by hundreds of thousands of learners in Standard 3 alone15

In addition to the decrease in ‘non-readers’, the EGRA study also found that the mean reading speed for mainland Tanzania increased from 17.9 words per minute in 2013 to 23.6 words per minute in 2016.16 What this meant in practice is that a growing

14 USAID and RTI conducted a baseline Early Grade Reading Assessment (EGRA) in October 2013. A follow-up EGRA was carried out in February 2016. It was administered to a total of 7,765 Standard 3 students randomly selected from within 650 schools. The 650 participating schools were also randomly selected to create a sample that would provide data at the national level and for the 25 regions of Tanzania, as well as Zanzibar proportion of the 8 million pupils enrolled in government primary schools are able to read and understand Kiswahili - a critical foundation for their future life chances.

15 It should be noted there was only very limited improvements in mathematics achievement levels over this period.

16 Reading speed is an important aspect of comprehension. The 2013 EGRA identified the low mean reading speed in Tanzania as a significant barrier to comprehension and an area which required attention. It is for this reason that it was selected as an EPforR disbursement linked indicator, the increase led to a major release of $36m of donor finance to the government. See blog posting (Attfield 2016).

It is undeniable that BRN brought about much greater emphasis across Tanzania on early grade instruction and the 3Rs. Early grade reading, writing and mathematics achievement levels were measured for the first time, the curriculum was simplified (reduction of non-core subjects in early primary), and there were major rounds of training for teachers and release of new materials The 3Rs interventions were developed in collaboration with the Tanzania Institute of Education (TIE), including lowcost, school-based, teacher continuous professional development aligned to the new 3Rs syllabus and the provision of supplementary reading materials. The Government’s commitment to the 3Rs was demonstrated in its focused use of resources from the major Global Partnership for Education financed LANES programme, which was often branded as components of BRN.

The Government of Tanzania benefitted from the presence of two substantial donorfunded education projects which worked in 8 of the country’s 25 Regions and contributed to improvements in early grade reading in this period. The USAID funded TZ21 project and the DFID funded Education Quality Improvement ProgrammeTanzania (EQUIP-T: with an extensive coverage of around 4,500 schools and 2.3 million students) both initiated work on improved early grade pedagogy and reading materials These were extensively used and adapted in the other 17 LANES supported 17 regions, with the BRN ‘brand’ and strong government backing assisting to overcome inertia to changes at the local government and school level

Information Flows, Reporting and Transparency

The Education MDU worked closely with Regions and Districts to introduce a streamlined monthly reporting system focusing on BRN initiatives. This was a simple spreadsheet which was completed and submitted electronically each month. The MDU then compiled and analysed the data for monthly review meetings and also ensured that detailed, individualised feedback on reports was sent to all Districts and Regions.

The quality of reporting improved noticeably month by month and at least 90% of Districts regularly submitted their monthly reports on time. Most importantly this reporting system and the use of email groups meant that District and Regional officials came together to share ideas as to how to tackle common problems. Although popular, the reporting system ceased to operate effectively from mid-2015 onwards, as the perceived importance of the BRN processes fell as political elections approached

In addition to internal improvements in information flow within the education system BRN also had a positive impact on the quantity and quality of publicly available education information. Tanzania experienced significant difficulties in conducting and publishing an Annual School Census in 2014 and 2015, largely because of issues to do with the transition of responsibility from MoEST to PO-RALG which took effect from late

2013 onwards. However the 2016 Annual School Census was carried out in a timely manner and, for the first time, comprehensive school level data was collected and made publicly available on the Government of Tanzania’s Open Data Portalhttp://opendata.go.tz

Whilst these improvements in the Annual School Census took place after BRN had officially finished the impetus and stimulus for change came quite clearly from the EPforR programme and the combination of both (i) technical and data processing support and (ii) a financial disbursement linked indicators for online publishing of the main datasets. School ranking conducted by the examination council NECTA , with results for all schools made available on their website, provided a further level of transparency. Although only a small proportion of the Tanzanian population would be able to access these online results, Regions and Districts took their own initiative to publicise and publish lists of most and least improved schools locally, meaning that parents and communities began to receive more information on exam performance because of BRN.

Resource Mobilisation to Schools

The introduction of a ‘direct to school’ capitation grant system in December 2015 is perhaps the most significant systemic achievement of BRN Education. Whilst the groundwork and analysis for this change was laid in 2014 and 2015 it took the election of President Magufuli in late 2015 to unblock the system and make change happen.

The amount of capitation grant funds reaching schools within 14 days of receipt by Local Government was a key BRN indicator which was reported on a monthly basis. This indicator remained ‘red’ throughout the BRN period due to a combination of: i.) Inadequate fund releases from the Ministry of Finance and ii.) The practice of some Districts to ‘sit on’ funds received and not release them to schools promptly, with limited due process to deter alternative fund use.

This situation was reported on monthly basis to the NKRA Steering Committee from late 2013 onwards. Whilst the issue of the amount of funds released by MoF was never adequately addressed, a more efficient distribution system was developed where funds could be released directly into school bank accounts. This system was broadly endorsed ‘in principle’ in mid-2014 and a press release was issued to this effect. However institutional inertia, budgetary constraints and covert opposition meant that by the final quarter of 2015 the new system had yet to be implemented.

This all changed in late 2015 when President Magufuli came to power. ‘Free Education’ was a generic campaign priority and he had made a public commitment to provide adequate government financing and fee free access at primary and secondary level. A plan was presented to the President for the direct to school capitation grant system and by the end of the year all 20,000 government primary and secondary schools were receiving their full allocation of funds in monthly transfers directly to their bank accounts. Although the amounts were relatively small, this represented a major break from the recent past, in which a sense of distrust and fear of local petty corruption was used to centralise resources and decision making on quality enhancing school inputs.

Enhanced focus on student attainment

One noticeable effect which was identified through a field survey was the increased focus on examination results and pass rates that BRN had brought about at school, district, regional and national levels (Integrity Research, 2016). Respondents felt that this has led to the proliferation of extra classes within primary and secondary schools (the nationally organised BRN ‘STEP’ classes but also many classes organised locally under the initiative of schools and districts) and a focus on the attainment levels of all students.

Whilst some respondents in the Integrity study felt that a pre-occupation with examination results and ‘teaching to the test’ had some negative implications, overall we would argue that this was a positive development considering the situation in 2013. Prior to BRN teacher absenteeism rates were very high (Tanzania’s 2014 SDI Survey (World Bank, 2015) showed encouraging signs that this issue was declining) and many teachers did not pay much attention to individual student performance. Whilst exam results are an imperfect proxy for learning achievement we would argue that: (i) the requirement for teachers to assess the individual performance levels of all students and initiate remedial measures to address gaps and improve pass rates was a positive development and (ii) the instrumental importance of passing examinations is undeniable the world over as a pathway to post-basic education and enhanced job opportunities.

Local Problem Solving and Innovation

One of the most encouraging aspects of BRN was the way in which individual regions, districts and schools displayed their own initiative in coming up with creative ways of addressing BRN priorities. These approaches came to national attention through the monthly BRN reporting system, which also acted as a transmission mechanism to make other districts and regions aware of local innovations which could be adopted. Examples of local initiatives and problem solving include:

- Mbinga District Council which used its own funds to provide tailored training on remedial classes and exam preparations to 1,100 teachers in 150 primary schools.

Mtwara Municipal Council which partnered with Schlumberger to provide training and support to secondary science teachers to boost CSEE performance.

Chato District Council which created its own Special Training Network for 3Rs, whereby each school has a 3Rs coordinator who operates as part of a learning network under the supervision of the Council.

Mbogwe District Council which collaborated with parents and used its own resources to provide incentives to secondary subject teachers based on grade performance of their students in terminal and mock examinations.

Dodoma Municipal and Mpwapwa District Councils which arranged their own in-house training at school and ward level for primary teachers utilising Teacher Resource Centres (TRCs).

Katavi Regional Education Office which developed an effective means of overseeing and monitoring construction progress across its constituent districts.

The levels of initiative and innovation displayed by Districts and schools in using local resources to help achieve BRN priorities suggests that it may be more effective for national Government to set clear objectives and then allow some autonomy in deciding how best to achieve these, rather than setting tightly controlled activity based plans as per the ‘3 Feet Plan’ approach. Being clear about the destination and allowing more autonomy and initiative as to how Districts reach that destination is arguably a more effective approach in a complex and diverse educational system rather than centrally drafting a tight plan which can quickly become outdated.

Culture Change and Capacity Development

At a national level one of the most notable effects of BRN was to shift conversations between senior officials and Minister to focus on results and outcomes rather than focusing on process and protocols. In the initial months the Minister of Education used to have almost daily conversations with the MDU and relevant Directors to discuss performance data and try to identify why indicators for some BRN activities were not moving fast enough. This helped to expose inadequacies in data, reporting, performance management and national understanding of local delivery issues which would not otherwise have been discussed.

By 2015 it had become accepted practice in the Ministry to scrutinise performance data, look in detail at local variations and track progress against trajectories. This contributed to a shift in mind-sets of officials to focus on what was happening in schools and district offices rather than focusing on internal processes and protocols.

Another significant change brought about through BRN was the development of a group of progressive officials within MoEST and PO-RALG with a new mindset and approach. These officials had a genuine interest in bringing about school level changes to improve educational outcomes and were equipped with the tools and techniques to interrogate data and to influence change. They no longer accepted ‘business as usual’. Although officials with this mindset are definitely still in a minority it is noticeable that they were seized upon by the new Presidential administration as a force for positive change. In early 2017 many of them occupy far more senior and influential roles within Government than they had back in 2013.