After School Tutoring: Evidence from a Randomized Controlled Trial Presenter: Ummul Ruthbah Co-author: Asad Islam Extended Abstract Numerous initiatives to improve educational outcomes all around the world have been taken in last few decades, but low levels of learning in most developing countries continue to remain a persistent barrier (Prichett, 2004; Uzewo, 2012). There are a number of challenges such as inadequate infrastructure and resources to facilitate learning, high cost of hiring teachers relative to small budgets, and the reluctance of qualified teachers to work in the rural areas where the needs are the greatest (Muralidharan and Sundararaman 2013). In many countries, parents often feel the need to supplement school-based education with private tutoring. Private tutoring now coexists with mainstream schooling and mimics the regular school system (Bray and Lykins, 2012). 1 Policy makers and educational experts are also debating the pertinence of private tutoring, and whether it should be allowed to coexist with traditional system of education.2 An academically weak child might fall behind at school, and hence might need more individual attention, which can be provided by private tutors. Many parents consider the level of instruction delivered at school not adequate and they resort to private tutoring to achieve acceptable academic standards (Kim and Lee 2010). In addition, because parents in developing countries are less educated, they may be less able to help their children with homework and need to rely on outside tutoring instead.
1 Substantial private tutoring industries can be found in countries as economically and geographically diverse as Cambodia, Egypt, Japan, Kenya, Morocco, Romania, Singapore, the USA, and the UK (Dang and Rogers 2008), and India. In Japan, 70% of students receive private tutoring by the time they complete middle school; 83% students in Malaysia receive tutoring by the time they reach senior secondary school (Bray 2007; Dang and Rogers 2008; Bray 2011). 83.1% of primary school students, 92.8% of middle school students, and 87.8% of high school students in South Korea attend private tuitions (Kim and Lee 2010). In Sri Lanka, 80% of sixth graders and 75% of eleventh graders take tutoring classes (Bray 2006). Over 50% Nepali secondary school students take tutoring classes (Jayachandran 2014). Supplementary private tutoring is also widely practiced in developed countries. 2 There are also arguments that private tutoring can result from corruption in the education system in some developing countries, where teachers require their students to go to their extra classes to supplement their income (Jayachandran 2014; Emran, Islam and Shilpi 2018).
Despite a large number of studies on education using randomized controlled trials (RCTs), the evidence on effectiveness of tutoring using RCTs is still missing, and is largely based on observational studies. Jayachandran (2014) uses survey and test score data from secondaryschool students in Nepal to assess the effects of tutoring. She finds adverse effects on students’ performance particularly among poorer children. Azam (2016), on the other hand, finds positive effects associated with private tutoring in India. Berry and Mukherjee (2019) uses a field experiment to estimate demand for private tutoring in urban India. Using the experimental variation in prices as an instrument for taking up tutoring, they find that private tutoring has no impact on the average test scores of the students. There are, however, a number of related studies evaluating the effects of supplementary teaching by contract teachers. 3 In contrast to private tutors, contract teachers work much closely with school teachers, and students are removed from their classrooms to study under the contract teachers. Banerjee et al. (2007) evaluated an intervention using contract teachers, called remedial education program, where young female teachers were hired to teach basic numeracy and literacy skills for underperforming children in India. They found that the average test scores of these students raised by 0.28 SD. Muralidharan and Sundararaman (2013) find that students in schools with a contract teacher perform better by 0.15 SD. Students from more rural areas benefitted more from these contract teachers. The study attributed the effects of the contract teacher to a reduction in pupil-teacher ratio, class size, multiple-grade teaching, and teacher absenteeism.
Duflo, Dupas and Kremer (2012) also found that contract teachers raised test scores in government schools in Kenya by 0.21 SD, compared to being taught by teachers appointed by government. Bold et al. (2018) report positive results in schools where a NGOs administered the contract teacher program. In contrast, they find that, in schools that received contract teachers from the Kenyan government, effects were close to zero, due to political pressures. These results indicate that having a weak institutional environment plays a part in the effectiveness of a program. Duflo and Kiessel (2012) also report results from a similar intervention in Ghana where students were taught after school hours by young female contract teachers who were high school graduates. They find an increase of 6.2% in the test scores of
3
These contract teachers are usually hired on fixed-term renewable contracts, generally less qualified than the regular school teachers, and are paid much lower wages.
the treated students. 4 Muralidharan, Singh and Ganimian (2018) conduct an RCT of an afterschool program for secondary school students in India using computerized instruction. They use teaching software designed to find what level a child is at and instruct them at that level. The results, after just four and half months of intervention, show students test scores were 0.37 SD higher in math and 0.23 SD higher in Hindi. The results suggest that teaching at the right level (TaRL) might be an effective solution considering that technology such as computeraided learning is still remote possibility in many parts of the developing world, particularly in rural areas. Indeed, TaRL has now been implemented by a number of countries in Africa and India providing instructions to the level of the child ability. In India, for example, the NGO Pratham in collaboration with J-PAL has implemented different TaRL models in many settings. 5 Banerjee et al (2016) evaluated a series of TaRL interventions across different states in India. The results show remarkable level of learning gains with students in Uttar Pradesh improving their test scores the highest by 0.7 SD, after 40 days of teaching outside school hours were provided to the students. In this paper, we provide evidence of a free after-school private tutoring among remote rural schools in Bangladesh. Our interventions include traditional tutoring by tutors outside school hours in a group. We also examine effect when targeting students individually (as in TaRL) to teach at their level of ability by offering them a free weekly one-to-one individualized session between a tutor and a student. In contrast to the intervention using contract teachers, our intervention did not reduce class sizes, neither did it provide additional teachers for the classroom. The tutors in our intervention were not incentivized by any promises of job security to perform better. Existing school teachers were also not allowed to do tutoring, thus addressing the concern that class teachers might teach less to offer tutoring outside school.
We implement the RCTs in 215 schools in two districts over two years. The program was available for students who were in grade 3 and in grade 4 in 2014 (year 1 of the intervention). In the second year, these same students were offered the free tuition. The RCTs involve a two-stage design where in the first stage schools were assigned randomly into either
4
Contract teachers have also been used in many developing countries including Cambodia, Cameroon, Indonesia, Guinea, many Indian states and many other African countries (Duthilleul 2005). There are arguments and findings that contract teachers are under-qualified and untrained, paying them low salaries are exploitative, and that learning outcomes are insignificant when applied in a larger context (De Laat and Vegas 2003; Bold et al. 2018). 5 See https://www.teachingattherightlevel.org/ for details of the evaluation and the results using TaRL in different contexts using different models. Earlier studies in India and Africa using TaRL include Banerjee et al. (2007) and Duflo, Dupas and Kremer (2012).
treatment or control schools. The second stage involves randomly selecting students from each grade within a school and offer them free tuition.
There were 117 treatment schools which were further divided randomly into two groups: (1) 58 schools with private tutoring (PT only from now on) schools where a tutor was hired to teach students in a group of 10-12 students in 3 days of the week; (2) 59 schools which include home visit along with private tutoring. Home visit was done once a week by tutors at students’ home (HPT treatment from now on). In PT only treatment, tutors provided two hours of teaching in group/batch. In HPT, in addition to treatment as in PT, tutors spent an hour in each week. The purpose of the home visit was to provide individual session and teaching at the students’ level. It also ensured more interactions between tutors and parents. Thus, the intervention provided private tutoring in both group settings as well as a one-to-one weekly interaction at children’s home. After two years of the program, we find that students in the treatment schools performed significantly better. Test scores of students in PT schools are 0.3-0.4 SD higher depending on students’ grades and subjects (Math and English). The effect size was almost double for the HPT group. We find that the untreated students in the treatment schools also increased in test scores, suggesting significant spillover effects. The effects also persisted one year after the completion of the program- students in treatment schools particularly those belong to HPT school- do continue to perform better. Our findings suggest that parents of students who were not offered free tuition also put more efforts in their children’s education, suggesting increased competition within students in treatment schools. The results suggest that the program did not cause any substitution of effort or expenditure on the treated children’s education, but did increase household involvement in education for all the students in treated schools.