Impact Evaluation of Education Quality Improvement Programme in Tanzania: Endline Quantitative Technical Report, Volume II

interviews conducted over the phone, number of pupils being replaced, etc. These indicators were constructed in a Stata do-file that ran on the latest cleaned dataset and was then uploaded onto the dashboard (that was created using the visual software, Power BI) that would break down each of the indicators by team, enumerator (where applicable) and week of data collection. This was reviewed on a daily basis by the fieldwork management team and used to feedback to weaker teams and to improve performance.

D.6.5 Field visits by fieldwork management team including back-checking of data

The quality assurance protocol involved visits by the fieldwork management team to the field as well as data back-checks. Two members of the fieldwork management team visited a number of schools and households across 8 of the 12 regions over a two-week period. The purpose was to verify that the school and household interviews were conducted properly, to collect any missing information from these schools or clarify certain issues that were flagged as errors during the daily checking process, and to hold debriefs and retraining sessions with the teams in the field.

D.6.6 Integration of Analysis and Survey Team

Another central element of quality assurance was the strong integration of the fieldwork management team and members of the quantitative analysis team, including the overall project manager. Members of both teams were involved in the fieldwork preparation and implementation, and in the analysis process which followed.

D.7 Fieldwork challenges

The EQUIP-T endline fieldwork incorporated many of the lessons that were learned from the midline experience such as reporting physically to all regions and districts to obtain permit letters and doing so early on, recruiting more enumerators for the training, training all trainees on the pupil test, and having a training facilitator. Yet the endline fieldwork still faced some challenges, most important of which are:

 The fieldwork window between the end of the mid-term break and the start of the exams at the end of the term is narrow. A few alternative options were assessed in December 2017 but it was decided that it was best to stick with the same time window as at baseline and midline to ensure results are comparable. In order to address the narrow window, one extra team of six enumerators was added to the model that balanced between survey being completed on time and having a manageable number of trainees. Furthermore, towards the end of the fieldwork larger schools were prioritised for completion as these were the most likely to be busy towards the end with exam preparations.

 Unscheduled school events on the day of the survey. In order to address this, as at midline, teams contacted the DEOs and head teachers one week in advance (to explore and confirm school timetables and accessibility “in the next month”). The teams did not disclose precisely the date of the visit to avoid schools “preparing” for the visit and to not distort the data collected on teacher attendance and punctuality. There were a number of few cases where teams arrived at the school and found out that the schools were closed or holding some event, and in those cases the fieldwork management team re-planned the school visits and teams were sent to the nearest school in the area that had not yet been visited.

 Rains posed some logistical challenges as roads and bridges were disrupted. OPM was well prepared to minimize the impact. As was done at midline, the Ngorogoro team was dispatched early from the central field start location to be as far ahead of the rains as possible and reduce their exposure to transport issues arising in the district during the rainy season. All supervisors were instructed to check for potential access issues to schools the day prior to visiting. In some instances the road to a school was not accessible by car and OPM had to transfer additional money to the teams to hire local transport such as motorcycles.

 In a few cases, teams arrived late at the school, due to either transport issues arising from the rain or to rescheduling due to some events at a school. This resulted in teams missing the headcount observation of teachers in the morning. A couple of these cases happened during the first week of fieldwork, and in those cases, a Kiswahili speaking member of the fieldwork management team was sent to those schools on another day to conduct the headcount observation of teachers. Additionally, during the visit to the field in mid-implementation, a member of the fieldwork management team visited one other school to conduct a missing headcount.

 In certain areas, particularly Ngorongoro and Simiyu, there are many non-Kiswahili speakers. This can make it difficult to interview parents. Some teams at endline used teachers who knew the vernacular language to translate for the scorecard interview with parents. It was emphasized with field teams that pupil tests are always administered in Kiswahili

 Internet and phone network coverage in some areas especially in Ngorongoro district was an issue. This caused teams in these remote areas to delay sending data on time to the data manager. In some very remote areas it also caused some difficulty in reaching the head teachers, and some absent school teachers for the missed teacher interviews that were planned to be done over the phone.

 A very high number of head teachers (20%) were absent on the day of the survey. In those cases, the assistant head teacher was interviewed instead or if not available then the academic master or a teacher who is familiar with school records. All modules that could only have been answered by head teachers were not asked to the alternative respondents, and at the end of the fieldwork almost all the head teachers that were absent were called over the phone to administer the remaining missing modules.

 Two of the 50 fieldworkers selected at the end of the training dropped out before the pilot in Dodoma due to illness and sudden unavailability. Additionally, two weeks after fieldwork started one interviewer dropped out but fortunately one of the two interviewers who had dropped out before the pilot was available for replacement which did not impact on the data collection timeline.