Section 3: The Research Methodology for Data Collection and Management

This research study builds upon existing evidence and data in the field of ECCE, with focus on the implementation of the 2007 education sector ECD Policy. The study seeks to provide rich verifiable primary data on present ECCE delivery modes and types (in ECD Centres), which is set within regional and international literature on effective ECCE implementation. The study has taken a deep, broad and hybrid approach to delivering evidence that will underpin the efforts to develop a revised ECCE policy and associated guidelines, standards and costs.

The research study was structured around the five main types (or modes) of ECD centre delivery that are defined in the 2007 ECD Policy. Using a mixed method approach, a team of researchers gathered both quantitative data (using a questionnaire and site observation schedule) and qualitative data (from interviews and learning process observations) to gain a triangulated understanding of how each model of ECCE service provision delivers its programme. 143 ECD Centre sites were studied, catering for 0-3 year olds, 4-6 year olds and 6-8 year olds. The breadth of delivery possibilities was studied, including parenting programmes, home-based care and education, nurseries, kindergartens, and school-based sites, both private and public, and both urban and rural. The target respondents were the owners, managers, teaching practitioners and users of the ECCE sites, including parents, community members and management committee members. Therefore, the study profiled existing ECCE programmes for children up to the age of 8, according to type53, provisions and geographical location Furthermore, the survey provides information on cost, provider and delivery mode54 with the aim of assessing the relevance, effectiveness, sustainability and efficiency of these models of delivery.

The process commenced with a desk research of all the 2007 Education Sector policy related documents and guidelines to inform the scope of the study survey and the development of tools. The desktop research also informed the approach that the Project team undertook to provide analysis of existing ECCE models and practices. All key Government of Uganda, MoES and MGSLD documents dealing with ECCE were reviewed in the desktop review.

3.1 Development of Research Instruments

A rigorous approach was taken to the development of the survey instrument and the various interview tools, which would be used during the research process to gather data from national, district, sub-county and ECD Centre levels. The survey instrument and site observation tool underwent fifteen iterations as they were developed by the team of ECD experts, senior social researchers and statisticians, before eventually being loaded on the Samsung Android tablets by GIS Mapping specialists. Every question was tested against the requirements of the ECD Policy and its supporting guideline documents, as well as the NIECD Policy to ensure that the data collected would provide adequate explanation of how the various elements of the policies are being implemented at every level in the education system and with what level of success. The final check on the contents of the survey was conducted by the project management and lead social researcher, who together checked every question against the policy to ensure complete coverage.

Alongside this, the ECD specialists in the project team developed qualitative interview tools based on international best practice and to ensure that the data that would be collected would be adequate to respond fully to the research questions posed by the project terms of reference. These covered areas such as preparation and in-service training of ECCE teachers, qualifications of practicing teachers, accreditation processes for those qualifications, employment conditions of pre-primary teachers, and the costs associated with delivery of the policy and of training.

These tools were then reviewed by the project management team to ensure that they covered all relevant aspects of the policy, were phrased appropriately for Ugandan respondents, and that they were not too long or onerous. Most interviews eventually took about one hour, although some with key officials and focus groups took considerably longer, as did delivery of the survey questionnaire (averaging around 1 to 2 hours). Although this is not preferred practice the project team were well aware that the complexity of the topic, the range of knowledge that certain respondents would have and the lack of later opportunities to undertake follow-up research, all merited such extensive interviews.

3.2 Scope and Approach

The data collection at District, Sub-County and Local level was conducted using a mixed methods approach, culminating in investigation of:

(a) the current status of ECCE in 30 communities in 10 districts across Uganda;

(b) processes and practices in 143 ECD Centres;

(c) documentation on the skills, abilities and qualifications of Teachers and Caregivers;

(d) the nature and capacity of key training and support institutions;

(e) types of networks and associations available;

(f) costs involved in delivering and accessing ECCE services;

(g) compliance with government requirements; and

(h) challenges that the ECD Centres are facing.

3.3 Sampling Strategy

A purposive sample of ten districts was drawn with two from each of the (traditional four) regions of Uganda, namely the Central Region (including one with island communities), Western Region, Eastern Region, and Northern Region (of which one District was in Western Nile with provision for refugee communities), as well as an additional two districts (Kampala and one from among the Karamoja districts). These ten districts were selected to represent a cross-section of Ugandan districts.

The process of selecting the ten districts was undertaken based on the following process. The potential pool of districts for selection included:

• All districts which were included in the Uganda National Early Childhood Development Service Provider Inventory (2016), which were stratified by region (based on the traditional four regions) and then by intensity of ECCE service provision extrapolated from data sources (including districts with high, medium and low intensity of documented ECCE provision).

• The provisionally selected districts were cross-checking against other indicators such as wealth, including urban and rural districts, intensity of interventions and educational performance – to ensure a cross-section across these variants, as in Table 5 below.

Table 5: Criteria used to determine the District Sample: No. Criteria Reasoning

1. Ineachregion,

• AdistrictwithhighnumbersofECCEservices

• AdistrictwithfewECCEservices

2. GPE/QEI/MoESprioritydistrictsandotherCSO supporteddistricts

3. Districts with minority populations such as refugees,pastoralistandfishingcommunities

4. Ruralvsurbandistricts

5. NationalexaminationresultsinPrimaryLeaving Exam(PLE)andNationalAssessmentofProgressin Education(NAPE)

Tounderstandthedriversandbarrierstoaccessandquality ECCEservices.Regionaldifferencesareknowntobe importantdeterminantsofaccesstoECCE55

Drawlessonsfromongoinginterventionsintermsofwhatis workingandnotworkingsuchasGPECCCPinterventions,so astoinformECCEmodels

Documentandconsideruniqueneedsanddynamicsofuse specifictothesesubpopulations

Documentcontextualenablers,inhibitorsandresourcesthat impactECCEprovision

Documenthighperformingandlesswellperformingdistricts tohelpunderstandtherelationshipbetweenECCEandlater performance

The final selected list of Districts for the study was:

No. Region District

1 North Arua

2 North Oyam

3 East Tororo

4 East Buyende

5 West Kabale

6 West Bushenyi

7 Central Kalangala

8 Central Nakasongola

9 Central/Capital Kampala/Wakiso

10 North-East/Karamoja Moroto

Finally, the modes of service delivery for study were stratified according to those outlined in the Policy:

I. day care centres for children aged 0 - 3

II. home-based centres for children aged 0 - 3 and 3 - 6

III. community-based centres for children aged 3 - 6

IV. nursery schools/kindergartens for children aged 3 - 6

V. lower primary classes for children aged 6 - 8

3.4 Data Collection

Quantitative Data Collection: Quantitative data collection involved a survey conducted among ECD Centre Head teachers/Head caregivers using a highly structured questionnaire, administered by the team of 10 researchers, each attached to one district of Uganda. The structure of the questionnaire was based on the project terms of reference, with the indicators and variables obtained from the MoES’s 2007 ECD Policy and Guidelines for Early Childhood Development. The key themes (concepts and dimensions) that were included in the questionnaire are aligned to the interpretation of the terms of reference, presented in 12 sections of the questionnaire, namely: Characteristics of the Head Caregiver/Respondent; Characteristics of the ECD Centre; ECCE Service Provision and Practices; Compliance to Establishment Procedures for Establishing Day Care Centre; Enrolment, Dropout, and Attendance; Human Resource Capacity; Administration and Management; Curriculum; ECCE Centre Inspection; Collaboration and Networking with stakeholders; Access Related Issues; and Budgetary Issues.

Qualitative Data Collection: Qualitative interviews were implemented to explore the perspectives and experiences and feelings of key policy implementers and stakeholders. The qualitative data collection entailed 4 sub-activities: Desk-top reviews; In-depth interviews; Focus Group Discussions; and Site Observations.

3.5 Implementation of the Research Study

A letter of introduction and contact with District Education Officers were established by the Pre-primary Department of MoES to inform them about the survey, introduce Cambridge Education and ask for their participation in the survey. A pre-test of the tools was conducted with the intention to assess the relevance of the questions, adequacy of variables, skip patterns, establishing the duration of interview session, as well as testing the functionality and application of the tablets. The results of the pilot study were analysed and interpreted, some questions modified.

On arrival in each district, the researcher conducted introductory meetings with the Chief Administrative Officer (CAO) and the District Education Officer (DEO) to present the letter from the MoES, to seek their approval to commence the survey, and to gain assistance in setting up interviews with the CAO, DEO, Community Development Officer (CDO), District Inspector of Schools (DIS), and the District ECD Focal Point as well as Centre Coordinating Tutors (CCT). Semi-structured interviews, were then conducted. The main purpose of these interviews was to collect views and data on knowledge of the 2007 ECD policy, the implementation status of the policy and related guidelines especially in regard to practices and adherence to establishment and registration of ECD Centres, ECD budget allocation, support supervision and regulation, capacity building, and to ascertain roles and responsibilities in the implementation of the policy.

Then, in each district, with the assistance of the DEO and other district officials, three sub-counties were identified. Two of these needed to be predominantly rural sub-counties, while one should be an urban subcounty – generally the sub-county within which the district headquarters is situated. The researchers were tasked with identifying sub-counties which would have different ECCE provision and patterns of use. Following the selection, the researcher visited each sub-county in turn and working with the LC3 Chairperson and Counsellors identified up to five ECD sites representing each of the types or modes of service delivery (i – v) in a particular parish or locality. Where the LC3 Chairperson and Counsellors were unable to identify a particular type in the selected parish then the researcher was to replace that type with an extra example of the predominant type. By so doing the study would gain a rapid overview of the types which are under-developed and those which are common across the 10 districts.

Following the selection of the five ECD sites, each would be visited by the researcher, spending a full day at each site. Data was collected over the course of the day, including structured observations of the physical site, a questionnaire for the Head (or most senior member of staff present) of the ECD centre, and separate focus group interviews with Centre Management Committee members, teachers/caregivers and parents. Focus Group Discussions were conducted in order to obtain enlisted discussion, important points of departure, cross-checking of information provided by the Head, and eventually a joint construction of meaning or consensus among ECD policy stakeholder.

Structured Observation was also part of the ECD centre Surveys, implemented with the aim of establishing the levels of compliance to MoES procedures and Guidelines for Establishing ECD Centres, as well assessing the quality of caregiving among the ECD centres. The researcher would observe between 30 and 40 minutes of the learning process in a live session and assess the quality of the learning process using a carefully calibrated set of variables in an observation tool. A structured checklist was used to observe the social context and physical settings in the form of: 1) Buildings, 2) Staffing, 3) Sanitation and hygiene facilities, 4) Emergency facility, 5), Furniture, 6) Storage, 7) Meals, 8) Outdoor spaces, and 9) roles and responsibilities of the caregiver (during care/teaching session), and any other necessary facilities/equipment/play material (which are not mandatory but essential).

The questionnaire and site observation data were coded and entered onto Android Samsung Tablets that each researcher carried, with individual password-protected access. The interview scripts were analysed using analytical induction. Analytic induction is a research strategy aimed at systematically developing causal explanations for types of phenomena. Meanwhile, the learning process observations were also analysed by the early learning specialists in the team using an analytical framework that was developed for this purpose based on the MoES’s Early Learning Standards, local conditions and international experience. Collectively, this data was triangulated and supplemented with data from the interviews with sub-county, district and national officials; NGO/CBO providers involved in ECD; and ECCE teacher training providers.

At national level, semi-structured interviews were also conducted. These entailed talking to policy implementers and stakeholders to explore and grasp their point of view on ECD policy formulation, implementation, outcome, challenges, key lessons and new priority interventions.

The overview of stakeholders involved in the research and their geographic levels is depicted in Figure 2 in Section 2 above.

3.6 Management and Supervision of the Research

A total of 14 researchers were selected from applications from over 20 potential researchers. Each submitted their curricula vitae for review, assessing their existing research experience, knowledge of ECD, experience of using Android tablets in research, and language skills. A shortlist of 14 candidates was generated and invited for training. During the one-week training course, the 14 candidates were narrowed down to the 10 best, using inter-rater reliability tests across use of a number of the tools and further verification of language skills. Two researchers were then assigned to be deployed in each of the 4 regions, one per district, with Kampala metropolitan and Moroto North added.

The week-long training was conducted by the core management team and lead social researcher in June 2017. The training focused on understanding of the ECD sector and the types of service provision, survey and interview techniques, use of the tablets, and observation of learning events. The training was very practical involving structured micro-interviewing sessions both with and without the tablets, as well as use of a video of an ECCE class to practice using the learning event observation schedule. A training pack comprising 15 different instruments was compiled and distributed to the researchers. Finally, the research protocol was carefully explained so that all the researchers were clear as to what process they would follow in the districts and sub-counties, as well as at each ECD Centre, and what the expectations there were around confidentiality, their security and use and the security of the tablets.

A pilot pre-test of the research tools was conducted in two ECD Centres on 12 June 2017 in Kampala, leading to some adjustments in the interview tools and the tablets in response to the experiences and findings. The main changes related to rephrasing particular questions in the survey which were difficult for the respondents to understand, reordering questions in the survey which failed to flow, removing some repetition, and reprogramming the tablets where the skip logic was wrongly calibrated.

Each researcher was then deployed to their District, and provided with hands on support and supervision from the core project team and lead social researcher. Daily check-in processes were initiated, involving calls with each researcher to ask an agreed set of short questions, to resolve any challenges and to provide guidance on next steps. Each researcher called back in once they had selected their sub-counties and the ECD Centres to ensure that they were following the correct procedure and were selecting an adequate range of centres. Where a particular type of provision was missing in a sub-county this information was shared with the team. This system also allowed researchers to feed back any queries that they had concerning typology of a particular centre and selection of centres to be researched as the process they followed in selecting the ECD Centres at district and sub-county levels was complex and was likely to yield some unexpected results, which would require intervention from the core team to resolve. In addition, the Project Administrator kept a log on Excel of the number of ECD Centres that each researcher had visited and uploaded. This ensured that adequate numbers of centres were visited and that data was unloaded timeously. This system worked well and ensured that no researcher fell behind in collecting and uploading data. The tracking systems on the tablets were also used to verify the position of the researchers and to ensure that they were following their agreed schedule of visits.

As a further level of quality assurance and support the lead social researcher and a senior member of the core team visited each of the researchers in the field to review their output and process during the first two weeks of the data collection process. This picked up some minor anomalies of practice which were rapidly corrected.

During the 18 days of data collection in the field, 143 questionnaires were collected from 143 ECD Centres across Uganda in over 8% of the districts. None of the ECD Centres selected refused access and all were happy to cooperate, however, in Karamoja, three selected centres were not completed due to closure owing to termination of the World Food Programme. Four of the questionnaires collected were lost as they were being transferred from the tablets. In some cases the potential respondents had to be reassured that the name of their centre and their own names would not be shared with anyone and would not be specified in the report. This was because many of the centres are not registered and are not paying tax. They were also told that the data they would share was to be used to improve the policy and delivery of ECCE services in Uganda.

At the end of the 3 weeks of fieldwork the researchers were required to spend two days in the project’s Kampala office inputting the interviews they had yet to transcribe and sharing their experiences in a debriefing exercise. The Lead ECD Policy Specialist came from South Africa to lead some of these debriefing sessions so that she was able to get a first-hand feel of what the situation is in the ECD Centres and the experiences of the researchers.

3.7 Data Analysis

The data from the study was analysed to understand the key variables and those variables were crosstabulated to better understand the effectiveness and efficiencies of the various types of delivery. In addition, this data was compared with data that the MoES collects at district and national levels to better understand the strengths and weaknesses of existing data collection processes and draw out the gaps in these processes. The data loaded on the tablets was converted into Excel and exported to SPSS, and analysed to generate statistics - mainly descriptive (frequencies, percent, mean) and some inferential Inferential statistics was derived using Pearson’s Chi-Square tests to assess the relationship between key independent variables/predictor and dependent variable/outcome (rural/urban) and establishing the statistical significance based on the probability value (p=< 0.005), and drawing up conclusions about relations. Cross-tabulation between rural/urban, mode of service provision and other variables indicated no statistically significant relationship.

Overall a maximum of 15 sites was planned to be profiled in each district, making a potential total of 150 profiles, with up to 30 profiles of each mode of service delivery. Due to technological and Karamoja specific access challenges (see section 3.8), only 143 out of 150 planned sites were profiled. As highlighted in Section 2 on findings, there was an imbalance in the modes of service delivery visited, which is significant as the researchers were aiming to visit equal numbers of each type or mode of service provision in each sub-county they selected. However, as outlined above, if through consultation with the local councillors and LC3 they were unable to identify a particular mode or type of service provision in that sub-county, they were instructed to visit any other mode available in order to ensure overall coverage of 5 Centres per subcounty, or the 15 Centres per Researcher.

The site profiles were then analysed using correlational statistical tests to isolate and relate key variables. From this analysed data, a set of themes were developed based on the policy and the data used to better understand these themes, and the way that they play out in each type of ECD Centre. This allowed the team to understand how well the various ECD types care for and stimulate children, the staffing approach used, the infrastructure provided, and the costs associated with each type. This analysis and measurement of how variables correlate and function in existing modes of service delivery will help determine the proposed changes in the policy and the most effective types and approaches for use in the national scale up of ECCE programmes.

The data collected and analysed during the research process is summarised in Section 2 above, to highlight key information that can be used to inform the development of a revised Policy. Analytical induction approach was used to analyse the qualitative data collected through observation, semi-structured interviews, Focus Group Discussions. Analytical induction approach involved derivation of conclusions (abstract concepts) from the specific data (empirical indicators) under the framework of the study objectives. The emergent themes (regularities and commonalities) across ECD sites and stakeholder supported the formulation of conclusions (hypotheses).

3.8 Study Limitations:

Every research study has certain limitations, with this one being no exception. Here we describe the limitations of this study and where appropriate indicate the methods used mitigate them and limit negative impact.

Geographical Scope:

The Research Study was conducted in 10 out of 112 districts of Uganda. Although the selection of the ten districts was purposively drawn to be representative of a range of Criteria (see Table 5 above), it is acknowledged that the sample may be viewed as small and therefore not able to provide a complete picture of the country. However, given the sampling process we believe we can make assertions which will allow the findings to tentatively talk to the national situation. Certainly, the triangulation through focus groups and in-depth interviews with stakeholders at all levels and across different organisations, has helped to underpin the research findings, while indicating and verifying their veracity and relevance.

The broad geographical focus and the requirements for visiting deep rural sub-counties led to one researcher failing to complete all 15 ECD centre visits. This is because the researcher encountered many centres which were closed due to a food crisis in the area. This led to that district failing to produce the data from the full number of centres, but is an indication that Centres may struggle to operate due to both climatic/environmental conditions but also where feeding is threatened.

ECD Delivery Modes:

The study found that a majority of the existing ECD services fell under the Nursery/Kindergarten category. There were very few Day Care Centres and Home-Based Centres across the ten profiled districts and therefore most of the lessons and findings are drawn from Nursery/Kindergarten and Community-Based Centres. This is not a limitation of the study insofar as it highlights this finding, which is important for MoES’s consideration, but it is important to be aware that the lack of a depth of information through the study about Day Care and Home-Based Centres will make it hard to provide concrete recommendations for their improvement. Recommendations for these improvements will therefore be sought from other programmes in the East African Region and beyond.

Participation of Centre Management Committees:

The participation of Centre Management Committees was limited in some ECD centres due to distances to the schools and expectations for financial incentives. Although every effort was made to conduct interviews with CMC members, in some centres only a few members of the CMC were interviewed. This is in itself a finding.

Sub-county Selection Bias:

In each district, one of three sub-counties selected was an urban sub-county and two had to be rural. Therefore, the bulk of the study findings necessarily relate to rural contexts. Again, this is justified as Uganda is a predominantly rural country and previous data have tended to focus more on urban ECD Centres, with national understanding of the dynamics of ECCE delivery in rural areas being less well developed.

Selection of Researchers:

The researchers were selected on the basis of a number of criteria, of which their language capacity was key. This was because without having a shared language with the respondents the researchers would not have been able to conduct basic data collection with parents and CMC members. However, as ECD is a relatively new area of research, few of the researchers had experience of the sector. Few of the researchers were trained educators, so understanding of classroom dynamics for the learning event observation had to be developed during the training, as described above. On the other hand, this also means that they come to the research task without preconceived ideas about ECD, which could be a benefit.

Use of Education Technology and Loss of Data:

Quantitative data collection was planned and conducted using Samsung Android tablets to allow for uploading data in real time and to enhance the efficiency for ongoing data analysis. However, given the weak internet connectivity in the rural districts, and the length and complexity of the questionnaires and observation schedules loaded on the tablets, the use of technology potentially became a hindrance. Inconsistent electricity supply often caused delays and lack of strong internet connectivity meant that data was not readily uploaded onto the database. As a result, four completed questionnaires were lost and were irretrievable. More would have been lost but other researchers had followed instructions and maintained parallel hard copy records. In fact, having to keep parallel hard copies defeats the purpose of adopting technological strategies for data collection. Therefore, future research studies should consider the type of data being collected, the depth of the tools being used (and thus their relative size on a tablet), and the capacity in the areas being visited for upload. In addition, the training of those using technological tools must be rigorous in ensuring their complete confidence with the process and introduce fail-safe mechanisms to better guarantee data is not lost. Finally, the use of technology of this nature opens individuals up to security risks as devices may be attractive to thieves. While paper is usually a more laborious approach, for smaller sample sizes such as involved in this study, it also has significant advantages that may outweigh technological approaches.