Comprehensive Systemic Evaluation Sub-directorate
IMPACT ASSESSMENT STUDY Abridged Report
2013
What is this Impact Assessment Study about? An important aspect of improving education is to monitor progress, and to take appropriate action based on clear, valid and reliable evidence. A range of national, regional and international studies* have consistently shown that South African learners are not achieving the standards expected, particularly in Language and Mathematics. Initiatives such as the Systemic Evaluations of the Foundation and Intermediate Phases, the Quality Improvement, Development, Support and Upliftment Programme (QIDS-UP) - which had both evaluation and resourcing components — and the Annual National Assessments have provided valuable information on learner achievement levels. These studies have informed government’s Action Plan to 2014: Towards the Realisation of Schooling 2025. We need to consider whether the findings and recommendations made as a result of various studies that have monitored education outcomes have had any significant impact on identified short-comings and challenges in the system. This Impact Assessment Study seeks to find answers to this question.
How was this Impact Assessment conducted? The Comprehensive Systemic Evaluation sub-directorate in the Quality Promotion and Standards Directorate designed and managed the Impact Assessment.
Respondents from a sample of 190 schools in 22 districts were interviewed by district officials, and completed a comprehensive questionnaire. Schools that had benefited from QIDS-UP resourcing and base-line assessments of learner achievements were selected to take part in the study. Respondents provided information on: 1. 2.
Learner Literacy and Numeracy achievements for Grade 1 to Grade 3, from 2008 to 2012, from both school-based annual assessments and the Annual National Assessments. Respondent views on the effect on learning and teaching of LTSM, equipment, reports and teacher support provided by QIDS-UP the effect on learning and teaching of the Systemic Evaluations, QIDS-UP, and ANAs reports, recommendations, and associated teacher development and school support.
Limitations The data that informs this study was collected through interviews with school staff, and thus relies on the opinions and memories of the respondents. Although district officials who conducted the interviews may have requested documentary evidence, only the completed questionnaires were available for analysis. In addition, data on the school-based assessments and Annual National Assessments were transcribed by hand from school records onto the Questionnaire, and from there entered into to the electronic data base. Although every effort was made to ensure accuracy, the nature of this process may allow inaccuracies to slip in.
* Regional and international studies include the Southern African Consortium for Measuring Educational Quality (SACMEQ) studies in 2000 and 2007, the international Trends in Mathematics and Science Studies (TIMSS) of 1995, 1999, 2002 and 2011, and the Progress in International Reading Literacy studies in 2006 and 2011.
What does the Impact Assessment show? Most respondents said the QIDS-UP resources, the various reports and recommendations for improving results (particularly the ANAs reports), associated teacher development and school support had contributed positively to improved learning and teaching.
Literacy and Numeracy achievements The sampled schools (‘QIDS-UP schools’) reported on both school-based Literacy and Numeracy achievements, as well as their Annual National Assessment results. The graphs below show that these schools reported consistently high school-based learner achievement over five years.
Annual National Assessment results reported by the sample schools show that learner performance at Grade 3 level was considerably better than either the provincial or national level.
Although the sample schools reported high learner achievement, this study is not able to attribute this directly to the various strategies undertaken to improve teaching and learning. Further research will need to be undertaken to demonstrate a clear causative link.
ANA and QIDS UP are contributing positively to schools. There is a great improvement on learner performance and educators are developed through some of their initiatives. Respondent comment
Provision of QIDS-UP resources and equipment
Effect of reports, recommendations, and associated teacher development
QIDS-UP targeted ‘schools serving poor communities where quality of education is often compromised due to lack of basic minimum resources, overcrowding and shortage of skilled personnel for effective learning and teaching’ (Department of Education, 2007).
61% of respondents said that the study reports (particularly the ANA reports) had engendered teacher development activities at various levels, including within the school. Respondents were mainly positive about the effects of teacher development on the achievements of learners in both language and mathematics, and could cite examples.
85% of respondents said resources provided by QIDSUP had contributed positively to improved learner skills. 70% of respondents said that the resources encouraged independent learning, while 18% said they encouraged independent learning ‘a little’. 79% of respondents said that learners benefited from teachers’ familiarity with resources. 69% of respondents said that the quantity of resources had been insufficient for all learners. The main strategy used to deal with shortages was sharing amongst learners and teachers, and photocopying.
69% of respondents who answered the relevant question rated learner achievement as ‘a lot’ or ‘quite a lot’ as a result of teacher development initiatives. 74% of all respondents (or 95% of those who answered this question) said their school had taken positive steps to improve performance as a result of the reports received. Many respondents mentioned school and subject improvement plans. A strong theme that arose was the need to encourage reading, particularly through the Drop All and Read campaign and the provision of classroom libraries.
Challenges Respondents cited large classes, multigrade teaching, insufficient resources, community socio-economic issues, including lack of parental support and low literacy levels, learner characteristics (especially ‘slowness’), and to a lesser extent, language issues as hindrances to improved learning and teaching.
Copies of the full Comprehensive Systemic Evaluation Sub-directorate Impact Assessment Study are available from: Quality Promotion and Standards Directorate Province of the Eastern Cape Department of Education Steve Vukile Tshwete Complex Zone 6, Zwelitsha, King William’s Town Eastern Cape, South Africa Telephone: Email:
040 6084606 Nozuko.Ngudle-Yafele@edu.ecprov.gov.za or nngudle9@gmail.com