2 1 E D A R G N O S N O Y I T T I C L A U Q D REFLE N A S D S R E A S D S N E C O STA R P E C N A R U ASS K C O O L ERT O C . F PRO
TRANSITION FROM NATED 550 TO A NEW NSC: How do they compare? Since the 2008/9 NCS examinations Grade-12 examinations and assessment were conducted in a similar way to the previous SCexamination, although the resulting process was executed by a new ITplatform, called the “Integrated Examination Computer System” (IECS). :
Research related to the comparability of curriculum and assessment demands of the NCS (2008) and the SC (prior to 2008) Click icon to add picture
The most important assumptions underpinning the research was first of all that the demands of the knowledge and skills domain of each of the curricula was comparable , so the NCS were initially a ‘blend� of the previous HG and SG
THE PROCESS OF ENSURING COMPARIBILITY
It was argued that the new NCS exam would enable learners to achieve at least at a level of the previous SG, while the exam papers would contain sufficient difficult items, so that learners can achieve at the levels equivalent to the A-grades, achieved by the previous cohorts of HGlearners. Access to tertiary institutions would directly relate to the ability of the learners to meet the cognitive levels of difficulty required for achieving at the top of the performance range.
Secondly it was assumed that a comparison of the analysis of the 2008 NCS exemplar examinations and old SC exams would give an indication of the level of difficulty of these exemplars in relation to the previous HG and SG versions of the exams. This was done in all the so-called “Gateway subjects� The main finding based on the analysis of the exemplar examinations was that in general learners who would have achieved A-Grades in the SC-HG exams will achieve A-grades in the new NCS-exams, where the A-grades were comparable to the SC-HG exams.
Further to this the old SC (NATED-550) system also made provision for condonement to a pass on the lower grade (LG) if a learner performed between 35 and 25%. Considering the research findings, Umalusi judged the use of constructed and hybrid norms to be correct in the light of the patterns emerging from triangulation of findings from all sources considered during the 2008 standardisation process.
Note: With the widespread use of an index of overall achievement as the basis for offering university places, has come the role of statistical moderation in ensuring that results of different assessments over time and across different cohorts are on a “common scale” before being combined with a tertiary ranking scale”. Based on the philosophy that ‘better” education adheres to a required standard which can be measured, and reflected in easily quantifiable results, and the willingness to reshape all intervening processes, seem to characterize the path to success in both education, business and higher education, rather than an artificial “raising of the bar”.
THE PASS RATE OF 40% IN THREE SUBJECTS AND 30% IN ANY THREE OF THE REMAINDER. The issue is not as simplistic as it may appear. It requires a very fine balancing act between national interest and the benefit to industry, trade and the higher education sector. It is what the pass rate represents that is important and that in itself is linked to standards – one simply needs to consider them together. Raising the pass rate without an equal raise in standards will be meaningless. It will not produce a better quality of learner to HEI’s and training institutions if it boils down to a numerical adjustment only
IMPROVEMENT IN 2012/13 RESULTS: WHY? A Careful analysis in why the 2012/2013 results have improved can mainly be attributed to the fact that previously disadvantaged schools are responding positively to the national and provincial intervention programs, such as the SSIP. This is very evident in the table below, however, one of the main systemic problems in my opinion relating to the pass rate is not the 30% required average but the value of Life Orientation.
105 102 81 145 146 61 94 97 141 100 38 71 127 87 139
59.32 56.04 48.50 73.60 65.18 73.49 80.34 70.29 81.98 68.49 97.44 66.98 77.44 57.24 77.22
193 214 129 178 171 120 170 183 119 210 69 124 213 169 242
190 206 128 172 170 115 168 183 115 209 68 118 211 167 236
2013-Achieved%
177 182 167 197 224 83 117 138 172 146 39 106 164 152 180
2013-Achieved
183 189 172 206 227 86 118 141 181 154 39 108 171 158 184
2013-Wrote
50.27 79.82 64.88 59.72 66.86 79.17 68.12 91.46 88.99 87.13 87.72 85.37 44.77 67.74 59.12
2013-Entered
93 87 109 126 115 76 94 150 97 88 50 35 77 84 107
2012-Achieved%
2012-Achieved
185 109 168 211 172 96 138 164 109 101 57 41 172 124 181
2012-Wrote
195 111 171 223 175 98 143 164 114 105 57 42 175 129 184
2012-Entered
PU PU PU PU PU PU PU PU PU PU PU PU PU PU PU
2011-achieved%
N N N N N N N N N N N N N N N
2011-Achieved
Y Y Y Y N Y Y Y Y N N Y Y Y Y
2011-Wrote
Centre name
SSIP No fee Technical School School Sector 2011-Entered
Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y
138 138 88 127 118 80 116 147 97 160 59 98 161 115 204
72.63 66.99 68.75 73.84 69.41 69.57 69.05 80.33 84.35 76.56 86.76 83.05 76.30 68.86 86.44
The value of Life Orientation (Amendment of Regulation 3 Government Gazette 35896): Because the school-based assessment and resulting Life Orientation marks provided by schools are noticeably higher than those marks achieved in the other Grade-12 subjects, it contributed to the public’s mistrust in the validity of the school-based assessment practices. Unfortunately this is true: The pass requirement for obtaining a NSC is HL = 40%; 3 subject = 40%; 3 subjects = 30%. Because of the uncommonly high performance in LO 1 of 3 subjects is virtually a given However, this subject is of little value for HEI’s and are aimed at the development of citizenship. It does not belong to the regime of academic subjects and should not be considered for promotion.
THE ROLE OF UMALUSI: Quality assurance, fairness, validity
In terms of the GENFETQA Act of 2001(as amended in 2008), Umalusi is required to approve the release of results once it is satisfied that the examinations have been conducted in a credible manner. The Act also indicates that Umalusi is permitted to adjust the raw marks when necessary. Gaining the approval of the Umalusi Council for the release of the results is a complex process, of which standardization is one of the last steps. Let us consider this as it benchmarks the credibility of the exam results.
Approval of results is determined by how well the assessment bodies responsible for the examinations have complied with all the policies, directives and guidelines related with the qualification being resulted. These policies and directives are issued by Umalusi and the Departments of Basic Education and Higher Education and Training, and also by the Independent Examinations Board (IEB) and SINCE 2013 THE South African Comprehensive Assessment Institution (SACAI) for their own systems. Such approval is also dependent on whether any irregularities could be considered to have undermined the credibility and integrity of the examinations.
STANDARDISATION Standardisation is a process used the world over to mitigate the effect of
factors other than the learners’ knowledge and aptitude on the learners’ performance. In South Africa, the standardisation of results has been used since 1918 by Umalusi’s predecessors, the Joint Matriculation Board (JMB) (1918-1992) and South African Certification Council (SAFCERT) (1992-2001). In other words, all South Africans who have obtained their qualifications through these bodies have had their subjects standardised prior to their results being announced. A large-scale examination system such as the National Senior Certificate (NSC) inevitably experiences many sources of variability, despite the best efforts of examiners, markers and moderators. These can include changes in levels of difficulty in question papers across years
To prevent that cohort from being unduly disadvantaged, their marks may be adjusted marginally towards the historical average. Variability may also result when errors occur in papers, and inconsistency may arise in the marking, for example, across provinces. Standardisation is the accepted process used to reduce fluctuations in learner performance that result from identified factors within the examination processes themselves rather than from the knowledge, aptitude and abilities of the learners.
It is important, however, to recognize that many quality assurance efforts precede standardisation. These include the moderation of examination papers, and where relevant, the moderation of internal assessment components which contribute to learners’ final marks; the monitoring of the conduct and administration of examinations; research into the quality and comparability of the papers across years, as well as the marking and moderation of marking. However, it is only at standardisation that the global picture of the performance of all candidates, say for 2014, in each of the subjects can be considered.
In order to guide the final standardisation meetings with each of the assessment bodies - the Department of Basic Education, the Department of Higher Education and Training, SACAI and the IEB - the Assessment Standards Committee holds pre-standardisation meetings. The Committee considers the following:: Detailed analyses of the question papers in terms of their compliance with prescribed subject assessment guidelines (SAGs); Detailed analyses of the question papers in terms of their cognitive levels; Comparative analyses of the current year’s question papers in relation to the question papers of the previous years; Input from internal moderators and chief markers Input by Umalusi’s evaluators and external moderators, as well as verification of the quality of the marking.
The Committee then deliberates on the preliminary results submitted by the relevant assessment body, and, in the light of all reports received, decides upon a course of action—to accept the raw marks (always the preferred option as it means that the other moderation processes have functioned well), or to consider an adjustment to the marks.