SkYE Programme - Quality Assurance Revisited by Mott MacDonald International Development

SkYE Talk: Issues Unpacked Quality Assurance Revisited: how can we make it responsive, as well as robust? Issue 02

Introduction The central purpose of the SkYE Caribbean programme is that disadvantaged youth in the Eastern Caribbean access vocational and technical training leading to recognised certificates and employment. However, challenges in the Technical and Vocational Education and Training (TVET) Quality Assurance systems in the four countries where SkYE works have significantly affected when graduates receive their certificates. Long delays in certification mean that graduates enter the job market without proof of the competences they achieved during training, a core purpose of competency-based qualifications. To understand these challenges, SkYE commissioned a review by two regional experts of the TVET Quality Assurance systems in St Vincent and the Grenadines (SVG), Saint Lucia, Dominica and Grenada.¹ In this issue of SkYE Talk we present key findings from the Quality Assurance review and consider how factors identified by the reviewers contribute to the delays in graduates receiving their certificates. We find that processes intended to protect the quality of certificates are undermining another vital purpose of vocational certificates, i.e. to signal to employers that graduates have gained the skills and competence that are needed for work. We end with a few high priority recommendations. 1

The full report of the TVET Quality Assurance Review by Orlando Hewitt and Paul Creese is available at www.skyecaribbean.com

Certification by National Training Authorities in the Eastern Caribbean The TVET systems in the four SkYE countries differ, but they share a common governance structure with a national apex body (National Training Authority or TVET Council) that has been established by relevant legislation, guaranteeing legitimacy and stability. All four of the TVET apex bodies are members of the Caribbean Association of National Training Authorities (CANTA) and are implementing the CARICOM² regional qualifications framework. The agencies in Saint Lucia, Grenada and St Vincent are accredited to offer Caribbean Vocational Qualifications (CVQs) at five levels of certification. Only Dominica is not yet accredited to offer CVQs. All four NTAs also have a national mandate to award National Vocational Qualifications (NVQs), although the Dominica TVET Council’s capacity to deliver this mandate is affected when the NTA Board is not in place. CVQs and NVQs are based on occupational standards, they rely on competency-based training methods used by instructors and assessors, and on verification of assessment by internal and external verifiers to maintain consistency in standards. The review identified both strengths and challenges within the Quality Assurance systems. Here we discuss four challenges that are undermining the role of qualifications in signalling a young person’s competence to employers through extended delays in certification.

Dominica

Saint Lucia

St Vincent and the Grenadines

Dominica TVET Council

Grenada National Training Agency

Saint Lucia Council for TVET

Sector Skills Development Agency (SSDA) / National Qualifications Department

NVQs only

CVQs & NVQs

Independent Assessor Model

InstructorAssessor Model

Grenada

External assessor submits assessment evidence to the NTA

Caribbean Community (CARICOM)

Internal Verifier checks assessment process conducted by instructor-assessor. External Verifier submits assessment evidence to the NTA

CHALLENGE 1 Funding The NTAs face significant financial resource constraints and these inevitably impact their operations. In the case of the NTAs in Grenada and Saint Lucia, the annual government subventions are woefully inadequate with the result that the organisations rely on assessment and certification fees, and external grants to finance their operations. The NTAs’ over-reliance on income from assessment and certification is driving up the unit costs of training because providers must pass assessment costs on to trainees, with the result that certificated training may not be affordable for young people from disadvantaged communities. The lack of funding impacts considerably on human resourcing for NTAs and thus on their operational capacity. With few staff, NTAs are constrained in the volume of assessment evidence that can be processed, which impacts the pace of certification. Governments in the four islands need creative solutions to resource external certification within their TVET systems. The agency in St Vincent works closely with the Ministry of Education, and Ministry staff are seconded to roles within the NTA. Across the globe, there are several examples of training funds financed by employer levies, donor financing and/or public subsidy, that are used to mobilise funding for the TVET system. A key factor in the success of Training Funds is the extent to which employers are involved in the governance of the fund, and that the system delivers the skills employers need. 3

CHALLENGE 2 Independent assessors and delayed assessment Among the three agencies awarding CVQs, the QA Review identified two distinct approaches to ensure good quality assessment.

1. Instructor-Assessor model

2. Independent Assessor model

Instructors carry out assessment to determine learners' competencies. In this model, instructors are trained and qualified as assessors, and formal assessments of competence can take place both during training, as well as at the end of training. The quality of assessment is moderated internally by an institutionally-based internal verifier.

Assessment is carried out by independent assessors who are not involved in training delivery. External to the training provider, they are contracted to plan, conduct, and review the assessment activities. This means that most formal assessment of competence takes place at the end of training. The argument for this approach is that the use of independent assessors removes bias from the assessment process. In this model, the role of the institutionally-based internal verifier is limited to advising instructors on competency standards and on how to determine when trainees are ready for external assessment.

While both approaches are consistent with established good practice for competency-based assessment through funding certificated training across all three countries that award CVQs, SkYE has found that the instructor-assessor model of quality assurance is associated with certification that is both more affordable and efficient. The model is associated with faster certification (within 4 months of the completion of training), and lower costs charged for certification. 4

CHALLENGE 3 Independent assessors and delayed external verification In both models of assessment, external verification of assessment is carried out by an NTA appointed External Verifier. The External Verifier focuses on sampling assessments including, records of evidence and verifying assessment decisions by checking that assessment and internal verification procedures are conducted reliably. The evidence required for external verification is submitted to the NTA by the training provider’s Internal Verifier (Model 1) or by the Assessor (Model 2) for every cohort of candidates submitted for certification. However, extended delays in the submission of assessments to the External Verifier are primarily associated with assessments conducted by independent assessors, further delaying certification.

CHALLENGE 4 Layers of committees involving staff and sector representatives Claims for certification are evaluated by a Certification Review Committee which is established and managed by the NTA. The Committee reviews the reports provided by the External Verifier and submits recommendations for certification to the NTA Board. Although the Committee evaluation is critical to the CVQ certification process, delays in the review process are a common cause of late certification. In Grenada, Saint Lucia and SVG, the Review Committees are usually made up of a combination of NTA staff and sector representative(s), and so delays reflect engaging the sector representative(s) and the already heavy workload of staff. Progress may be stalled again if there is a protracted wait for sign off on the Review Committee Reports by the NTA Board. The NTA Board is made up of stakeholder representatives including Ministries and employer groups (Chamber of industry & commerce, employer federation etc.), trade unions and training provider representatives, which can contribute to inconsistency in holding Board meetings particularly when political appointments need to be changed and at the expiration of boards.

Recommendations

The current situation, where graduates are waiting for certificates – sometimes for more than a year – undermines the usefulness of qualifications to employers and graduates entering the labour market, and therefore risks damaging their credibility over the longer term. It is a situation that needs to be addressed in the interests of the certification system itself. It is incumbent on TVET apex bodies to engage in continuous improvement to improve their efficiency, cost-effectiveness and to address the needs of their key stakeholders - the training providers, graduates and employers. Significant steps have been made in establishing quality assurance mechanisms that protect the reliability of qualifications. However, the review identified challenges that are contributing to extensive delays in certification and need to be addressed urgently.

• There is a dire need to adequately fund the NTAs so that they have the required operational capacity. • Training providers need to be empowered to deliver robust assessment themselves, through training instructors as assessors (Instructor-assessors) and internal verifiers so that system capacity matches the quality assurance demands more efficiently. • The bureaucratic requirements of external validation and approval need to fit better with the qualification’s purpose and level. • Governments may also wish to consider appointing NTA Boards that are able to remain quorate during periods of political change, enabling the certification system to continue unimpeded. 7

www.skyecaribbean.com

@SkYECaribbean

@SkYEintheCaribbean

@skyecaribbean

The Skills for Youth Employment (SkYE) Programme in the Caribbean is a 5-year programme funded by UK aid which aims to develop a more productive, better trained and inclusive workforce across four countries- St Vincent & the Grenadines, St Lucia, Dominica and Grenada. The objective of the programme is to develop relevant market skills among young people (aged 15-30) in the Eastern Caribbean to promote greater youth employment, greater productivity and greater inclusiveness in the labour market, including better training and employment access for disadvantaged youth and people with disabilities. The programme aims to improve the quality, reach and sustainability of technical and vocational education and training in the four focus countries.