The positive effects of using e-learning resources on the success rates and progression of learners in Functional Skills A report by Katie Martin BA (Hons), University of Sunderland
The positive effects of using e-learning resources on the success rates and progression of learners in Functional Skills. By Katie Martin BA (Hons). Copyright Š 2015 Guroo Ltd. All rights reserved. First edition printed 2015 in the United Kingdom. A catalogue record for this publication is available from the British Library. ISBN 978-1-909589-20-9. No part of this publication shall be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information retrieval system without written permission of the publisher. Published, Designed and Set by Guroo Ltd. www.guroo.info. Printed in Great Britain.
Foreword – Maureen Emmett, Former Senior Policy Advisor, Qualifications & Curriculum Agency (QCA) and Functional Skills implementation and policy lead – now Company Director, EM Skills
Having been involved in the design and delivery of Functional Skills since 2007, I’m always interested in research that helps the sector to understand how learners can be supported to improve their English, maths and ICT skills. This report clearly sets out to aid that understanding and add to the evidence base. My work, as part of the strategic team at QCA that developed and piloted Functional Skills, recognised the importance of good teaching and learning resources that could support alternative approaches to delivery, to reflect the differing needs of learners. E-learning is a key component of a blended approach to the teaching of Functional Skills and its increased use has enabled more and more adults and young people to access and achieve the qualifications. The report, although limited in scope, clearly illustrates the link between e-learning and learner success and will help managers and practitioners to design a curriculum that recognises how to develop the skills of young people and adults as well as ensuring they achieve these worthwhile qualifications. More work needs to be done in the sector so that we can fully understand the factors that ensure learner success and I look forward to seeing the next report using data from a wider spread of learners.
The positive effects of using e-learning resources on the success rates and progression of learners in Functional Skills
Page 2
Executive Summary: Study based on independent data provided by 7 different training providers with a sample size of 177 learners.
There is a statistically significant link between the use of e-learning resources and improvement in success rates in Functional Skills. • •
15 hours of e-learning use results in an average improvement in success rate of 9%. 5 hours of e-learning use results in an average improvement in success rate of 6%.
There is a statistically significant link between the use of the e-learning resources and positive learning progression in Functional Skills. Positive progression was recorded by 80% of learners who used e-learning compared to 47% of learners from the control group who undertook an “assessment only”.
The positive effects of using e-learning resources on the success rates and progression of learners in Functional Skills
Page 3
Introduction Since their introduction in 2007, Functional skills have become an integral part of the vocational education system. In the year 2013/14 more than 1 million Functional Skills awards were made, making Functional Skills the most popular qualification after GCSE. Functional Skills are typically taken by learners in Apprenticeships programmes, adults and post-16 learners who either haven’t achieved or have little chance of achieving grade C or above at GCSE. Almost all post-16 learners starting a Functional Skills course will take an initial assessment and skills diagnostic. This gives the training providers and FE colleges an accurate picture of the capability of learners, and crucially for this report, it gives a standardised and reliable start point from which to measure progression towards achievement. The vast majority of learners (more than 90%) will use initial assessment supplied by either ForSkills/Guroo or BKSB. The data in this report is based on data from learners and training providers who use ForSkills/Guroo initial assessment and e-learning resources. Seven FE Colleges and Private Training Companies provided outcome result data in Functional Skills for just under 200 learners. ForSkills/Guroo provided initial assessment data and e-learning use data on the same learners.
Methodology A range of training providers including FE colleges and private training companies were identified and contacted. For each training provider approximately 40-50 learners were identified, up to 30 who had only completed the initial assessment and 20 who had also used e-learning resources provided by ForSkills/Guroo. The data provided by ForSkills/Guroo was initial assessment and e-learning data and included: • • •
The initial assessment result in each subject The date of the initial assessment For the 20 learners who had used e-learning resources, the number of tasks and by calculation, the number of hours of e-learning completed
The data provided by training providers was the outcome data: • •
The outcome (pass/fail) of the external Functional Skills examination Subject, level and the date of the examination.
7 training providers responded with outcome data. Learners who had left without completing their course or those still studying were excluded which meant our sample covers 114 learners who had used the ForSkills/Guroo initial assessment only as the control group and a further 63 learners who had used the ForSkills/Guroo system for e-learning in addition to the initial assessment as the subject group.
The positive effects of using e-learning resources on the success rates and progression of learners in Functional Skills
Page 4
Median use of e-learning was approximately 15 hours for the subject group between initial assessment and final external assessment. The graph below shows the distribution of use of the 63 learners in the subject group.
Number of Learners 120 100 80 60 40 20 0 None
1-9
10-20
20+
E-learning hours completed For each group of learners, the outcome results of the external assessments that that student took were recorded and the sum total of passes and fails calculated for each group. A success in an external Functional Skills exam is simply whether that student passed at the level for which they were entered. The data was analysed using a t-test to establish its significance and the Pearson test to establish correlation1.
1
The t-test assesses whether the means of two groups are statistically different from each other. Pearson's chi2 squared test (χ ) is a statistical test applied to sets of categorical data to evaluate how likely it is that any observed difference between the sets arose by chance. It is suitable for unpaired data from large samples.
The positive effects of using e-learning resources on the success rates and progression of learners in Functional Skills
Page 5
Success rates of learners Success in this context means a learner passing the external assessment/exam for which they were entered. Students can be entered into maths, English, ICT or a mixture and are typically entered at the level indicated by the initial assessment. The success rates measured as Functional Skills passes for each group of learners are shown in the table and line graph below. E-learning Hours Completed Functional Skills Success Rate Entries Pass Fail
None 83.0% 159 132 27
1-9 89.7% 29 26 3
10-20 92.3% 39 36 3
20+ 98.0% 49 48 1
Success Rate 100.0%
98.0%
95.0%
92.3% 89.7%
90.0%
85.0%
83.0%
80.0%
75.0% None
1-9
10-20
20+
e-learning use in hours
Taking the median e-learning use of 15 hours, the success rate improves from 83% amongst the group of students who recorded no e-learning use to 92% - an increase of 9%. The average time between initial assessment and taking the external assessment was 8 months.
The positive effects of using e-learning resources on the success rates and progression of learners in Functional Skills
Page 6
Progression of learners The data collected also allowed analysis of progression measured from initial assessment to outcome result and the time taken. The mean time from initial assessment to result was just over 8 months. For both the control group (no e-learning use) and subject group (median e-learning use of 15 hours) there were three possible outcomes when looking at progress, they either made progress, maintained the level obtained in their initial assessment or they decreased – made negative progress dropping a level or levels. Within the maintaining category will be some learners who record “working at Level 2� in their initial assessment which means the best they can do in the outcome result is maintain. The column graph below shows it is overwhelmingly apparent of the subject group that had used e-learning resources 80% made progress compared to 47.2% for the control group. Learners who had made negative progress in the control group were almost double the rate at 16% compared to 8.3% for the learners in the subject group using resources.
Percentage of progress rates 90 80 70 60 50
Progress Maintained
40
Decreased 30 20 10 0 Resources (Subject Group)
No Resources (Control Group)
The positive effects of using e-learning resources on the success rates and progression of learners in Functional Skills
Page 7
Conclusion Although each person learns differently and one method of learning will never work for everyone, e-learning does provide a flexible approach and, for those learners who engage and embrace e-learning, this report shows a measureable and positive impact on outcomes. Although the data sets for the control group (114 learners) and the subject group (63 learners) are relatively small, the results show there is a clear link between use of e-learning and both progression and outcome success rates. More information may be obtained from Richenda Wood of Livewire: Richenda Wood, Managing Director, Livewire Public Relations,
DDI : +44 (0)20 8339 7442, Tel: +44 (0)20 8339 7440, M: +44 (0)7941 125580 E: richenda.wood@livewirepr.com
The Researcher – Katie Martin This research was conducted by Katie Martin, BA (Hons) Social Sciences, University of Sunderland. It was researched and completed during the period January - March 2015. Prior to undertaking this research, Katie spent 5 years as a teaching assistant working mainly with lower ability students in Hartlepool. During this time, she saw that technology can be used as a “great leveller” delivering greater equality of opportunity to students with lower academic ability by engaging them outside the traditional classroom environment. Katie enjoys a challenge and believes that everyone has something they are good at!
The positive effects of using e-learning resources on the success rates and progression of learners in Functional Skills
Page 8
References and sample data Garrison, D. R. (2011). E-learning in the 21st century: A framework for research and practice. Taylor & Francis. The data below is an anonymised sample of the data of two learners from different training providers.
The positive effects of using e-learning resources on the success rates and progression of learners in Functional Skills
Page 9